Tech Deep-Dives

Blender + LLM Integration

The core of our technical innovation is connecting language models to Blender for automated scene generation. Here's how it works:

Architecture Overview

Scene Description → LLM → Blender Python Code → Render → Validation
        ↑                                                         │
        └─────────────── Feedback Loop ────────────────────────────┘

Key Components

1. Scene Parser

Converts natural language scene descriptions into structured data that the LLM can reason about. Handles locations, characters, props, lighting, and camera requirements.

2. Code Generator

The LLM generates Python code using Blender's bpy API. This code creates geometry, sets up cameras and lights, and configures render settings.

3. Headless Renderer

Blender runs without a GUI, executing generated scripts and producing renders. This enables batch processing and CI/CD integration.

4. Validation Loop

Renders are analyzed against the original intent. Discrepancies trigger corrective iterations until quality thresholds are met.

Read our full research post →

Proof of Concept Scripts

We've built two proof-of-concept scripts that demonstrate our approach:

poc_create_scene.py

Demonstrates basic scene creation from text descriptions. Capabilities:

• Parse scene requirements from structured input
• Generate Blender Python code for scene setup
• Create basic geometry and lighting
• Export renders in multiple formats

poc_validation_loop.py

Implements the feedback loop that checks renders against intent:

• Render current scene state
• Compare output against requirements
• Generate corrective instructions
• Iterate until quality threshold is met

Pipeline Architecture

Task Management

We use a task-based branching strategy where each unit of work gets its own branch. This allows parallel development and clean integration:

• task/5-storyboard-act-1
• task/6-website-production-blog-setup
• task/7-build-blender-automation-pipeline

Orchestration

Claude Code acts as our orchestration layer, managing task execution, coordinating between systems, and handling the feedback loops that make iteration possible.

Rendering Pipeline

Our rendering pipeline supports:

• Headless rendering for automation
• Multiple output formats (PNG, JPG, EXR)
• Configurable quality presets
• Batch processing for sequences

TaskYou: AI Orchestration

Our production is managed by TaskYou, a task orchestration system that coordinates Claude agents working on the film.

How It Works

Task Created → Git Worktree → Claude Agent → Pull Request → Human Review → Merge
       ↑                                                              │
       └───────────────── Remote Monitoring ──────────────────────────┘

Isolated Git Worktrees

Each task runs in its own isolated git worktree. This means multiple agents can work on different parts of the production simultaneously without conflicts. When an agent completes its work, the changes exist in a clean branch ready for review.

Tasks Become Pull Requests

Every completed task automatically generates a pull request. This gives us:

• Full visibility into what changed and why
• Code review workflow for quality assurance
• Easy rollback if something doesn't work
• Complete audit trail of production decisions

Remote Monitoring & Approval

We can monitor agent progress remotely and approve or reject their work from anywhere. The human-in-the-loop ensures that AI assistance doesn't mean AI autonomy — every significant change gets human review before becoming part of the production.

Benefits for Film Production

• Parallelization: Multiple scenes can be developed simultaneously
• Reproducibility: Every change is tracked and reversible
• Quality gates: Nothing ships without human approval
• Async collaboration: Work progresses even when humans aren't actively directing

Challenges & Open Questions

The Semantic Gap

Translating "make it look more dramatic" into specific Blender operations is non-trivial. We're building a vocabulary of operations that bridges natural language and technical parameters.

State Management

Blender scenes have complex state. Ensuring the LLM understands current scene state before modifications requires careful context management and potentially scene serialization.

Quality Assessment

How do you programmatically determine if a render is "good enough"? This is an active area of research. We're exploring vision models for automated quality checking.

Iteration Speed

Even simple renders take time. We're experimenting with preview renders, proxy geometry, and selective re-rendering to speed up the feedback loop.

Roadmap

What we're working on next:

1. Asset library integration: Connect to character and environment assets
2. Animation support: Extend beyond static scenes to animated sequences
3. Vision-based validation: Use multimodal models for quality checking
4. Batch processing: Generate multiple shots in parallel
5. Version control for scenes: Track scene changes like code

Follow the blog for updates as we build these systems.