Filmmaker & Studio User Guide
Step-by-step documentation for account onboarding, BYOAPI configuration, and the 9-stage virtual production pipeline.
Stage 1: Production Setup & Story Bible
Phase 1Create your film project, establish the visual canvas, and lock core cinematic rules.
Click 'New Production'
From the dashboard or top navigation, click 'New Production'. Enter your film Title, Slug, Logline, Target Runtime (e.g. 180s for a 3-minute short), and Genre.
Set Cinematic Format & Aspect Ratio
Choose between 9:16 Vertical (mobile cinematic / TikTok / Shorts) or 16:9 Widescreen. All downstream keyframes and video takes will automatically lock to this framing.
Configure the Story Bible
Establish visual rules in the Story Bible tab: color palette (e.g. cold neon noir, warm vintage 35mm), lighting style (chiaroscuro, practicals only), and narrative constraints.
Stage 2: Screenplay Breakdown & Scene Structure
Phase 2Ingest your script and decompose the narrative into sequences, scenes, and shots.
Import or Author Your Screenplay
In the Screenplay tab (/screenplay), paste your standard Fountain or industry screenplay text.
Execute Multi-Agent Scene Breakdown
Run the Script Breakdown agent (or use the MCP tool draft_screenplay). The agent parses sluglines (e.g. INT. CABIN KITCHEN - NIGHT), establishes dramatic beats, and tags characters present in each scene.
Verify Scene Deck
Inspect your scenes in the Scenes deck (/scenes) to review scene order, sluglines, estimated runtimes, and shot counts.
Stage 3: Digital Casting & Actor Likeness Anchors
Phase 3Design digital actors and lock photorealistic canonical face likenesses before shooting.
Create Character Profiles
Navigate to Characters (/productions/[id]/characters). Define each character's age, physical traits, costume, hair, and emotional demeanour.
Synthesize Close-Up Portrait Candidates
Click 'Generate Portrait' or use castTools. The AI generates 3 candidate 9:16 headshots conditioned on the character prompt.
Approve Canonical Likeness Reference
Review the portraits side by side. Click 'Approve as Canonical' on the winning portrait. This locks the actor's face embedding into the PuLID consistency pipeline.
Crucial Rule: Once a character's likeness is approved, all downstream keyframes and takes reference this exact image to guarantee the same actor appears throughout the entire film.
Stage 4: Location Scouting & Environment References
Phase 4Anchor physical world continuity and spatial environments.
Add Locations to Production Roster
Under Locations (/productions/[id]/locations), create entries for key sets (e.g. 'Old Timber Kitchen', 'Glacier Ridge').
Generate / Upload Location Reference Stills
Synthesize location reference images with consistent lighting, architectural materials, and props.
Lock Global Continuity Facts
Add critical scene-level facts (e.g. 'Green LED digital clock on back counter reads 2:13 AM'). These facts automatically inject into prompt adapters so AI models never hallucinate conflicting props.
Stage 5: Cinematography & Shot List Planning
Phase 5Direct optics, lenses, camera movement, and visual action for every shot.
Open the Shot Workbench
Click into any shot from the scene deck. The 6-section Shot Workbench opens.
Configure Section 2 • Camera Plan
Set Shot Size (e.g. Extreme Close-Up, Wide Establishing), Lens (e.g. 50mm Anamorphic Prime), Camera Height, Angle, and Movement (e.g. Slow Push-In, Static Tripod).
Configure Section 3 • Visual Plan
Define Subject, Action, Lighting, Environment, and Prompt text. The prompt adapter automatically checks length boundaries (<= 1000 chars) and injects continuity facts.
Approve Camera & Visual Plans
Click 'Approve Camera Plan' and 'Approve Visual Plan' to lock upstream specifications.
Stage 6: Keyframe Generation & Human Visual Approval
Phase 6Synthesize candidate stills, review them on the Scene Review page, and lock the approved visual frame.
Generate Keyframe Candidates
In Section 4 of the Shot Workbench (or via batch generate_keyframes), synthesize 3 candidate stills conditioned on character and location anchors.
Batch Review in Scene Review Page
Open the Scene Review page (/productions/[id]/scenes/[id]/review) and switch the toggle to [Pictures]. All shot keyframe candidates appear side by side.
Human Approval (Green APPROVE / Red REJECT)
Click the best candidate still to inspect in lightbox. Click green APPROVE. The approved keyframe is locked to R2 private storage and marked as the authoritative frame.
Stage 7: Video Take Synthesis (Runway Gen-4.5)
Phase 7Transform approved keyframe stills into motion video takes with Director Notes overrides.
Run Provider Preflight
The generation engine verifies the approved keyframe binary bytes in R2, verifies Runway developer API credentials, and validates duration (4s / 5s / 10s).
Dispatch Image-to-Video Generation
Click 'Generate Take' (or use MCP generate_takes). If active Director Notes exist (e.g. 'She stays still, mouth closed, no speaking'), they automatically OVERRIDE conflicting plan text.
Scene Review & Take Selection
On the Scene Review page (/review) in [Videos] mode, hover or tap each card for inline 9:16 playback. Click APPROVE on the winning take. If rejecting, enter a mandatory reason so the next generation corrects it.
Stage 8: Voice Design, Dialogue Takes & Automated Lip Sync
Phase 8Add human emotion, voice auditions, dialogue speech takes, and automated Fal lip syncing.
Build Dialogue Cues
Run build_dialogue_cues via MCP or Audio tools. Spoken lines from the script are linked to characters and tagged as 'visible_lipsync' (when face/mouth is on screen) or 'offscreen'.
Audition Character Voices with ElevenLabs
Use generate_voice_auditions to generate 3 candidate voice auditions per character. Review audio via /api/assets/[id]/file streaming, refine tone, and approve the winning voice.
Synthesize Dialogue Takes
Use generate_dialogue_takes to record spoken lines with emotional cadence and dramatic pacing.
Run Automated AI Lip Sync (fal-ai/sync-lipsync/v2)
For visible_lipsync shots, call generate_lipsync. The engine takes the cut-locked video take and approved audio, runs Fal's sync-lipsync v2, and locks the lip-synced candidate to the shot.
Stage 9: Timeline Assembly & Rough Cut Screening
Phase 9Cut-lock approved takes and screen your continuous rough cut.
Cut-Lock Approved Shots
When a shot has both approved picture and approved audio, mark it CUT-LOCKED. This commits the take into the master film timeline.
Screen Continuous Scene Playback ('▶ Play Scene')
In the Scene Review page, click '▶ Play Scene'. The full-screen rough cut player plays all shots back-to-back in exact sequence order (with 2-second gray cards for inserts/graphics), letting you verify rhythm and continuity.
Master Timeline Review & Export
Open Timeline (/timeline) to inspect overall runtime, shot cuts, audio track alignment, and export your master cinematic cut.