OpenMontage

Overview

OpenMontage turns a single command such as --topic "Why Airplane Windows Are Round" --target-minutes 9 into a finished explainer documentary. The film has narration, captions, generated explainer graphics and real footage from stock libraries and public archives. The system runs local-first on free-tier providers. Python owns the deterministic machinery (orchestration, persistence, retries, QA, rendering), while all creative content comes from pluggable LLM providers and is saved as explicit, inspectable artifacts.

The problem

A long-form explainer video (8–10 minutes, YouTube style) takes a person days of work: research, scripting, voiceover, finding footage, editing, captions and review. Most "AI video" tools shortcut this by animating a handful of still images. The result looks like a slideshow, and the tools rarely check whether what's on screen matches what's being said.

The solution

OpenMontage treats video production as a staged, auditable pipeline:

• Research → Script → Explainers → Narration → Beat planning → Assets → Asset QA → Edit → Render → Final QA → Freeze
• Every stage writes a JSON-schema-validated artifact, is cached, and can be resumed. Expensive stages are never redone unnecessarily.
• A strict creative boundary: Python never writes creative content. If no usable LLM provider is available, the pipeline fails before narration rather than speaking placeholder text.
• Real motion footage first: a visual planner turns each script beat into search queries across 16 free stock and archive sources, and CLIP-based scoring checks the candidate clips for relevance.
• Self-checking output: automated QA detects black, frozen or crushed frames, slideshow risk and pacing problems, then repairs shots within a budget.

Key features

Production

• One command from topic to finished documentary, with resume from any stage.
• Stage versioning invalidates only what changed.
• Artifacts include a research brief, long-form script, beat plan and visual search plan.
• Local narration with Piper TTS, with measured speaking pace (words per minute) tracking.
• Timed, frame-safe captions.
• Generated explainer graphics: charts, stat cards, callouts and diagrams.
• Music search and mixing.
• A production freeze step produces an immutable final bundle plus a production_report.json.

Footage retrieval

• 16 free stock and public-archive sources, including Pexels, Pixabay, Wikimedia, Archive.org, NASA, NOAA, ESA, JAXA, the Library of Congress and NARA.
• Wave-based search, candidate pools, CLIP relevance scoring, source reuse limits and a check that enough footage exists before committing.
• Per-source circuit breakers, download deadlines and caching of failed searches.

Quality

• Asset QA covering black and limited-range black frames, frozen ranges, motion confirmation, slideshow risk and pacing (long holds, oversized beats).
• A repair ladder that replaces bad shots using per-pass budgets.
• Final QA runs on the rendered MP4.

Platform

• 13 pipelines, including:
  • documentary montage and animated explainer;
  • talking head and screen demo;
  • podcast repurposing and a clip factory;
  • localization dubbing and more.
• 100+ tools covering text-to-speech, image and video generation, avatars and lip-sync, enhancement, and 3D.
• Budget governance: estimate, then reserve, then reconcile.
• Backlot, a live local storyboard board for watching a run.
• Agent-first mode: coding agents can drive the pipeline from YAML manifests and markdown skills.

How it works

A Python orchestrator runs the stages in sequence and persists an artifact after each one. LLM calls go through task-specific provider chains that fail over automatically (Groq, Gemini, OpenRouter, Ollama, and optionally OpenAI or Anthropic). Rendering uses Remotion (React) and HyperFrames (HTML/CSS/GSAP), and FFmpeg handles probing, trimming and muxing.

Tech stack

Engineering highlights

• A clean determinism boundary. Orchestration, persistence, retries, repair, QA, rendering and freezing are deterministic Python. Creative content is always a saved artifact from a provider, never hard-coded text.
• Resumable and cached. Every stage is resumable. Stage versioning and render fingerprinting mean only the changed parts are recomputed, and the final render is reused when inputs match.
• Provider resilience. Task-specific failover chains mean a single provider hitting its free quota no longer stops a run.
• Retrieval engineering. Wave-based search, per-source circuit breakers, per-download deadlines, caching of failed searches and supply preflight keep long runs moving even when sources misbehave.
• Automated video QA. Frame analysis detects black, frozen and crushed segments, then a budgeted repair ladder swaps defective shots automatically.
• Schema-validated artifacts. Canonical JSON schemas (research, script, beats, visual plans) make every stage inspectable and testable.
• Proven end to end. One completed documentary ran fully autonomously, producing 60 beats and 82 shots (average shot about 4.5 s), with no beats left without footage and 8 automated QA repairs.
• Extensible architecture. A tool registry, a BaseTool contract, selector and provider tools, and 13 pipeline manifests.

Challenges & what I learned

1. Keeping AI creative but the system reliable. Challenge: LLM output is variable, and long JSON responses sometimes get cut off. Approach: schema validation, bounded recovery from truncated output, and a hard rule that the pipeline fails early instead of narrating placeholder text. Lesson: treat model output as untrusted input to deterministic code.
2. Free sources are unreliable. Challenge: stock and archive APIs time out, change their HTML or return irrelevant clips. Approach: circuit breakers, deadlines, caching of failed searches, CLIP relevance scoring and supply preflight. Lesson: retrieval at scale is mostly failure handling.
3. Shots that look fine but aren't. Challenge: downloaded clips can contain black leaders, frozen frames or crushed blacks. Approach: frame-level QA with defects scoped to time ranges and a budgeted repair ladder. Lesson: automated review is what makes autonomous production usable.
4. Matching narration and visual timing. Challenge: in long-form runs, fixed scene lengths can outrun the narration and leave silent stretches. Approach (in progress): narrate first, measure the actual speech duration, then size beats to fit it, and enforce script length from measured words per minute. Lesson: derive timing from what you have measured, not from targets.

Status & what's next

• The autonomous documentary runner works end to end and has delivered a complete film. Several other topic projects are at various stages.
• Current focus: timing driven by narration length, plus stricter enforcement of script length.
• Planned next:
  • a human review checkpoint in Backlot, so shots can be approved or swapped before rendering;
  • sound design and music ducking;
  • a script fact-check pass with citations;
  • tracking of licence attribution for every clip;
  • thumbnail, title and chapter generation;
  • vertical short cutdowns;
  • one-click Docker setup.
• Intended as a human-in-the-loop production tool, with AI disclosure for published content.