A 40-module orchestration platform that runs two completely different production modes. Trending news gets the fully autonomous treatment — 26+ agents, zero manual steps. Personal, deeply human stories get the opposite: an AI interviewer that finds what's missing in a family's own history and asks, out loud, exactly the question that fills it in.
Share this case study →The hard part: an interview agent that has to ask a family the exact right question — grounded in real gaps in their own photo timeline, never an invented memory — plus a custom LLM router and MCP tool servers built from scratch to run the other 40 modules underneath it.
Postgres/pgvector state every assembly agent reads and writes — no fixed call sequence, no agent goes first
Python, TypeScript/Remotion, Docker, pgvector, MCP, FFmpeg, Manim, CUDA (RTX 3090)
Trending news runs fully autonomous: a swarm of specialized agents hands work off through a shared database rather than a fixed sequence of function calls, from raw signal to a published, multilingual video with zero manual steps. But personal, deeply human stories get a different front end entirely. A knowledge-graph module builds a real timeline of a family's life from what they've uploaded, then runs gap analysis against it — an identity that keeps appearing but was never named, two people who are close but never photographed together, an event with no clear emotional read, a place the family visits constantly with no faces in frame. Each gap becomes a targeted question, projected into a structured four-act mind map where the code owns the structure and the model only fills in the words. That mind map becomes a live, two-round interview: the agent asks the family the exact question the gap analysis surfaced — 'when was the last time you made dumplings with mom?' — pulling up the actual photo of that person while it asks, and listening by voice, not text. It edits the mind map through a fixed toolset of ten tools, never a free rewrite, with node IDs generated by code, not invented by the model, and three redundant memory layers so nothing a family says is ever lost mid-session. Only once the family confirms their story does it hand off to the same autonomous production pipeline that runs the news — script, voice, visuals, assembly, publishing.
All 26+ agents negotiate through one gateway for two scarce resources: GPU compute and LLM capacity. It monitors GPU memory in real time — idle local model, zero-cost local routing; busy GPU, cloud fallback — and every agent tags its requests with a caller identifier so usage is tracked per-agent. Agents call the gateway using semantic tier names instead of concrete model names, so any agent can be reassigned to a different model by changing one config value. This is the same gateway that powers this site's own chat assistant.
Assembly is itself a nested multi-agent workflow: a timeline agent decides segment timing, a media-resolver agent finds or edits assets, an audio-builder agent mixes narration with music, a continuity agent makes shot-transition decisions, a quality-gate agent inspects the result. The continuity, quality-gate, and assembler agents are deliberately zero-LLM and rule-based — deterministic agents handle measurable checks, LLM agents handle judgment calls. Every step is independently resumable via progress-file tracking, so a crash at step 12 of 15 resumes from step 12.
One agent implements a simplified version of Walter Murch's six rules of film editing, computing what's measurable — an 8x8x8 HSV color histogram scoring visual similarity between shots — then autonomously choosing a hard cut, a crossfade, or a black-frame pause. If OpenCV is unavailable, it degrades to hard-cut-only decisions with a warning instead of failing. The pipeline gets dumber, never crashes.
A completely deterministic agent inspects every finished video for black frames, silence, shot-size repetition, resolution consistency, and degraded assets — but never blocks on a failing grade, only reports it, because blocking on every issue in a fully automated system would deadlock production. One real documented bug: a blackdetect threshold was once misconfigured and normal dark-toned footage triggered false positives. The fix, and why it works, lives directly in the code comments.
Python, TypeScript/Remotion, Docker, pgvector, MCP, local + cloud LLMs, FFmpeg, Manim, CUDA on RTX 3090.