An unattended production pipeline: a topic goes in, a finished video comes out

The challenge

Producing short-form video is a chain of tasks that are individually simple and collectively expensive: pick a topic, write a script, record narration, source visuals, cut them together, write the metadata, publish. Every stage waits on a person.

The interesting question was not whether each stage could be automated — it was whether the whole chain could run unattended without a person checking between every step, which is where these pipelines usually quietly fail.

The context

Nothing here was one system. It was a language model, a text-to-speech service, an image generation service, a video renderer and object storage, each with its own latency, failure modes and cost per call.

Stages also run at wildly different speeds. Scripting takes seconds; rendering takes minutes. Anything built as a single synchronous run would spend most of its life blocked.

The approach

I built it as a queue-driven pipeline rather than a script. Each stage does one thing, reads its input from durable state and writes its output back, so any stage can be retried, resumed or replaced without touching the others.

The binding constraint turned out to be the interface between stages. A model producing prose that a human reads is forgiving; a model producing input for the next automated stage is not. So every generative step was made to emit strict structured output, validated before anything downstream is allowed to use it.

The solution

A topic enters the queue. The model produces a scene-by-scene script as structured data — narration text plus explicit visual descriptions for each scene, in a fixed schema.

Each scene's narration goes to text-to-speech and its descriptions to image generation, both in parallel. Assets land in object storage with their metadata recorded against the job. The renderer assembles narration, visuals and timing into the finished video, and the metadata needed to publish it is written alongside.

The orchestration runs on a self-hosted workflow platform, which keeps the sequencing visible and editable instead of buried in code, and keeps the whole thing on infrastructure I control.

AI & automation

Generative models do three jobs here — topic and title generation, scripting, and image generation — and the pipeline treats all three as unreliable components, because they are.

The scripting prompt specifies the exact JSON shape it must return, and the result is schema-validated before the pipeline moves on. Anything malformed fails at that boundary and is retried rather than being allowed to produce a broken video ten minutes later. Prompts are versioned with the pipeline, so a change in output can be traced to the change that caused it.

Engineering

The engineering is mostly about failure. Every external call is retried with backoff, because these services time out often enough that a pipeline without retries never completes a run.

Stages are idempotent and keyed by job, so a retry does not regenerate assets that already exist and does not pay for them twice — cost control is a correctness problem here, not an afterthought. State is durable, so a restart resumes the job rather than starting it again. The whole thing is containerised so it runs the same on a laptop and on a server.

Outcome

The pipeline runs end to end without supervision, and the parts that break are the parts that were always going to break — third-party APIs, not the orchestration. More usefully, it taught me exactly where multi-stage generative systems fall over: not in the model quality, but at the joins. That lesson goes into every AI pipeline I design for someone else.