How We Made a 55-Second AI Action Reel on One GPU: Open Models, an MCP Studio and an ffmpeg Finishing Pass

We posted a 55-second Instagram reel this week: a skydive over Palm Jumeirah, a jet ski past Burj Al Arab, a Lamborghini Urus launching off a dune, four recurring characters with their own voices, and a twist ending where the hero tells the camera the whole thing was AI. Every frame came out of a studio we self-host on one GPU, and Claude drove the whole production through tool calls.

This post is the engineering side: how the studio is wired so an agent can use it, the one-GPU queue, and the ffmpeg finishing pass that turns raw clips into something you can post. The code below is simplified to the essentials.

The models

A licensing note up front: the H3 license asks users in the US, UK, EU and South Korea to apply to MiniMax, and commercial products must name the model. Keep the video model behind an interface so you can swap it.

H3's reference mode is what makes a series possible. You pass up to nine images (our character sheets plus a location still) and the dialogue, and it returns a 4 to 15 second clip at 768p with lip movement and stereo sound. Same faces, every shot.

Step 1: expose the studio as MCP tools

The agent never touches the GPU. It calls tools, gets a job id, and polls. With the official Python MCP SDK that is a few lines:

from mcp.server.fastmcp import FastMCP import db # thin wrapper over Postgres mcp = FastMCP("studio") @mcp.tool() def generate_video(prompt: str, character_ids: list[str] = [], reference_images: list[str] = [], duration_s: int = 10, aspect: str = "9:16", quality: str = "full") -> dict: """Queue a video clip. Returns a job id; poll job_status for the result.""" job_id = db.enqueue("video", dict(prompt=prompt, character_ids=character_ids, reference_images=reference_images, duration_s=duration_s, aspect=aspect, quality=quality)) return {"job_id": job_id, "eta_s": db.estimate("video", duration_s, quality)} @mcp.tool() def job_status(job_ids: list[str], wait_seconds: int = 0) -> list[dict]: """State, progress and (when done) presigned download URLs.""" return db.status(job_ids, wait_seconds=min(wait_seconds, 50)) if __name__ == "__main__": mcp.run() 

Two details matter more than they look:

1. Return an ETA with every job. The agent plans around it: it queues all stills first, reviews them while videos render, and does not sit polling a six-minute job every five seconds.
2. Let job_status block for up to 50 seconds. One long poll replaces a dozen chatty ones and saves a lot of tokens.

Step 2: one GPU, one heavy job, smart ordering

On an RTX PRO 6000 (96 GB), a 10 second clip at full quality takes about 346 seconds and a 5 second clip about 125. Swapping between the video, image and image-edit models costs 35 to 40 seconds each time. So the worker does two things:

• Runs one heavy job at a time (video and image models do not share the card well).
• Pulls images before videos when both are waiting, so the model swaps as rarely as possible.

-- claim the next job; images first, then oldest UPDATE jobs SET state = 'active', started_at = now() WHERE id = ( SELECT id FROM jobs WHERE state = 'waiting' ORDER BY (type = 'image') DESC, created_at FOR UPDATE SKIP LOCKED LIMIT 1 ) RETURNING *; 

The box is a spot instance. If it is off, the first job boots it (about 5 minutes plus a minute to load weights) and it can shut itself down when the queue stays empty. Spot pricing for this instance type is listed around $1.43 an hour, so a reel with all its retakes lands in the single-digit dollars.

Step 3: the finishing pass in ffmpeg

Raw clips are not a reel. Ours go through one Python script per cut:

# segment: (file, start, end, keep_clip_audio, captions) vf = f"trim={s}:{e},setpts=PTS-STARTPTS,scale=1080:1920:flags=lanczos,unsharp=5:5:0.8,fps=30" af = f"atrim={s}:{e},asetpts=PTS-STARTPTS,volume={1.0 if dialogue else 0.0}" 

Three lessons from this reel:

Mute the clips' own sound except dialogue. H3 generates ambient audio, which is great in isolation and muddy under a song. We keep clip audio only on talking shots.

Duck music with an envelope, not a compressor. Our first mix used sidechaincompress and the song vanished under the final pitch. A plain volume envelope keeps the song about 9 dB under the voices and always audible:

k = f"clip((t-{a0:.2f})/0.25,0,1)*clip(({b0:.2f}-t)/0.4,0,1)" # 0 outside dialogue, 1 inside fc = f"[mus]volume='1-0.72*{k}':eval=frame[duck];[dlg][duck]amix=inputs=2:normalize=0,loudnorm=I=-14" 

Watch out for filter graph surprises. Chaining acrossfade on segments split from the same input silently dropped a whole chorus in our build (31 seconds of music for a 48 second video). asplit plus concat fixed it. We only caught it because we measure loudness per three-second window after every render.

Step 4: automatic QA before a human sees anything

Every cut gets two checks:

• Frames: a contact sheet with one frame every 0.5 seconds, reviewed for anatomy, faces, captions and safe zones. This caught a character's head going through a car roof twice.
• Words: a speech-to-text pass on the final mix. One line the voice model rendered as "it's all a AI" got cut at the word level and re-checked until it read "It's AI."

from faster_whisper import WhisperModel model = WhisperModel("small", compute_type="int8") for seg in model.transcribe(audio, word_timestamps=True)[0]: print(round(seg.start, 1), seg.text) 

What it adds up to

Four open models, a queue, a worker and an ffmpeg script. The agent does the directing, the GPU does the rendering, and the edit layer is where quality actually comes from. For context, Higgsfield, which sells this kind of capability as a hosted product, says it crossed a $1 billion annualized revenue run rate in September 2026. The opening for smaller teams is a private or niche version: one brand's characters, one industry's ads, one kids channel.

If you want a studio like this built for your product or brand, that is what we do at Axionry. Book a call at https://axionry.com/book or see our builds at https://axionry.com