A GTA VI streamer this week sits in front of a camera for 6 to 10 hours. Somewhere in there are maybe ten moments worth a Short. Watching it back to find them takes longer than the stream. We built Clipjack to do that part, and the target was a 6-hour VOD in about 4 minutes of wall time. This is how the pipeline is shaped.

1. Cut the stream into pieces and run them side by side

The slow way is to run a model over every frame. A 6-hour 1080p60 VOD is about 1.3 million frames. Instead the ingest step cuts the stream into 15-minute pieces with a 30-second overlap (so a moment on a boundary isn't cut in half) and processes them four at a time. Each piece is read once and produces cheap signals, and the output is one per-second signal table for the whole stream.

2. Audio and chat carry most of the signal

For streams, the streamer's voice is the best detector there is. Each audio piece goes through speech-to-text with word timestamps, which gives us loudness, pace, laughter and the words themselves ("no way", sudden silence after a crash). On Twitch and Kick we also replay chat: messages per second, emote spikes (LUL, KEKW, OMEGALUL), clip-command spikes. When chat and the streamer spike at the same second, that's almost always a moment.

3. Video signals are cheap, on purpose

Video runs on a 160p, 2 fps proxy, not the source. From it we read hard cuts, motion, flashes, the WASTED/BUSTED grey-out, HUD changes, stillness, and where the facecam box is. None of this needs a GPU. It confirms or rejects audio and chat candidates, and it tells us what to trim (menus, map screens, loading).

4. A judge, not a threshold

Every second gets a fused score, and the peaks become candidate windows. Only those go to the expensive model: 6 to 12 frames, the transcript window and the context around it. It decides keep or drop and, more importantly, the exact in and out points. That part matters most. When we measured 180 GTA Shorts from channels that get reach, 82% had the conflict on screen by 0.5 s and the payoff at a median 86% of the clip, with the cut 0 to 2 s after. Most raw clips fail on trimming, not picking.

5. Render last, and only what survives

Only the 10 to 20 picks get rendered: vertical 9:16, facecam detected and stacked over the gameplay, word-level captions. Rendering is the most expensive step per second of output, so it runs on the smallest set.

The numbers

Parallel pieces plus cheap signals is what turns hours into minutes. The expensive model only ever sees the candidates, a small slice of the stream. If you're building anything that has to search long video, the lesson we'd pass on: let cheap signals cut the haystack, and spend the model on boundaries, not on detection.

Demo: https://youtu.be/UoLW5tiUFLA. We launch on Product Hunt on Tuesday. Happy to answer questions on any stage.