One MP4, 27 Runs, 15 Different Transcripts. Lower-Case Them and 26 Are the Same 213 Words

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them turns video into text — paste a YouTube, TikTok, Instagram or Vimeo link, or a direct MP4, and get the spoken words back with each passage tied to the second it was said.

Like every Actor I publish, it has a scheduled check: once a day, send one fixed file and make sure data comes out. Boring, cheap, and it has now produced something more useful than the alarm it was built for — a month of identical requests against the same audio, 5 October to 4 November 2026. Twenty-seven runs, one 60.01-second MP4, quality: fast (Whisper large v3 turbo), language left to auto-detect and detected as English every time.

Twenty-seven identical requests produced fifteen different transcript strings.

If you are building anything that compares transcripts — change detection on a weekly webinar, a cache keyed on output, a regression test that asserts on text — that number is the one that will hurt you. The good news is the shape of the difference, because it is not what it looks like.

What actually moved

The file is a passage from Pride and Prejudice read aloud three times over, which turns out to be an accidentally excellent test fixture: each run transcribes the same two sentences three times, so you get within-run evidence for free.

This is not a build drifting under me. The Actor was rebuilt through the month (0.1.1 up to 0.1.17), but seven of these runs are the identical build 0.1.17, and on their own they produce four different word counts (211, 214, 215, 216) and three different segment counts (9, 13, 18). Nothing in the request pins the sampling either — the recogniser call sends no temperature and no seed, so there is no determinism to be had and no bug to fix.

Now normalise. Two steps:

import re def norm(t): t = t.lower() t = re.sub(r"[^a-z0-9' ]+", " ", t) # 1: drop punctuation and hyphens  t = re.sub(r"\s+", " ", t).strip() return t.replace("some one", "someone") # 2: one compound word 

Step 1 alone takes 15 distinct strings down to 8. Step 1 plus step 2 takes them to 2 — and the second of those is a single run. Twenty-six of the 27 transcripts are byte-identical once you lower-case them and collapse one compound: 213 words, every time.

So the honest word-error rate on this file is not "15 variants out of 27". It is one run out of 27.

The three coin flips

Counting all 81 occurrences (three repeats × 27 runs), the model never changes what it heard. It changes how it writes it down:

And — this is the part I did not expect — all 27 runs are internally inconsistent. Not one run punctuated the three identical repeats the same way. A typical row: "the rightful property of some one or other of their daughters" in the first repeat, "someone" in the second, "some one" again in the third. Same voice, same words, same request, eleven seconds of compute, three different answers inside one string.

Which explains the word counts completely. The Actor computes them the dumbest possible way, and should:

'wordCount': len(text.split()), 'charCount': len(text), 

some one is two tokens, someone is one. Three occurrences per file means wordCount can legally land anywhere in a four-wide band without a single syllable being misheard. The 211 at the bottom of the range is the one run that wrote little-known twice — a hyphen eating two spaces.

The one run that was actually different

For completeness, the outlier: 20 October, 19 segments, 212 normalised words against everyone else's 213. Two genuine recognition errors, both in the later repeats of the same sentence:

• repeat two: ...must be in want of a wife. However, however little known... — a doubled word
• repeat three: ...in the minds of the surrounding families is considered the rightful property... — the words that he are gone

Three words wrong out of 213, in 1 of 27 runs. That is the real failure mode to budget for, and it is a different thing from the fourteen cosmetic variants that an exact-match test would also flag.

What I changed in my own pipelines

Never diff raw transcripts. If your job is "has this recording changed", compare normalised text, not the field as delivered. Raw comparison on this file reports a change 14 times out of 27 and is wrong every time.

Never key a cache or a dedup on transcript equality. You will store fifteen copies of one answer. Hash the normalised form, or better, hash the source URL plus durationSeconds — which was 60.01 on all 27 runs and is the field the billing agrees with.

Never treat wordCount as an identity or a volume metric. It is a token count over a string whose tokenisation is unstable. For cost, durationMinutes is the field that matters: all 27 runs billed audio-minute × 1 + file-processed × 1 = $0.011, because billing is per started minute of audio and the audio never changed.

In tests, assert with a tolerance. Mine now compares normalised word sequences and allows a small edit distance — on this fixture, the threshold that passes all 27 runs and would still catch a real break is three words in 213.

If you need the spelling of names and jargon to be stable, that is what vocabularyHint is for — it is the only lever here that actually constrains the output, and it constrains vocabulary, not punctuation.

Calling it

curl -X POST "https://api.apify.com/v2/acts/lergassy~video-to-text/run-sync-get-dataset-items?token=YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{"urls":["https://example.com/clip.mp4"],"quality":"fast","includeSegments":true}' 

Today's scheduled run of exactly that shape: 10.1 seconds, one row, $0.011, on 1 GB of memory. The row carries transcript, wordCount, charCount, segmentCount, durationSeconds, language, model, and segments like these:

[ {"start": 0, "duration": 4.56, "text": "It is a truth universally acknowledged, that a single man in possession of a good fortune,"}, {"start": 4.94, "duration": 1.36, "text": "must be in want of a wife."}, {"start": 6.62, "duration": 4.04, "text": "However little known the feelings or views of such a man may be on his first entering"} ] 

A social video link costs more than a file, and it is worth knowing why before you plan a batch. A 152-second Instagram reel in the same series billed 3 audio-minutes + 1 file + 1 video-link = $0.119 and took 20 seconds: the $0.09 is the audio being pulled out of a video page through a residential exit node, and a direct file link never pays it. The Actor is Video to Text.

One error row from the same month explains the charge model better than I can: budget-reached, "This file would be charged $0.091 but only $0.060 remain within the run budget." Cap a run with "Max cost per run" and the cap is counted in dollars against the whole row — file plus minutes plus video-link fee — before the download starts. Error rows are free and carry status: "error", so check status before reading transcript.

When this is the wrong tool

If you need a transcript that is byte-stable across runs, no hosted recogniser will give you one — store the audio and the run ID and re-read the row you kept. If you need per-word timings, these segments are phrase-level: nine of them for a minute of continuous reading, in the most common case. And if the words are already sitting in a YouTube caption track, don't pay to re-recognise them. Where this earns its keep is everything with no caption track at all — meetings, lectures, voice notes, downloaded MP4s — at nine-tenths of a cent a minute, with fifteen different strings that all mean the same 213 words.