FFmpeg 9.0 upgrade check: a 60-line harness that diffs your recipes across two builds

📦 Code: github.com/USER/ffmpeg-upgrade-check (replace before publishing)

TL;DR

FFmpeg 9.0 "Lei" (released 2026-08-04) bumped every core library and removed some old support. Exit code 0 does not mean "same output". We'll build a ~60 line Python script that runs your real command lines with the old and new ffmpeg, then diffs ffprobe fields and SSIM. Python 3.10+, any ffmpeg.

🎯 What we're building

A script that takes two ffmpeg binaries, one source clip, and a JSON file of your recipes, and prints OK, DIFF, or FAIL for each one. Think of it as a snapshot test where the snapshot is "whatever the old build did".

What changed in 9.0, per the release coverage I read (news write-ups, not the changelog itself, so skim the official notes for your own recipes):

⚠️ Note: I could not find the names of the removed NVENC options in any source I read, so I won't guess them. The harness below is how you find out which of your flags are affected.

1. Pick the recipes that matter 🛠️

Not every ffmpeg command in your repo. The five to fifteen that produce things customers see. Put them in a JSON file as argument lists (no shell strings, so quoting bugs can't bite).

// recipes.json { "ladder_360p": { "args": ["-vf","scale=-2:360","-c:v","libx264","-crf","23","-preset","medium", "-g","60","-c:a","aac","-b:a","128k"] }, "audio_only_aac": { "ext": "m4a", "compare_video": false, "args": ["-vn","-c:a","aac","-b:a","96k"] }, "faststart_copy": { "args": ["-c","copy","-movflags","+faststart"] } } 

Optional per-recipe keys: ext (output extension), compare_video (skip SSIM for audio-only), and min_ssim (default 0.99).

2. Write the harness

#!/usr/bin/env python3 # upgrade_check.py """Run the same ffmpeg recipes with two builds and diff the results.""" import json, re, subprocess, sys, tempfile from pathlib import Path OLD, NEW = sys.argv[1], sys.argv[2] SRC = sys.argv[3] RECIPES = json.loads(Path(sys.argv[4]).read_text()) def run(ffmpeg, args, out): cmd = [ffmpeg, "-hide_banner", "-v", "error", "-y", "-i", SRC, *args, out] p = subprocess.run(cmd, capture_output=True, text=True) return p.returncode, p.stderr.strip() def probe(path): p = subprocess.run( ["ffprobe", "-v", "error", "-show_entries", "stream=codec_name,profile,pix_fmt,width,height,r_frame_rate,nb_frames,channels,sample_rate", "-show_entries", "format=duration", "-of", "json", path], capture_output=True, text=True) return json.loads(p.stdout) def ssim(ffmpeg, a, b): p = subprocess.run([ffmpeg, "-hide_banner", "-i", a, "-i", b, "-lavfi", "[0:v][1:v]ssim", "-f", "null", "-"], capture_output=True, text=True) m = re.search(r"All:([0-9.]+)", p.stderr) return float(m.group(1)) if m else None failures = 0 with tempfile.TemporaryDirectory() as d: for name, r in RECIPES.items(): ext = r.get("ext", "mp4") a, b = f"{d}/{name}.old.{ext}", f"{d}/{name}.new.{ext}" rc_a, err_a = run(OLD, r["args"], a) rc_b, err_b = run(NEW, r["args"], b) if rc_a != 0 or rc_b != 0: print(f"FAIL {name}: old rc={rc_a} new rc={rc_b}\n new stderr: {err_b[:200]}") failures += 1 continue pa, pb = probe(a), probe(b) diffs = [] for sa, sb in zip(pa["streams"], pb["streams"]): for k in sa: if sa.get(k) != sb.get(k): diffs.append(f"{k}: {sa.get(k)} -> {sb.get(k)}") if len(pa["streams"]) != len(pb["streams"]): diffs.append("stream count differs") q = ssim(NEW, a, b) if r.get("compare_video", True) else None if q is not None and q < r.get("min_ssim", 0.99): diffs.append(f"ssim {q:.4f} < {r.get('min_ssim', 0.99)}") if diffs: failures += 1 print(f"DIFF {name}: " + "; ".join(diffs)) else: print(f"OK {name} (ssim={q})") sys.exit(1 if failures else 0) 

Three design choices worth knowing:

• Probe fields, not just success. profile, pix_fmt, nb_frames, and sample_rate are where silent drift tends to show up.
• SSIM between old and new outputs, not against the source. We aren't grading quality; we're asking "did the upgrade change the picture?"
• Exit code mirrors failures, so you can drop it straight into CI.

3. Run it 🚀

Grab a short representative clip (a few seconds with motion and audio is plenty) and run:

# ffmpeg-old and ffmpeg-new are the two builds you are comparing python3 upgrade_check.py /opt/ffmpeg-8.1/bin/ffmpeg /opt/ffmpeg-9.0/bin/ffmpeg sample.mp4 recipes.json echo "exit=$?" 

I only had FFmpeg 6.1.1 in my sandbox, so I could not produce a real 8.1-vs-9.0 diff. To show what the output looks like and to prove the harness works, I ran 6.1.1 against itself with one deliberately broken recipe added:

$ python3 upgrade_check.py ffmpeg ffmpeg in.mp4 recipes.json OK ladder_360p (ssim=1.0) OK audio_only_aac (ssim=None) OK faststart_copy (ssim=1.0) FAIL broken_recipe: old rc=8 new rc=8 new stderr: Unrecognized option 'no_such_option'. Error splitting the argument list: Option not found exit=1 

That FAIL is what a removed option will look like on 9.0. A DIFF line is the other thing you are hunting for, for example profile: High -> Main or nb_frames: 120 -> 119.

💡 Tip: x264 is deterministic for the same build and input, so a clean self-run gives ssim=1.0. Across versions, a score a hair below 1.0 may be a legitimate encoder improvement. The point is that a human decides, instead of nobody noticing.

4. Triage what comes out

My suggested order for a 9.0 upgrade, driven by what the release removed:

1. NVENC recipes. Check your driver/SDK image first (SDK older than 11.1 is no longer supported), then run your hardware recipes.
2. Odd audio. Run a folder of weird uploads through, not just clean fixtures. The CELT removal is narrow, but legacy Ogg files are the sort of thing that lives in a big library.
3. Stream-copy recipes. Muxers moved. Compare durations, stream counts, and seek behavior.
4. New features last. Upgrade first, adopt the GPU ONNX backend later, so a regression has one suspect.

5. Wire it into CI and pin your build 🔒

# .github/workflows/ffmpeg-upgrade.yml (sketch) name: ffmpeg-upgrade-check on: [pull_request] jobs: diff-builds: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: ./scripts/fetch_ffmpeg.sh old new # your script; downloads two pinned builds - run: python3 upgrade_check.py ./bin/old/ffmpeg ./bin/new/ffmpeg fixtures/sample.mp4 recipes.json 

Also record the exact build string (ffmpeg -version | head -1) in your output metadata. When support asks "when did this change?", you can answer in one query.

What's next

• Read the official 9.0 release notes against your own flags; this post leans on news coverage.
• Add a min_ssim per recipe if you deliberately want to accept small encoder changes.
• Pair this with a VMAF check if you already track perceptual quality. Our harness only asks whether the output moved, not whether it got better.

If you run it on a real 8.x to 9.0 diff, I'd love to hear what showed up. #ffmpeg #video #devops