Two weeks ago I ran 100 images through four AI image detectors to pick the engine behind a small checker I built. Since then OpenAI shipped GPT Image 2.5 (September 8) and Google shipped Nano Banana 2.1 (October 6). A detector that was fine on last season's models can fall behind quickly, so I retested the checker as it actually runs today on the newest generators I could get clean samples from.
The question was narrow: how often does an image from a new generator come back as anything other than "Likely AI-generated", and how often is a real camera photo flagged?
Rules first, results second
I wrote the protocol down before the first detection call: which images, how they're picked, how answers are counted, and what I'd change on the site depending on the result. The one decision rule: if any generator is missed more than 1 time in 5, results that come back "Likely a real photo" or "Can't tell" get a short note saying how many images from that generator we missed.
A miss means anything other than "Likely AI-generated". The checker has three answers: Likely AI (detection score 0.8 or above), Likely a real photo (0.2 or below) and Can't tell (in between). So a Can't tell on an AI image counts against it.
No threshold tuning, no swapping out images after seeing a score. Spares were allowed only for download or format failures, and they were fixed before any score existed.
Where the images came from
Provenance was the hard part. A file called "midjourney.png" on the internet proves nothing, so each bucket needed a source that records the model:
- GPT Image 2.5, Nano Banana (2.1, 2, 2 Lite, Pro), FLUX.3 and Flux.2 [dev]: made with Civitai's on-site generator, which writes down the model it called. Text-to-image only, no source image, no LoRAs, at most one image per user. 20 per family.
- Midjourney V7: original PNG downloads from a CC0 archive on Hugging Face whose embedded prompt carries
--v 7and a Midjourney job id. 20 images, but all from one creator, so they share a style. - Midjourney, recent community posts: 20 r/midjourney showcase posts. Reddit strips the prompt, so the version can't be verified. I report this row separately and never pool it with the others.
- Real photos: 40 camera originals from Wikimedia Commons "Taken with …" categories, shot in 2024 or later: 24 from recent phones (iPhone 17 Pro, 16 Pro, Pixel 10 Pro, Galaxy S25 Ultra and others) and 16 from mirrorless cameras.
That's 140 images. I kept links and fingerprints, not the images.
Two findings before any model ran
None of the 60 Civitai files carried Content Credentials. That includes the 20 GPT Image 2.5 images. In my first test, all 8 original OpenAI files carried OpenAI-signed credentials (6 I made through OpenAI's image API, 2 from a public dataset), and the checker settled them without calling a model. Files that come out of a third-party generator site don't necessarily keep them, so here the detection model had to decide. If you're trying to work out whether ChatGPT made a particular picture, I wrote up when OpenAI's credentials settle it and when they don't, and what OpenAI's own verifier can and can't confirm.
Labels anyone can write were common. 6 of the 20 Nano Banana files and all 20 Midjourney V7 files carried an AI label in their metadata. The checker doesn't trust that kind of label on its own, because anyone can add it to a real photo, so those images still went to the model. Only signed credentials from signers on the C2PA trust lists settle an answer: in this run that happened for 3 real photos, which carried Google Pixel camera credentials.
A small product bug turned up as well: two 4K PNGs from Civitai were over the site's 25 MB upload limit. On the live site those uploads fail with "too large".
Results
Run on October 28, 2026, through the same code path the site uses, with Sightengine as the detection model.
| Generator | Images | Likely AI | Can't tell | Called a real photo | Missed (95% range) |
|---|---|---|---|---|---|
| OpenAI GPT Image 2.5 | 20 | 20 | 0 | 0 | 0 of 20 (0–16%) |
| Google Nano Banana 2.1, 2 and Pro | 20 | 20 | 0 | 0 | 0 of 20 (0–16%) |
| FLUX.3 and Flux.2 [dev] | 20 | 19 | 1 | 0 | 1 of 20 (1–24%) |
| Midjourney V7 | 20 | 19 | 1 | 0 | 1 of 20 (1–24%) |
| Midjourney, community posts (version not verified) | 20 | 18 | 1 | 1 | 2 of 20 (3–30%) |
78 of the 80 verified AI images came back Likely AI and none were called a real photo. Every miss in the main rows was a Can't tell: a Flux.2 [dev] image scored 0.79, just under the 0.8 cut-off, and a Midjourney V7 image scored 0.55. In the community row, one post scored 0.77 (Can't tell) and one scored 0.06, the only AI image in the run the checker called a real photo.
Real photos: 38 of 40 came back Likely a real photo. One was flagged as AI with a score of 0.99, and one got a Can't tell at 0.64. That's 1 of 40 flagged (2.5%, 95% range 0–13%). Both errors were recent iPhone shots, an iPhone 17 and an iPhone 17 Pro. Two photos aren't a pattern, but it's the case I'd watch, because a false alarm lands on a real person. If the image you're checking is a viral picture or a screenshot rather than an original file, the clues are different; I keep notes on checking those.
No generator crossed the 1-in-5 line, so the decision rule said no warning, and I didn't add one.
What the numbers don't say
Twenty images per generator is small. "0 of 20 missed" is still compatible with missing about 1 in 6, which is what the 95% range shows. The Midjourney V7 images come from a single creator. The community Midjourney row can't be tied to a version. Civitai's generator is one way to reach these models, not the only one, and files from other apps may behave differently, especially around credentials. And vendors update their models, so this is a snapshot of one week.
Disclosure
I built this checker and it uses Sightengine as its main engine, with Hive as a backup, so I have a stake in these numbers looking good. That's why the protocol was written first and why every row is published, misses included. No detection company paid for this or saw it in advance.
The full table, the method and the earlier 100-image comparison of four engines are on the accuracy page. If you have an image from one of these generators that it gets wrong, I'd like to see it.
An AI agent working for me wrote this post from the test logs, with no line-by-line human edit, so it's marked as AI-generated. The images, the runs and the numbers are real; the method and the full table are on the accuracy page.
