In October I tested 8 AI song detectors on 65 songs. The winner caught 23 of 25 AI songs and flagged none of the 40 human ones. That looked great, and it was also the easy case: clean 30-second Apple Music previews of released songs by acts the press had already named as AI.
People don't only paste clean previews. They paste a cover where someone swapped the singer's voice, a demo with an AI backing track under a real vocal, a phone recording from a gig, a lo-fi loop, a song made last week with Suno v6. So a few days later I built a small test for each of those, wrote the pass line down first, and ran four detection engines on them. Mostly, nothing passed.
The pass line, and why 20 clips can only fail an engine
Each input type got 20 clips: 10 AI, 10 human. To pass, an engine needed at least 85% correct and at most 5% of human clips flagged as AI.
With 10 human clips, a single false alarm is already 10%, so in practice an engine had to flag none. And "0 of 10" still allows a real rate up to about 28% at 95% confidence. So these small tests can sink an engine, but they can't prove one works. That's on purpose: the question was "can the checker give a firm answer on this kind of audio?", and I wanted the default to be no.
The four engines:
- Audio fingerprint model: the one that won the first test. AI generators turn sound out of a neural audio decoder, which leaves a row of evenly spaced peaks in the spectrum; this model measures them.
- Second audio model: a second, small neural-network model.
- Deep detection: a separate detection model that reads one 30-second excerpt.
- Isolated-vocal analysis: split the vocal out first, then judge the voice on its own.
What came back
Correct out of 20 for each engine, with human clips wrongly flagged as AI in brackets (out of 10):
| Input type | Fingerprint | Second model | Deep detection | Isolated vocal |
|---|---|---|---|---|
| AI covers (voice swaps) | 10 (0) | 12 (1) | 20 (0) | 20 (0) |
| AI backing under a human vocal | 10 (0) | 11 (0) | 12 (0) | 10 (1) |
| Suno v6, 10-second clips | 13 (0) | 14 (6) | 18 (1) | 9 (2) |
| Live / played through a speaker | 10 (0) | 14 (5) | 10 (1) | 10 (1) |
| Lo-fi | 11 (0) | 16 (3) | 14 (5) | 9 (2) |
Read across a row and the pattern is the same almost every time: the fingerprint model never accuses a human song, but on these inputs it barely finds any AI either. The models that do find the AI start flagging real musicians.
Covers. The fingerprint model missed all 10 voice swaps, which makes sense: a voice swap takes a real recording and replaces the singer, so most of the audio never went through an AI decoder. Deep detection and the vocal analysis both scored 20 of 20, which looks like a win until you see the set: 19 of the 20 clips come from a public research test set, a handful of singers and older voice-conversion models. I don't trust that to stand for modern voice clones, so the checker still says "Unclear" for covers.
AI backing under a human vocal. Everything failed; the best of the four got 12 of 20. These were controlled 7-second mixes built from 20 different songs, 10 with a MusicGen backing track under a real vocal and 10 fully human, because I couldn't verify a single released hybrid song to test instead. It's the same lesson as TaTa's "Pulse" in the first test, built on Timbaland's demos with a voice from Suno, which scored below 0.001: if part of a song is real, detectors usually go with the real part.
Live and speaker recordings. I played 10 AI and 10 human recordings through a simulated room with noise, a band-pass filter and AAC compression, a rough stand-in for a phone at a gig. The fingerprint model missed all 10 AI clips; the peaks it looks for don't survive the room. The second model found the AI but flagged half the human live recordings. I haven't tested a real phone recording of an AI song played through real speakers.
Lo-fi. 10 instrumentals a platform labels as AI against 10 human lo-fi tracks from 5 producers. My guess is that tape hiss and bit-crushing look like generator artifacts to some models: deep detection flagged 5 of the 10 human tracks.
Suno v6 got its own test
The 10-second Suno row was too small and too clustered to trust in either direction (the 10 AI clips came from 3 prompt ideas and 2 lyric sets), so I ran a bigger one with the method and both song lists fixed before any detector ran:
- 50 Suno v6 songs from 46 creators, at most two each. Each is a file its creator published on Internet Archive or Audius with its own Suno song ID, and Suno's public page for that ID says it was made with v6.
- 50 human songs: 30-second Apple Music previews of current chart hits from 10 countries, released September 2025 to October 2026, by artists with releases before 2022, so none of them is a new AI act.
| Engine | Correct of 100 | Human songs flagged | Suno songs missed |
|---|---|---|---|
| Fingerprint | 100 | 0 of 50 | 0 of 50 |
| Second model | 98 | 0 of 50 | 2 of 50 |
| Deep detection | 88 | 10 of 50 | 2 of 50 |
| Deep detection + one of the first two agreeing | 98 | 0 of 50 | 2 of 50 |
Deep detection on its own flagged 10 of 50 chart hits as AI, 17 of 50 when I re-encoded the previews to MP3. That's 1 human song in 5, which would be a disaster in a public tool. Requiring it to agree with one of the first two models brought the false alarms to zero on this set.
The fingerprint model's 50 of 50 here and its 13 of 20 on short clips don't contradict each other so much as warn you: clip length, prompts and who made the songs all move the number, and one small test can't tell you which.
Then I checked the agreement rule again the same day on a fresh set, all as uploaded files: 50 new Suno v6 songs from 50 different creators, and 119 human songs that weren't only chart hits (30 chart hits, 29 newer artists who've played NPR's Tiny Desk, 30 independent netlabel releases, 30 live concert recordings). It called 45 of 50 Suno songs "Likely AI" and 3 of 119 human songs "Likely AI" (2.5%, 95% interval 0.9–7.2%). The bar I'd set for switching it on was at least 95% accuracy and at most 3% human false alarms; it got 95.3%, so uploads now use it. What an uploaded Suno file gets today, and what still returns "Unclear", is on the Suno detector page.
The same song, three platforms, three answers
One more test I didn't expect to matter: 20 songs (10 AI, 10 human), first 10 seconds of each platform's own preview. The second model got 15 of 20 on Apple Music previews with 4 human songs flagged, and 19 of 20 on Spotify's previews with none flagged. Same songs, same model, different file.
That matters because Spotify doesn't share audio with other sites. When you paste a Spotify link into a checker like mine, it reads the title and artist and analyses the Apple Music preview of the same track instead. And Spotify's own badge answers a different question: its AI label marks an artist's public identity, not how a song was made. There are two kinds, "AI Persona" when the artist tells Spotify and "Likely AI Persona" when Spotify applies it; I wrote up what Spotify's AI label means and where it shows.
YouTube links got a separate test on October 26: 60 videos from 42 acts whose albums a streaming platform labels as AI, and 106 human videos (chart hits, older songs, covers and live concerts). The checker called none of the 106 human videos "Likely AI" and gave a firm answer on 74 of the 166, all correct. The other 92 got "Unclear".
What this did to the AI acts people actually search for
Five acts people keep searching for. Scores are from the fingerprint model on Apple Music 30-second previews in the first test (October 21), 0 = human-like, 1 = AI-like:
| Act | What's on the public record | Song: score |
|---|---|---|
| Cain Walker | Apple Music calls it an AI music project credited to Dallas Little; Spotify applied a "Likely AI Persona" label | "Don't Tread On Me" 0.999, "Can't Break Me" 0.9999 |
| Bleeding Verse | Its Spotify bio says "Human lyrics, turned into songs with AI" | "If You Loved Me Then" 0.9999 |
| Sienna Rose | Anonymous neo-soul act; a streaming service told Rolling Stone and the BBC that many of her songs are flagged as AI | "Into the Blue" above 0.99, "Where Your Warmth Begins" 0.9999 |
| IngaRose | Its own profiles say "Celebrate Me" was composed with Suno, lyrics human-written; No. 1 on US iTunes in April 2026 | "Celebrate Me" 0.99 (0.53 after downsampling to 22 kHz AAC) |
| Xania Monet | Billboard reports poet Telisha Jones made the R&B singer with Suno and signed her to Hallwood Media | "How Was I Supposed to Know?" 0.98, "Let Go, Let God" 1.00 |
Every one of these scored far above the line on a clean preview. IngaRose's row shows how fast that goes: the same 30 seconds, downsampled and re-compressed, fell from 0.99 to 0.53, straight into "can't tell".
Then the new rules hit two of them. On November 2 I ran a few of their songs through the live checker. Cain Walker's "Don't Tread On Me" and Bleeding Verse's "If You Loved Me Then" came back "Likely AI". Cain Walker's "Ain't My Problem" and Bleeding Verse's "Only When It's You" came back "Unclear": the fingerprint and the second model were both above 0.98, but deep detection gave a human-like signal, and when the signals disagree the checker won't pick a side.
Two songs from openly AI acts ending at "Unclear" is a miss, and I'm not going to pretend otherwise. It's also exactly what the agreement rule costs. I'd rather say "can't tell" about a self-declared AI band than call a real singer AI.
What I'd tell someone checking a song
- Figure out what you're holding before you trust any answer. A streaming link or an original file is the case detectors are tested on. A cover, a live clip, a lo-fi edit or a TikTok rip is a different problem, and most tools, mine included, can't judge it reliably yet.
- Treat "Unclear" as an answer, not a failure. A detector that never says it is guessing on the inputs above.
- Platform labels mark artists, not songs. They're useful context, not a verdict on one track.
- Never use a detector alone to accuse a real musician. Not mine, not anyone's.
Limits and who's writing this
- Each input type is 20 clips, mostly controlled mixes I made or picked. Different songs could score differently, and the intervals are wide.
- The AI side is Suno-heavy. I have no usable Udio samples, so nothing here says anything about Udio.
- The engine table measures individual models at fixed thresholds, not the checker's three-answer policy, so there's no single new accuracy number to quote.
- I'm not neutral. I built isthissongai.app, so I have an interest in these numbers. Who runs it and how its pages are made is on the about page.
Every bucket, every interval and the full method are on the accuracy page. If you have a cover, a live recording or a part-AI song where you know the answer, I'd like to test it.
An AI agent working for me wrote this post from the published test results, with no line-by-line human edit, so it's marked as AI-generated. The tests and the numbers are real and are published in full on the accuracy page.
