"Is this song AI?" is a question a lot of people now type into a search box, and it's one a chatbot can't answer, because it can't listen to the track. I wanted to build a small checker for it, so first I had to find out which detection engine actually works. Vendor accuracy claims don't say how they were measured, so I ran my own test: 8 detectors on the same songs, and 65 songs in total for the one that came out on top.
It's a small test with real gaps, and I list them at the end. But the results were lopsided enough to be worth sharing.
The test set
25 AI songs from 17 acts that the press has identified as AI, mostly released in 2025. Among them: The Velvet Sundown, Xania Monet, Breaking Rust, Cain Walker, Sienna Rose, Aventhis, Solomon Ray, King Willonius ("BBL Drizzy") and TaTa ("Pulse", made by Timbaland's team with Suno). Country, gospel and soul show up a lot, because that's where many of the AI acts in the news are.
40 human songs, picked to be hard. I didn't want an easy pass from acoustic ballads, so the set includes:
- heavy Auto-Tune and vocoders: Cher's "Believe", T-Pain, Bon Iver's "715 - CRΣΣKS"
- electronic production: Daft Punk, Skrillex, Radiohead's "Everything In Its Right Place"
- country, soul and gospel in the same styles as the AI songs: Chris Stapleton, Zach Bryan, Leon Bridges, Kirk Franklin, CeCe Winans
- recent hits a model can't have memorised for long: "Espresso", "Not Like Us", "DtMF", "The Fate of Ophelia"
Audio: the official 30-second preview from Apple Music (AAC 256 kbps). That's what a checker can legally fetch when someone pastes a link, so that's what I tested on.
The pass line was fixed before I ran anything: at least 80% correct, and no more than 10% of human songs flagged as AI, on a core set of 40 songs (20 AI, 20 human). If no engine passed, I wasn't going to launch. Later I added 25 more songs (5 AI, 20 human) to narrow the error range on false alarms.
The results
All engines on the same 40 core songs, each with a simple 0.5 cut-off:
| Engine | Correct | Human songs flagged as AI | AI songs missed |
|---|---|---|---|
| fakeprint (open-source, runs locally) | 95% | 0 of 20 | 2 of 20 |
| Deezer's album AI label (35 songs found on Deezer) | 94% | 0 of 18 | 2 of 17 |
| fakeprint CNN (second model, same project) | 92.5% | 2 of 20 | 1 of 20 |
| Gemini 4 Argon, asked to listen (35 usable replies) | 88.6% | 3 of 19 | 1 of 16 |
| Gemini 3.8 Flash, asked to listen | 82.5% | 1 of 20 | 6 of 20 |
| Hive, general AI-generated audio detection | 62.5% | 0 of 20 | 15 of 20 |
| SONICS 5-second model (ICLR 2025) | 57.5% | 1 of 20 | 16 of 20 |
| SONICS 120-second model | 50% | 20 of 20 | 0 of 20 |
AI or Not would have been number nine, but its music model returned HTTP 403 on the free plan, so it isn't in the table.
A few notes so nobody over-reads this:
- These are 30-second previews of 40 songs, not product reviews. For Hive I could only reach its general AI-audio and deepfake endpoint, not its music-specific model, so the 62.5% doesn't rate that product.
- The SONICS 120-second model expects far more audio than a preview gives it. Feeding it the 30-second clip tiled to length helped (65%, but 9 of 20 human songs flagged). It's a research model trained on Suno and Udio songs from its own dataset, and it shows.
- The Gemini numbers are flattering on the human side. Both models recognise famous songs, so "this is human" on Queen or Eminem may just be memory. Even so, they flagged human songs: Gemini 3.8 Flash called Eminem's "Lose Yourself" AI, and Gemini 4 Argon flagged "Lose Yourself", "Bohemian Rhapsody" and Holding Absence's "Afterlife". Argon also returned unusable replies on 5 of the 40.
- The fakeprint CNN flagged Billie Eilish's "bad guy" and "Bohemian Rhapsody". Strong on AI songs, but that's two famous human songs called AI out of 20.
Then I ran the winner on all 65 songs:
| fakeprint, all 65 songs | Result |
|---|---|
| Correct | 63 of 65 (96.9%) |
| Human songs flagged as AI | 0 of 40 (95% interval 0–8.8%) |
| AI songs caught | 23 of 25 |
| Adding Deezer's album label ("either says AI") | 64 of 65; still 0 of 40 human songs flagged |
| Highest score on a human song | 0.0007 |
| Lowest score on a caught AI song | 0.873 |
That last pair is the part I find most convincing. The scores don't sit near the line. Human songs, including the Auto-Tune and vocoder tracks, were all below 0.001; the caught AI songs were all above 0.87.
Why I care more about false alarms than accuracy
"Accuracy" mixes two very different mistakes, and they don't cost the same.
A missed AI song is a shame. A human song called AI lands on a person: a singer who gets accused in the comments, an indie artist whose track gets pulled from a playlist, a label that has to defend a release. If most songs people check turn out to be human, which seems likely, even a small false-alarm rate does real damage.
Here's the arithmetic with made-up but plausible numbers. Say 1 in 10 songs people check is really AI, and a detector catches 95% of AI songs but also flags 10% of human ones. Out of 1,000 checks it makes 95 correct "AI" calls and 90 wrong ones. Nearly half of its "AI" answers would be accusations against real musicians, even though its headline accuracy looks fine.
Two more things worth knowing:
- Zero isn't zero. 0 of 40 human songs flagged is a good result from a small test, but with only 40 songs the 95% interval still runs up to about 8.8%. Treat any "likely AI" verdict as one piece of evidence.
- Give the detector a way to say "I don't know". The checker I built answers "Likely AI-generated" at 0.8 or above, "Likely human-made" at 0.2 or below, and "Can't tell" in between. None of the 65 Apple previews landed in the middle, but some re-encoded copies did, which is exactly what that middle band is for.
Does compression break it?
I re-encoded every preview three ways and also scored Deezer's own MP3 previews:
| Copy | AI songs called "Likely AI" | Human songs called "Likely human" |
|---|---|---|
| Apple preview (AAC 256k), the main test | 23 of 25 | 40 of 40 |
| Re-encoded to MP3 128 kbps | 21 of 25 | 40 of 40 |
| Re-encoded to Opus 64 kbps | 20 of 25 | 40 of 40 |
| Downsampled to 22 kHz, then AAC 96 kbps | 19 of 25 | 40 of 40 |
| Deezer's MP3 128k previews (58 songs found there) | 18 of 22 | 36 of 36 |
Heavy compression costs some AI catches; those songs mostly slid into "Can't tell". But no human song moved towards AI in any version, so the model is reacting to the generator, not to the codec. One odd case: Breaking Rust's "Livin' On Borrowed Time" scored 0.998 on Apple's preview and 0.0006 on Deezer's. I don't know why; Deezer may carry a different master. That's why the checker fetches Apple's preview first.
How the fingerprint works (and one porting trap)
AI music generators build a song as a compressed code and then turn it into sound with a neural audio decoder. The decoder's upsampling layers leave a row of evenly spaced peaks in the spectrum, roughly between 1 and 8 kHz. You can't hear it, but you can measure it. Researchers at Deezer described this in a 2025 ISMIR paper (Afchar et al., "A Fourier Explanation of AI-music Artifacts").
The open-source fakeprint model (lofcz/ai-music-detector, MIT licence) turns that into a tiny detector: resample to 16 kHz, take an 8192-point FFT, average the spectrum over the clip, subtract its lower envelope so the music's own shape drops out, and feed the remaining peaks to a 15 KB logistic regression. In my runs it took about a tenth of a second per song and cost nothing.
The trap if you port it: the resampler has to match training, which used torchaudio's default sinc filter. A different resampling filter can create spectral peaks of its own, and then you're measuring your resampler instead of the generator. The model's author ships alignment tests for exactly this. I ported it to run in the browser, so uploaded files never leave the device, and checked the port against the Python scores on all 65 songs: the largest difference was under 0.0005, and no verdict changed.
What this test can't tell you
- New generators can slip past it. The model card covers Suno up to v5 and Udio up to v1.5. Suno released v6 on September 9, 2026, after these songs were made, and I haven't tested it, or ElevenLabs Music, Google Lyria or Mureka. Expect misses until it's retested.
- Part-AI songs usually pass as human. TaTa's "Pulse" uses an AI voice with human production and scored below 0.001. Hive, both Gemini models and Deezer's label missed it too. The check answers "was this whole track generated?", not "was AI used anywhere?".
- The licensing has a grey area. The model's code and weights are MIT, but its training data includes the SONICS dataset, licensed CC BY-NC 4.0 for research, plus data the author doesn't publish. Deezer has also filed patent applications on AI-detection methods; I haven't checked whether they cover this one.
- It's small and I picked it. 25 AI songs, all from acts named in the press and mostly from 2025, and 40 human songs I chose. A different set could score differently, and the range on the AI miss rate is still wide (2.2–25%).
- Deezer's label comes from an undocumented interface that can change, and it only applies to whole albums. I also looked at it while choosing AI songs, so its AI-side result isn't blind. Its human-side result (0 of 36 flagged) is.
- I'm not neutral. I built a checker on top of fakeprint and Deezer's label, so I have an interest in good numbers. No vendor paid for this or saw it first. Hive ran on free trial credit and Gemini through OpenRouter at standard prices (about $0.003 per check).
If you want to check one song
- Look at who's behind it. An artist with no live shows, no interviews and a burst of releases in a few months is a reason to look closer, not proof.
- Check the platform's own labels. Deezer tags albums its detector found to be AI; it's worth a look if the song is there.
- Run a detector as one more clue, preferably one that shows its score, its reasons, and is allowed to say "can't tell".
- Don't use a detector's answer alone to accuse a real musician. Not mine, not anyone's.
I put this into a small free checker: paste a Spotify, Apple Music, YouTube or Deezer link, or upload a file, which is analysed in your browser and never uploaded. It's at isthissongai.app (3 free checks a day, no sign-up). Every song, every score, the re-encoding results and the full list of limits are on the accuracy page. If it gets one of your songs wrong, I'd really like to hear about it.
Sources: Afchar et al., A Fourier Explanation of AI-music Artifacts, ISMIR 2025; lofcz/ai-music-detector model card; SONICS dataset.
I used an AI assistant to help draft and edit this post. The test set, the runs and the numbers are from my own test, published in full on the accuracy page.
Top comments (0)