I did not set out to test synthetic-speech detectors. I was validating my own voice pipeline and needed to know whether the processing stage changed what a provenance checker would say about the output.
It did. And it did so on recordings where nothing had been generated at all.
The premise being tested
A synthetic-speech detector is used as though its score answers one question: was this audio generated? Those scores are entering forensic, journalistic and platform-moderation settings on that reading. In those settings a false positive is not a benchmark loss — it is a genuine recording being called fake, with consequences for the person who made it.
Here is the structural problem. Detectors are trained on corpora where synthetic audio and particular signal-processing histories are almost perfectly confounded. Every generated clip in a text-to-speech corpus has passed through a vocoder or a neural codec. Almost no bona fide clip has. A model minimising training loss has no incentive to separate what its training data never separated.
So I ran the confound backwards.
The design
Hold everything fixed — speaker, words, performance, room, microphone. Apply a benign transformation that produces no generated speech tokens. Measure the paired change in a frozen pretrained detector's score on the same utterance.
The transformations are deliberately unremarkable and the statistics deliberately conservative. The question is not whether an adversarial perturbation can fool a classifier. Of course it can. The question is whether ordinary processing of genuine speech is enough to move a deployed one.
Thirteen detectors, declared and hashed before any of them was scored. Four pre-registered experiments on the same 47 utterances from three speakers:
| experiment | transformation | family size | Holm survivors |
|---|---|---|---|
| Detector panel | neural-codec round trip | 13 | 9 |
| Processing 2×2 | denoise / EQ+compression / both | 39 | 30 |
| Reconstruction ladder | Griffin-Lim ×4, BigVGAN, Vocos, 2 rate controls | 78 | 61 |
| Codec family | EnCodec 6 kbps, 12 kbps, DAC 44.1 kHz | 39 | 33 |
Two design decisions were fixed in advance and both matter.
No AUC gate. Running the causal test only on detectors that discriminate on this corpus would be circular — by my own hypothesis, discrimination on this corpus is substantially a measure of codec sensitivity, so gating on it would preferentially admit the detectors most likely to show the effect. Every detector ran on every condition regardless of its AUC.
A negative control on purpose. The panel includes a detector already known to be inverted, so it is visibly not a set of models picked to agree.
Genuine speech, called synthetic, with zero generation
A codec encode-decode of real studio recordings moved 9 of 13 detectors past Holm correction.
Three of those were fully prospective — nothing about their behaviour had been observed when the protocol was frozen — and all three moved on 47 of 47 utterances, at matched-pairs rank-biserial of exactly −1.000, the maximum a rank statistic can reach.
It held across three professional microphones capturing the same physical performance simultaneously, so it is not a capture-channel artefact. It held on a speaker outside the corpus.
And it is not about neural codecs. Griffin-Lim — phase retrieval published in 1984, no neural network anywhere — moved 12 of 13 detectors in the same direction. An ordinary denoise-and-master chain, the kind any production pipeline applies without thinking about it, moved two detectors by −2.95 and −4.22 native units.
The phenomenon is about reconstruction and processing history, not about codecs being special.
There is no best detector to pick
The obvious mitigation is to choose a better detector. The measurements close that door.
| detector family | codec round trip | ordinary denoise + EQ + compression |
|---|---|---|
| codec-trained | collapses | barely moves |
| ASVspoof-era | null | collapses |
The two families fail on opposite inputs. The ASVspoof-era models could not separate real speech from synthetic speech on this material at all — and ordinary mastering still moved them on 41 and 46 of 47 genuine human recordings.
Choosing a detector is choosing which false positive to accept, not avoiding one.
Which transformation moves which detector tracks each checkpoint's documented training exposure across three architectures. I report that as an association among frozen checkpoints, not as a causal effect of training data — the checkpoints differ in frontend construction and optimisation as well as in corpus, and no isolating intervention was performed.
The prediction I got wrong
I predicted that a detector trained on codec audio would resist a codec pass.
It collapsed hardest of anything in the panel.
That prediction is in the frozen pre-registration, so it is in the paper. An independent adversarial audit then ran three rounds against the frozen estate, parsing the stored raw scores directly rather than executing my code. It reproduced every statistic — and thirteen of my own claims were withdrawn as a result. The withdrawals are published, because a corrected claim is more informative than one that was never challenged:
| withdrawn | replaced by |
|---|---|
| "the mechanism is analysis–resynthesis" | no mechanism identified |
| "any operation that discards phase flips these detectors" | the tested operations do |
| "ASVspoof-era detectors are blind to codecs" | insensitive to this codec specifically |
| "Griffin-Lim moves the panel harder than any codec" | EnCodec also reaches median −1.000 |
What this does not establish
- Detectors are not broken in general. The supported claim is narrower: on this material, detector output can be strongly affected by a recording's reconstruction and signal-processing history rather than uniquely identifying synthetic provenance.
- No mechanism is identified. The paper says so explicitly.
- No listening test was run. Nothing here says any transformation is perceptually transparent.
- Three speakers, 47 utterances, neutral speech, one language. Enough for a paired effect. Not a population claim.
- Seven of thirteen detectors come from one research group. That is not seven replications.
Not an evasion method
This is a detector-robustness and forensic-attribution result and I do not present it as anything else. No parameter anywhere in this work was chosen, searched or tuned against a detector output.
The one time an intervention was tuned to a statistic, it made the audio more separable rather than less. That experiment is retained in the paper deliberately, because a failed attempt that points the opposite way to the hypothesis is worth more than a quiet deletion.
The part that generalises past audio
An instrument's score is not interpretable without knowing the provenance of what it was computed on.
Before trusting a detector panel, the question worth asking is which detectors on it can do the task on your material at all — and what else, besides generation, moves them. That is the question the companion tool answers, and it is the reason I wrote it as an instrument for measuring detectors rather than as another detector.
Reproducibility
The deposit is 26 files: the paper, the frozen per-utterance detector scores, the result and pre-registration freeze records, the five figures, and the two scripts that regenerate the figures and the rank-biserial bootstrap intervals. Both scripts were verified to run from the deposit alone and to reproduce every figure and interval file byte-for-byte.
The 11.5 GiB audio evidence estate is not released. Speaker and sample identifiers are retained deliberately as provenance under signed contract; only workstation path prefixes are stripped.
- Article (canonical): https://ai.bedvibe.studio/called-synthetic/
- DOI: https://doi.org/10.5281/zenodo.22819223 — all versions: https://doi.org/10.5281/zenodo.22819222
CC BY 4.0.
Top comments (0)