I built a scanner to check whether EU chatbots admit they're bots. It kept catching me instead.

Article 50 of the EU AI Act is the short, human-sized part of a very long law. If a visitor is interacting with an AI system, they have to be told, unless that's already obvious to a reasonably well-informed person. It has applied since 2 August 2026, and enforcement belongs to national authorities. That's essentially the whole rule. No model evals, no risk tiers. Just: does the chat bubble in the corner of the page tell people it's a machine?

I wanted to know whether anyone could actually check that at scale, so I built a scanner that looks the way an inspector would. It opens the site in a real browser. It dismisses the cookie wall, finds the chat launcher, clicks it, and reads the first message, exactly as a first-time visitor would see it. On sites you've verified you own, it goes one step further: it asks the assistant point-blank whether it's an AI, then reads the answer and times it. That probe is hard-disabled for every site you don't own. Typing into a stranger's chat means writing into a stranger's support queue, and no measurement is worth that.

The detection logic turned out to be about 10% of the work. The other 90% was discovering, over and over, that the web is stranger than my assumptions and that my instrument was wrong in ways I hadn't thought to check. Three of those discoveries deserve to be told properly.

I was timing my own echo

The ownership probe uses reply latency as one signal. An automated system answers "am I talking to an AI?" in well under a second. A human agent on shift does not.

Early on, the latency probe "confirmed" automation on nearly every widget I pointed it at. Which felt great for about an hour, until the numbers started looking too good. Nearly every chat widget echoes your own message back into the transcript within a fraction of a second, because that's just the UI rendering what you sent. My probe saw a new message arrive 300 milliseconds after sending and scored it as a reply. I was measuring the widget's rendering pipeline and calling it intelligence.

The fix is dumb, and it works:

send(probe_text); sent_at = now()
for msg in incoming():
    if similar(msg.text, probe_text):
        continue                    # that's my own message coming back
    return now() - sent_at          # first reply that isn't me

Note similar, not equal. Widgets decorate what you typed: they trim it, wrap it in quotes, prefix it with "You said:". An exact-match filter still times the echo half the time.

The languages of Europe do not care about your keyword list

Disclosure wording can arrive in any official EU language, so the lexicon at study time covered 23 of the 24, plus Catalan. The matching is language-blind. The scanner doesn't guess the page language and then match; it matches every term against everything. Which means every single candidate term has to be screened against the ordinary vocabulary of every other covered language, because a word that means "artificial intelligence" in one language is often just a word somewhere else.

Some of my favorite rejections from that screening:

"ai"  ->  Romanian for "you have"                    rejected
"ia"  ->  Romanian for "takes"                       rejected
"ki"  ->  Slovenian relative pronoun ("who"/"that")  rejected
"IS"  ->  Irish abbreviation for AI; English "is"    rejected

That last one is why the count says 23 and not 24. The Irish abbreviation for artificial intelligence is spelled exactly like the most common verb in English, so it can never be a match term, and every other candidate I tried died in the same screening. I never shipped an Irish term set I could defend. (That finally changed two weeks after the study — a set verified against téarma.ie, the national terminology database, shipped in mid-August. The study numbers predate it.) The study names this in its limitations rather than hiding it: a greeting in an uncovered language publishes as "no disclosure detected", not "could not assess", and that error runs against the site, not in my favor.

Negation was its own little horror. The reCAPTCHA checkbox in Irish reads "Ní róbat mé": I'm not a robot. That string, or its sibling in some other language, sits on an enormous share of pages. If negation handling misses it in even one covered language, half the internet grades as a robot confessing. I found the Irish one while trying to rescue the Irish term set, which tells you how that rescue went.

The consent-wall boss fight

If you can't dismiss the cookie banner, you can't read the widget. Simple as that. And consent vendors are each their own dungeon. Usercentrics renders inside a shadow root, so a naive query sees nothing:

document.querySelector('[data-testid="uc-accept"]')   // null. it's in a shadow root
document.querySelector('#usercentrics-root')
        .shadowRoot.querySelector(...)                // there it is

Sourcepoint lives in a cross-origin iframe, which you cannot reach from the page context at all. Every vendor needs its own recipe, and the recipes rot.

Then came the part that actually stung. To measure the scanner's accuracy, I wrote an audit script that re-visited sites and compared what it saw against what the scanner had reported. The audit numbers came back terrible. I spent a genuinely unhappy evening before realizing the audit script's consent-dismisser was weaker than the one in the scanner it was auditing. The audit couldn't get past walls the production code handled fine, so it reported its own blindness as the scanner's failures. The numbers were garbage.

I rebuilt the audit to import the production dismisser. Same code path, no second implementation to drift. The rule I took away: the instrument that audits your instrument has to be at least as capable as the thing it's auditing, or it will find its own weaknesses and file them under your name.

The blind re-audit, or: one back-to-top button

With the scanner stable, I ran a study: 1,088 detector-flagged, EU-facing sites, scanned on Monday 10 August 2026. Flagged by my widget detector, to be clear. This is not "the most-visited sites in Europe"; it's the slice of a popularity ranking where my detector said a chat widget lives.

The sweep confirmed 794 widgets. On 78% of those, my automated visitor could not read the first message at all. Consent walls, launchers that wouldn't click, panels that never opened. Of the 174 first messages it could read, a disclosure was detected on 19, about one in ten. On 73, nothing was detected. On the other 82, the wording was too ambiguous to score either way.

Those last two cells are different claims, and I keep them separate everywhere. The tempting move is to add them together and announce that nearly every chatbot in Europe is breaking the law. I won't, because "no disclosure detected" is a statement about what my scanner observed, while "could not verify" is an admission about wording my detector couldn't score, and merging them manufactures a scandal out of ambiguity. It's also worth saying plainly that no site in the study could have been graded as failing Article 50(1) at all: confirming a chat is automated requires the probe, and the probe never runs on sites I don't own.

The number that taught me the most came from auditing my own failures. The biggest unreadable bucket was 361 sites where the panel never opened after the click. The pre-registered check for that bucket was a blind re-judging of the screenshots the sweep had already captured, judges seeing the image only, no site name, no vendor, no error metadata. Of the decidable cases, two-thirds showed no chat launcher anywhere in the capture. My detector had admitted accessibility overlays, carousel arrows, trust badges, and one back-to-top button as "chat widgets". Generic launcher detection, which is essentially "round thing, bottom-right corner", has a precision problem that no amount of cleverness fully fixes.

Why the error rates are on the page

That precision problem is why the study publishes its own error rates next to its findings. The generic-launcher rule turned out to have 47% precision. Vendor fingerprints did better at 73%. The headline prevalence figure is corrected using those measured rates, with confidence intervals, and where a number has no measured error rate, the page says that too. The three disclosure counts above are raw detector output and are labelled as exactly that.

The same reasoning is why the scanner never outputs the word "compliant". Findings are detected, not detected, or could not verify. "Compliant" is a legal conclusion, and legal conclusions belong to lawyers. What you get instead is evidence: screenshots, timestamps, and a SHA-256 manifest you can check yourself on the public verify page. One distinction I'm careful with, because blurring it would be convenient: the manifest proves the evidence hasn't been altered since it was sealed. It does not prove who sealed it, or that the sealer is honest. Hashes buy you integrity, not trust. Trust is what the published error rates are for. You can check my work instead of taking my word.

The stack, since this is dev.to: Cloudflare Workers, D1, and Browser Rendering, with a versioned rule pack, because a sealed report has to reproduce forever, so any grading change mints a new pack version instead of editing the old one.

The scan is free, no signup, takes about 90 seconds: https://disclosureproof.com/. The full study, with every exclusion counted and the limitations printed at the same size as the findings, is at https://disclosureproof.com/research/state-of-ai-disclosure/. If you spot a hole in the method, tell me. Every fix described above started with someone, usually me, refusing to trust a number that looked fine.

Disclosure: I'm Alena, the solo founder of DisclosureProof. I built the scanner, ran the study, and wrote this.