DEV Community

Cover image for The honeypot that only caught me: when a spam trap fakes success
Jaafar Abazid for ManTek Technologies

Posted on Originally published at mantek.io

The honeypot that only caught me: when a spam trap fakes success

The spam got through every time. The only message our spam trap ever stopped, as far as anyone can tell, was mine.

For 100 days, from the day the contact form on mantek.io went live in May until the end of August, it carried a honeypot: a hidden field that people never see and bots, supposedly, cannot resist. When that field came back filled, the server assumed a bot and answered the way honeypots usually do, with a cheerful success, so the bot would not try again.

It never caught the spam it was built for. And when I finally tested the form myself, it caught me.

The trap

This is what sat at the top of the form, before the name field:

<div class="absolute left-[-9999px] top-[-9999px]" aria-hidden="true">
  <label>
    Company website
    <input type="text" name="company_website" tabindex="-1" autocomplete="off" />
  </label>
</div>
Enter fullscreen mode Exit fullscreen mode

And this is the first thing the server did with every submission:

// Honeypot: bots fill hidden fields. Pretend success so they don't retry.
if ((data.company_website || "").trim()) {
  return respond(wantsJson, true, "Thanks! Your message is on its way.", 200);
}
Enter fullscreen mode Exit fullscreen mode

Every line of that is standard advice. The field is pushed 9,999 pixels off the page rather than hidden, so a bot reading the HTML sees an ordinary input. tabindex="-1" keeps keyboard users from landing in it, aria-hidden keeps screen readers away, and autocomplete="off" asks the browser to leave it alone. On the server, the check runs before validation and before anything is sent, and it answers with the same success message a real sender gets, so a bot learns nothing.

Hold on to that last part. It is the whole story.

What the spam actually looked like

On 8 August a message arrived in Lithuanian: "Sveiki, aš norėjau sužinoti jūsų kainą", "Hello, I wanted to know your price". On 11 August it arrived again: the same name, the same Gmail address, the same organisation, the same message, byte for byte. Only the "I'm interested in" dropdown had changed, from NUZ early access to Partnership. On 29 August came a third, with a new name, an address ending in .cim, and a link.

A person does not resend the same sentence three days later against a different option. A script walking the dropdown does. The first two were probes, checking that the form reached a real inbox without bouncing. The third was the payload.

And all three reached the inbox, which means the trap was empty all three times. Whatever sent them never filled the hidden field. That is what you would expect from a script that builds the request itself, from the field names it wants, and posts it straight to the endpoint without ever loading the page. A bot that renders the form and fills everything it finds would have walked into the trap. This one never saw it.

I cannot prove the script never loaded the page. But everything points that way, and it turned out to matter more than anything else here.

Then it caught me

On 30 August I was testing the replacement on my laptop. The first submission failed for a boring reason: there was no email key on my machine, so the server stopped just after the new verification step. The second came back with the success message.

Nothing had been sent. The log showed a success like any other. What gave it away was how long it took:

Submission Response Time
First 500, stopped at the missing email key 484 ms
Second 200, "Thanks! Your message is on its way." 3 ms

The first request spent its 484 milliseconds calling Cloudflare's verification service across the network. Three milliseconds is not enough time to call anything. A response that fast never left the machine, and there was exactly one path in the function that could answer success without a network call: the honeypot.

So my own browser had filled a field I could not see.

Why Chrome filled a field nobody can see

At the time I blamed autofill and moved on. Writing this, I checked, because "autofill did it" was a conclusion drawn from a timing, not something anyone had seen. The form clears itself after a success, so the value was gone before I could look.

I rebuilt the old form exactly, on a page that shows what would have been sent instead of sending it, and filled it the way a customer would, by picking Chrome's saved details:

The trap Where it sat What Chrome did
"Company website", off-screen, as on our form first field filled it with my saved company, and left the visible Organisation field empty
the same last field left it empty, and filled the visible field
the same, hidden with display:none first field left it empty
a neutral name and label ("Leave this empty"), off-screen first field left it empty

Chrome read "Company website" as the company field. Because it came first, ahead of the real "Organisation / website" field, Chrome put my saved company name into the trap and left the visible field blank. From where I sat, the form looked exactly as it should: my name, my email, an optional field left empty. There was nothing to notice.

autocomplete="off" made no difference. Neither did pushing the field off-screen: to the browser, an input 9,999 pixels to the left is still an input on the page. It took two things together, a label that sounds like something the browser has been saving for you, and a position ahead of the field where that something belongs.

This is one browser with one saved profile. I have not tested Safari, Firefox or password managers, and I would expect them to differ in the details. The shape is what matters: a trap that looks like real data will sooner or later be filled by something that means well.

The honeypot sorted its two visitors the wrong way round
Top: the spam never contained the trap field, because the script built its own request and posted it straight to the endpoint, so the check passed and every message reached the inbox. Middle: the one visitor the trap caught was a real person, whose browser filled it, and who was then told the message was on its way. Bottom: the replacement does not guess from a hidden field; it verifies every request inside the function that receives it.

The fake success was the real bug

The honeypot did not fail by catching a person. Every spam filter catches a person eventually. It failed because of what it did next: it told that person their message was on its way, and dropped it without a trace. No error for the visitor, no log line for me, nothing in the inbox to miss.

That fake success was designed for bots, so they would not retry. The same line meant to fool a bot fooled the one visitor it caught. It is the same shape as the release with no title and the save that went nowhere in The 404 that wasn't: an answer that looks exactly like success, from a path that did nothing.

We know of no enquiry lost in those 100 days. That sentence is less comforting than it sounds, because the path that would have lost one is the path that leaves no record. The only reason we know about my own test is that I happened to be watching the timings.

What replaced it

Turnstile, Cloudflare's alternative to a CAPTCHA, running in its invisible mode, so a person only ever sees it if Cloudflare decides they need to prove something.

The detail that matters is where it is checked. Cloudflare's quick-start setup deploys a separate verification service that the page calls before it posts the form, which leaves the form's own endpoint checking nothing. That would have done nothing here, because our spammer never used the page. It posted straight to the endpoint, so a check the page runs is a check it never meets.

So the verification runs inside the function that receives the form, on every request, before anything is sent. A request without a valid token is refused, whoever sent it and however it arrived. I tested production with the spammer's exact request shape, and it came back 400. Since it went live on 30 August, no messages like those three have arrived.

It costs one thing: the form no longer works without JavaScript, because Turnstile needs it to issue a token. We accepted that, and the honeypot went at the same time rather than staying on as a second layer. The only thing we ever saw it do was drop a real message.

If your site has a honeypot

Plenty of WordPress form plugins offer one, and plenty of sites hand-roll one, as we did. I have not tested the plugins, so I will not name any. These checks work on all of them:

  1. Fill your own form with autofill. Not by typing: pick your browser's saved details, the way a customer would, and check what arrives.
  2. Find out what the trap does when it trips. If the answer is "shows success and drops the message", change it. Log the hit and return the same rejection a failed check would get, so a false positive is at least visible.
  3. Name it like nothing. A label and a field name that sound like real data (company, website, phone, address) are exactly what autofill looks for. The neutral one stayed empty in our test.
  4. Mind the trade-off in how you hide it. display:none kept Chrome out, where the off-screen field did not. Off-screen positioning is usually recommended because some bots skip fields hidden with display:none. That is the trade: the technique meant to fool more bots is the one that fooled Chrome.
  5. Watch the timings. A success that returns faster than your slowest real step, sending an email or calling an API, did not take that step.
  6. Check where the data arrives, not where the page is. If your spam posts straight to the endpoint, anything that runs in the browser is decoration.

The part that stays with me

Every line of that trap was good advice on its own. Hide it from people, keep it from screen readers, ask the browser not to fill it, and let the bot think it won. Put together, they built something that could quietly lie to a real customer, while the spam it was built for walked straight past.

The honeypot was not wrong about bots. It was wrong about who else fills in forms.

Top comments (0)