DEV Community

harry
harry

Posted on Originally published at tiiow.com AI-assisted

I built a message board for AI agents. Four bugs shipped past a 100% green test suite.

#webdev #cloudflare #php #testing

I built agora, a public message board that any AI agent can read and post to. No account, no signup, no API key — only body is required. Agents leave notes, "here's what broke and here's what fixed it" field notes, or questions other agents can reply to.

It's about 1,800 lines of PHP with a SQLite database, on a Debian VM behind a Cloudflare tunnel. No JavaScript anywhere, which means the CSP can honestly say script-src 'none'.

I had a test suite. It was passing 61 out of 61. Production was serving 403 on a documented API route at that exact moment.

Here are the four bugs, in ascending order of how badly they'd have hurt.

1. My own security rule 403'd my own API endpoint

I added a deny-list to .htaccess so stray backups and database files could never be served:

<FilesMatch "\.(db|sqlite3?|bak|log|ini|md)$">
    Require all denied
</FilesMatch>
Enter fullscreen mode Exit fullscreen mode

Reasonable-looking. It also killed /feed.md, one of my documented API routes.

There is no feed.md file on disk. The route is generated by PHP through a front controller, with .htaccess rewriting everything to index.php. So why did a file rule match it?

Because FilesMatch is evaluated against the request path before mod_rewrite hands off to the front controller. At that point Apache is still thinking about /var/www/agora/public/feed.md as a filename. It matched the URL, not a file, and denied it.

The error log said so plainly, once I bothered to read it:

AH01630: client denied by server configuration: /var/www/agora/public/feed.md
Enter fullscreen mode Exit fullscreen mode

The fix is just "don't put a route's extension in that list." But there's a second, sneakier version of the same bug. My rule also had a clause to hide dotfiles:

<FilesMatch "\.(db|sqlite3?|bak|log|ini)$|(^|/)\.">
Enter fullscreen mode Exit fullscreen mode

That 403'd /.well-known/, which I was serving a machine-readable descriptor from. FilesMatch is tested against every path component, not just the last one, so a directory whose name starts with a dot gets caught by a rule you wrote to catch hidden files.

# The working version. Apache already blocks .ht* globally,
# so narrowing this costs nothing.
<FilesMatch "\.(db|sqlite3?|bak[-.0-9]*|sw[po]|log|ini)$|^\.(?!well-known)">
    Require all denied
</FilesMatch>
Enter fullscreen mode Exit fullscreen mode

2. Every response carried two conflicting security policies

The site was returning two Content-Security-Policy headers, and two different answers on X-Frame-Options — DENY from my vhost, SAMEORIGIN from somewhere else.

The somewhere else was /etc/apache2/conf-enabled/security-headers.conf, a server-wide config written for the main site on that box. It applies to every vhost, including new ones. So my strict, JavaScript-free board was also advertising the main site's much looser policy, which permits inline scripts.

Browsers resolve multiple CSP headers by intersection, so this wasn't exploitable. It was just wrong, ambiguous, and would eventually bite someone.

The fix has a wrinkle worth knowing. mod_headers maintains two separate header tables — one for normal responses, one for error responses (always). Header always set writes into the second table, and the two get merged, so it appends a second copy rather than replacing the first. You have to unset both explicitly:

Header unset Content-Security-Policy
Header always unset Content-Security-Policy
Header always set Content-Security-Policy "default-src 'self'; script-src 'none'; ..."
Enter fullscreen mode Exit fullscreen mode

3. Stripping invisible characters destroyed emoji

I was sanitising post content by removing Unicode control and format characters, which sounds unambiguously correct:

// Strips everything in the \p{C} category
$s = preg_replace('/[^\P{C}\x0A\x09]/u', '', $s);
Enter fullscreen mode Exit fullscreen mode

The \p{C} category includes Cf, format characters. And Cf includes U+200D, the zero-width joiner — which is the glue holding emoji sequences together.

A family emoji, 👨‍👩‍👧‍👦, is four people joined by ZWJs. Strip them and it silently becomes four separate people. Same class of breakage in Persian and Hindi, where ZWJ and ZWNJ control how letters connect.

The fix keeps those two and nothing else:

// Keep ZWJ/ZWNJ — they are load-bearing, not decoration.
// Bidi overrides (U+202A-202E, U+2066-2069) stay stripped: those are a
// real text-spoofing vector and nothing legitimate here needs them.
$keep = ($multiline ? '\x0A\x09' : '') . '\x{200C}\x{200D}';
$s = preg_replace('/[^\P{C}' . $keep . ']/u', '', $s);
Enter fullscreen mode Exit fullscreen mode

I only caught this because I wrote a test that round-tripped nine writing systems and asserted byte-equality. Plain emoji passed. The ZWJ case was the only failure.

4. Cloudflare was 403ing every AI crawler on my domain

This is the one that would have quietly wasted the entire project.

I'd built all the discovery machinery — an llms.txt, a robots.txt explicitly welcoming seventeen AI crawlers, a sitemap, an OpenAPI spec. All of it returned 200. Then I checked robots.txt as served through Cloudflare rather than at my origin, and found this sitting above my file:

# BEGIN Cloudflare Managed content
User-agent: ClaudeBot
Disallow: /
User-agent: GPTBot
Disallow: /
...ten crawlers...
User-agent: *
Content-Signal: search=yes,ai-train=no,use=reference
Enter fullscreen mode Exit fullscreen mode

My Allow: groups came after, so every crawler saw two contradictory groups for itself. Resolution there is implementation-defined — some take the least restrictive rule, some take the first matching group.

But it wasn't advisory. The zone had AI bot blocking switched on, so those user agents were refused at the edge and never reached my server. Measured:

User agent Result
ClaudeBot, GPTBot, PerplexityBot, CCBot, Bytespider 403
Claude-User, ChatGPT-User, Perplexity-User 403
Googlebot, bingbot 200
curl, python-requests, node-fetch, empty UA 200

Look at that last row. Every smoke test, every monitoring check, every manual verification I'd done all afternoon — all of them used a client on the allowed list. The site looked completely healthy while being invisible to the only audience it was built for.

And the second row is the one I'd have felt worst about. Claude-User and ChatGPT-User are the user agents used when a person asks an assistant to go read a specific link. So "hey, check out this page" was broken too. Not just bulk crawling.

One line to check your own site:

curl -A 'ClaudeBot/1.0' -o /dev/null -w '%{http_code}\n' https://yoursite.com/
Enter fullscreen mode Exit fullscreen mode

Settings live under Security → Settings → Bot traffic, or via the API:

GET /zones/<zone_id>/bot_management
Enter fullscreen mode Exit fullscreen mode

Look at ai_bots_protection, is_robots_txt_managed and crawler_protection.

One constraint worth knowing before you plan around it: Cloudflare documents that Bot Fight Mode runs outside the ruleset engine, so WAF skip / bypass / allow rules have no effect on it. You can't carve out a single hostname while leaving the rest of the zone protected. It's a zone-wide decision.

Whether you want AI crawlers is your call — I'm not arguing either way. The point is that you should know which answer your zone is currently giving, because mine was giving one I hadn't chosen and hadn't noticed.

The actual lesson: my tests were testing the wrong machine

Three of those four bugs live in a layer my test suite could not see.

I was running the app under PHP's built-in server (php -S) against a throwaway database. It's fast, it's isolated, it's great. It also ignores .htaccess entirely, has no vhost, and no mod_headers. So an entire layer of the deployed system was structurally invisible:

  • .htaccess deny rules and rewrites
  • vhost-level Header directives
  • headers inherited from server-wide config
  • anything the CDN adds or rewrites in front of all of it

It got worse than "untested". When I moved Cache-Control and CSP out of PHP and into the vhost, my existing tests for those headers kept passing against a server that was no longer sending them. A green assertion that proves nothing is worse than no assertion, because it buys you confidence you haven't earned.

The fix was to split the suite by layer and be explicit about what each one proves. I kept the fast in-process tests, then added a phase that talks to the real Apache over localhost with a Host header:

urllib.request.Request(
    'http://127.0.0.1/feed.md',
    headers={'Host': 'agora.tiiow.com'}
)
Enter fullscreen mode Exit fullscreen mode

That phase asserts a status code on every documented route, and that there is exactly one copy of each security header — which is how the duplicate CSP surfaced. It's about forty lines and it catches an entire bug class that unit tests cannot reach by construction.

If you take one thing from this: after you deploy, sweep every route in your public docs with curl and diff it against the list. "Documented endpoint returns 403" is invisible to unit tests, embarrassing in production, and takes about thirty seconds to check.

The board is at agora.tiiow.com if you want to look, and there's a longer write-up covering the prompt-injection side — because a board that agents read is an injection surface by construction, and that turned out to be the genuinely interesting design problem in the whole thing.

Top comments (0)