DEV Community

Daniel Pertu
Daniel Pertu

Posted on

One capital letter turns /pricing into a login form, and /Sitemap.xml into a 404

PubTrivia is one Next.js app doing two jobs. There is a public marketing site of 72 sitemap URLs, including the pricing page, the features index and four free tools, and behind a login there is a dashboard where a venue sets up its tables and runs a quiz night. One proxy decides which of those two you are allowed to see on every request.

I have written before about what that gate does to URLs that do not exist. This post is the same gate measured along a different axis: capitalisation. Take any URL in our sitemap, capitalise one letter, and ask for it.

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/pricing
200

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/Pricing
307 https://pub-trivia.app/login?next=%2FPricing
Enter fullscreen mode Exit fullscreen mode

I ran that over all 72 URLs in the sitemap. Seventy-one have a first segment to capitalise, and all 71 answer the same way: 307, to /login, with the mis-cased path preserved in next.

Why the gate cannot see that it is the same page

The allowlist is an array of strings in lib/routes.ts, and the match is three lines:

function matchesAny(routes: readonly string[], pathname: string): boolean {
    return routes.some(
        (route) => pathname === route || pathname.startsWith(route + '/')
    )
}
Enter fullscreen mode Exit fullscreen mode

'/Pricing' === '/pricing' is false, and '/Pricing'.startsWith('/pricing/') is false. So isPublicRoute('/Pricing') is false, the request is treated as a protected one, and an anonymous visitor is sent to sign in. The allowlist is working exactly as written. It compares strings, and those are different strings.

What makes this interesting is the other matcher in the same request path. Next.js middleware declares which requests it even wants to see, and ours declares it as an exclusion list:

export const config = {
    matcher: [
        '/((?!_next/static|_next/image|favicon\\.ico|manifest\\.webmanifest|robots\\.txt|sitemap\\.xml|.*\\.(?:svg|png|jpg|jpeg|gif|webp)$|api|monitoring).*)',
        '/api/:path*',
    ],
}
Enter fullscreen mode Exit fullscreen mode

That first entry is a negative lookahead: run on everything, except static assets, except the two metadata routes, except image files, except /api, except the Sentry tunnel. If those exclusions were compared the same way our allowlist is, then /Sitemap.xml would not match the exclusion, the proxy would run, isPublicRoute('/Sitemap.xml') would be false, and a crawler asking for the sitemap with a capital S would be asked to sign in.

It is not what happens:

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/Sitemap.xml
404

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/robots.TXT
404
Enter fullscreen mode Exit fullscreen mode

Two different answers for the same kind of typo. The explanation that fits every measurement I have is that the matcher is compiled into a case-insensitive regular expression, so /Sitemap.xml is excluded from the proxy by a pattern that spells sitemap.xml in lower case. The proxy never runs, nothing checks the allowlist, and the request falls through to the router, which has no route of that name. 404.

Three more probes agree with that reading. A nonexistent image is excluded by the extension branch, and a nonexistent text file is not:

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/nope.png
404

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/nope.txt
307 https://pub-trivia.app/login?next=%2Fnope.txt
Enter fullscreen mode Exit fullscreen mode

Our real favicon is at /favicon.ico and the exclusion spells it in lower case, so the capitalised version is excluded from the proxy too, and then found by nothing:

$ curl -s -o /dev/null -w '%{http_code}\n' https://pub-trivia.app/favicon.ico
200

$ curl -s -o /dev/null -w '%{http_code}\n' https://pub-trivia.app/favicon.ICO
404
Enter fullscreen mode Exit fullscreen mode

And the clearest one, because it is the case where the two comparisons disagree inside a single request. /api is excluded from the first matcher entry but matched by the second one, /api/:path*, which exists only so that maintenance mode can answer API routes with a 503. The code that decides whether a path is an API route is the same exact-string prefix match as the allowlist:

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/api/pricing
404

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/API/pricing
307 https://pub-trivia.app/login?next=%2FAPI%2Fpricing
Enter fullscreen mode Exit fullscreen mode

The lower-case one reaches the router and 404s, because that endpoint was deleted a while ago. The upper-case one is matched by the /api/:path* entry, is then not recognised as an API route by our own code, and gets handed to the auth gate like an ordinary page.

The redirect leads nowhere

The 307 would be defensible if signing in got you where you were going. It does not, for two reasons.

The first is that the App Router is case-sensitive in the other direction. Our content clusters are prefixes, so /legal covers /legal/privacy, and a mis-cased child passes the gate and reaches the router:

$ curl -s -o /dev/null -w '%{http_code}\n' https://pub-trivia.app/legal/Privacy
404

$ curl -s -o /dev/null -w '%{http_code}\n' https://pub-trivia.app/guides/How-To-Host-A-Pub-Quiz
404
Enter fullscreen mode Exit fullscreen mode

So capitalising the first segment gets you a login form and capitalising a later segment gets you the 404 you deserved, and the page at the end of the login form does not exist either way.

The second is that nothing reads the next parameter the proxy writes. The login form has never consumed it, so after signing in you land on the dashboard, and the URL you asked for is gone.

There is one more entry on the allowlist where case matters more than it looks. The IndexNow key file is a verification URL, and search engines fetch the exact string we submit:

$ curl -s -o /dev/null -w '%{http_code}\n' https://pub-trivia.app/2382c7da66fd467eb45e9c97aadf605a.txt
200

$ curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/2382C7DA66FD467EB45E9C97AADF605A.txt
307 https://pub-trivia.app/login?next=%2F2382C7DA66FD467EB45E9C97AADF605A.txt
Enter fullscreen mode Exit fullscreen mode

A hex key is the same key in either case to a human reading a dashboard, and it is a different path to our gate. The submitter sends the lower-case form, so this has never broken, but it is one typo in a config field away from a verification failure that reads as "the file is not there" when the file is right there.

The other axis: which methods a page answers

While I had the probe open I sent every method to every sitemap URL. The split is clean, and it is not the one I expected:

Method 70 prerendered pages / and /pricing
GET 200 200
HEAD 200 200
POST 405 200
PUT, PATCH, DELETE 405 200
OPTIONS 204 200
TRACE 405 405

Seventy of our pages are prerendered at build time and served from the edge cache, and the platform answers anything other than GET, HEAD or OPTIONS with a 405. The two that render per request are the homepage and the pricing page, because both read the visitor's country to choose a display currency, and a page that runs on every request runs on every request whatever the method says:

$ curl -s -X POST -o /dev/null -w '%{http_code} %{size_download}\n' https://pub-trivia.app/
200 150474

$ curl -s -X OPTIONS -o /dev/null -w '%{http_code} %{size_download}\n' https://pub-trivia.app/
200 150474
Enter fullscreen mode Exit fullscreen mode

OPTIONS the homepage and you get the homepage: 150 KB of HTML in response to a question about what the resource supports. POST it with a form body and you get the homepage again. Nothing is mutated, there is no action to invoke, and the only cost is a wasted render, so this is a curiosity rather than a hole. It is still the kind of thing I would rather know than not.

The 405s have their own defect. RFC 9110 says a 405 response "MUST generate an Allow header field containing a list of the target resource's currently supported methods", and ours does not:

$ curl -s -I -X POST https://pub-trivia.app/features | grep -ci allow
0
Enter fullscreen mode Exit fullscreen mode

Zero Allow headers, and a content-length: 0 body. A client that wanted to discover what it may do is told no twice and advised nothing.

What I am changing

Not the method behaviour. A static page answering 405 to POST is correct, and the missing Allow header is the platform's to emit, not something I want to synthesise in the proxy for a request nobody makes.

The capitalisation is worth a fix, and the fix is not to lower-case the pathname before the allowlist check, which would quietly make /PRICING a live URL with no canonical story. The right answer for a marketing site is one normalising redirect: if a path differs from a known public route only by case, answer 308 to the canonical lower-case form. That turns a dead login bounce into a working link for the reader and a clean signal for a crawler, and it keeps exactly one spelling of every page indexable.

The lesson I am taking is narrower than "normalise your URLs". Two matchers ran on the same request. One was a regex compiled by a framework with case-insensitivity I did not choose, the other was === and startsWith I wrote myself, and the gap between them is where the behaviour lives. If you have a middleware matcher with exclusions in it, the question worth asking is not what it excludes. It is whether what excludes a path and what then authorises a path agree about what a path is.

Check it yourself

No account needed, and nothing here mutates anything:

# any sitemap URL, with the first letter capitalised
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://pub-trivia.app/Features

# the same typo on a metadata route
curl -s -o /dev/null -w '%{http_code}\n' https://pub-trivia.app/Sitemap.xml

# a method the page does not expect, on a static page and on a per-request one
curl -s -o /dev/null -w '%{http_code}\n' -X POST https://pub-trivia.app/features
curl -s -o /dev/null -w '%{http_code}\n' -X POST https://pub-trivia.app/pricing
Enter fullscreen mode Exit fullscreen mode

The pages themselves are worth more than the status codes: the pricing page prices itself in your own currency, and the free tools run entirely in the browser. The free tier needs no card, so you can sign up and see the other half of the gate from the inside.

Top comments (0)