Googlebot lives in a datacentre, and it was indexing the wrong currency

Here is a bug that never shows up in your browser, never throws, never appears in your error tracker, and quietly damages the thing you spend the most money on.

We charge in GBP. We localise the displayed price by reading x-vercel-ip-country on the server and converting, so a visitor in Amsterdam sees euros and a visitor in London sees pounds. Straightforward, and it works.

Then it occurred to me to ask what Googlebot sees.

Crawlers have an IP address too

Googlebot is not in your user's country. It is in a datacentre. So is Bingbot, and so is the Slack unfurler, and so is whatever fetches your Open Graph card when someone posts your link on LinkedIn. Every one of them hits the geo branch and gets a currency chosen by the location of a server rack.

The consequences are not subtle:

• Search results. Google indexes and can surface the price it saw. A UK business with pound prices gets a dollar figure in the SERP snippet.
• Structured data. If you emit Product / Offer JSON-LD, the priceCurrency you hand the crawler is the one you are declaring. Declaring USD while charging GBP is not a rendering quirk, it is a wrong claim about your own product.
• Link previews. Someone shares your pricing page in a Slack channel and the unfurl quotes a currency nobody involved uses.
• Cache poisoning, if you are unlucky. Any layer that caches by URL and not by country will happily serve a crawler-shaped response to a human.

And critically: you will never see it. Your browser is not a crawler. Your staging environment is not a crawler. The only way you find this is by asking the question.

The fix is four lines

One shared predicate, used by everything that branches on geography:

export const BOT_PATTERN =
  /vercel|bot|crawler|spider|googlebot|bingbot|slackbot|twitterbot|facebookexternalhit|linkedinbot|whatsapp|headless/i

export function isBotUserAgent(userAgent: string | null | undefined): boolean {
  if (!userAgent) return false
  return BOT_PATTERN.test(userAgent)
}

And then the currency resolver refuses to guess for a crawler:

export async function getDisplayCurrency(): Promise<DisplayCurrency> {
  const headerList = await headers()

  // Crawlers are geolocated wherever their datacentre is, so Googlebot would
  // index dollar prices for a business that charges in pounds.
  if (isBotUserAgent(headerList.get('user-agent'))) {
    return BASE_CURRENCY
  }

  const country =
    headerList.get('x-vercel-ip-country') ??
    (await cookies()).get(COUNTRY_COOKIE)?.value ??
    null

  return currencyForCountry(country)
}

BASE_CURRENCY is GBP, which is the amount that actually gets debited. That is the whole principle: when you cannot know who you are talking to, quote the currency you actually charge in. Not the most common one, not a guess from the IP, the real one.

The second caller, and why this lives in its own file

The same predicate runs in our middleware, for a completely different reason:

const isBot = isBotUserAgent(userAgent)

if (isBot) {
  // Bots can see public pages but still need auth for protected routes.
  // Skip all session and device tracking for bots.
  return NextResponse.next()
}

We enforce a two-device-per-account limit, which means every authenticated request does device bookkeeping. A crawler following a link into the app would otherwise mint device rows for nobody. Skipping that for crawlers is both cheaper and more correct.

Two callers, two motivations, one answer to "is this a robot". That is exactly the situation where the predicate belongs in a shared module rather than inline in each. If they drift, one of them is wrong and nothing tells you which.

Yes, user-agent sniffing is unreliable. Use it anyway, here.

The usual objection: user agents are self reported and trivially spoofed. True, and it does not matter for this.

Think about what each kind of error costs you:

• False positive (a human whose UA contains "bot"): they see the real charge currency with correct small print. Mildly less convenient. Nothing is broken.
• False negative (a crawler you did not match): you are back to today's behaviour, which is what you had before the fix.
• Adversarial spoofing: someone deliberately sets a Googlebot UA in order to... see your prices in pounds. There is no attack here. Nothing is gated on this.

This is a presentation heuristic with a safe default in both directions. That is the specific shape of problem where UA sniffing is completely fine. The rule of thumb I use: sniff the user agent when being wrong costs you a slightly suboptimal render, never when being wrong costs you access control.

Go and check yours

This takes thirty seconds and you may not like the answer.

# What a human sees
curl -s https://your-site.com/pricing | grep -oE '[£$€¥][0-9,.]+' | sort -u

# What Google sees
curl -s -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  https://your-site.com/pricing | grep -oE '[£$€¥][0-9,.]+' | sort -u

Run the same pair against cogniprep.app/pricing and you will get a different answer from each: your local currency in the first, GBP in the second, because GBP is what the card is actually charged.

Better still, do all three:

1. Load cogniprep.app/pricing in a browser. Local currency, with the conversion rate spelled out in the footnote.
2. Load it again behind a VPN in another country. Different currency, correct in the first paint.
3. Now curl it with the Googlebot user agent. Pounds, every time, from anywhere.

If your own site returns the same currency to a human and a crawler and you did not write code to make that happen, check whether you are localising at all. If it returns different currencies and you did not write code to make that happen, you have this bug.

The general version of the lesson

Anything you branch on that comes from the network, rather than from the user, has a third case beyond "the values I expected". Geo IP has crawlers. Accept-Language has crawlers. So does time zone, if you infer it server side.

Ask, for every such branch: what does this do when the requester is a robot in Virginia? If the answer is "publishes a claim about my pricing that is not true", that is worth four lines of regex.