Your server logs show requests to an article. Your browser analytics dashboard shows almost nothing for that page. Which one should you trust?
Start with what each system measures. A request log records requests at a particular point in your infrastructure. A browser analytics tag depends on code running and a measurement reaching its collection endpoint. Those are different events.
I work on WebDecoy, where we build tools for investigating bot and AI traffic. Here is a debugging sequence you can use with your existing logs and analytics, before adding another tool.
1. Match the scope before comparing counts
Choose one hostname, one page and a short time window. Write down the timezone for both reports.
For example, compare requests for /pricing/ between 14:00 and 14:15 UTC with browser events for that same page and interval. This is a hypothetical example, not a customer measurement.
Check whether one report combines subdomains, includes query strings, or normalizes trailing slashes differently. Separate HTML requests from assets and API requests. Check status codes too: a redirect, an error response and a successful HTML response do not produce identical browser behavior.
Even after aligning the scope, a request count and a pageview count are different units. Do not expect them to match exactly.
2. Identify where the request was observed
A CDN can serve a cached response without contacting your origin. An application-level logger may therefore miss traffic that an edge log records.
Write down the observation point for each source:
| Source | What it can establish |
|---|---|
| Edge request log | A request reached that edge and received the recorded response |
| Origin/application log | A request reached that origin or application handler |
| Browser analytics event | The instrumentation sent an event that the analytics system recorded |
Check sampling, retention and filtering before treating any of these as a complete inventory.
3. Inspect the HTML that a client actually receives
For a page you own or are authorized to test, inspect an ordinary response:
curl --silent --show-error --location \
--dump-header /tmp/page-headers.txt \
--output /tmp/page.html \
'https://your-site.example/pricing/'
Replace the example address with your own page. This command downloads the response; it does not execute the page's JavaScript.
Compare the saved HTML with the rendered DOM in your browser. Is the main text present in the response? Does it appear only after JavaScript runs? Is the analytics tag included on this route?
Google's JavaScript SEO documentation distinguishes fetching HTML from rendering JavaScript and notes that not all bots can run JavaScript. Avoid assuming every crawler behaves like Googlebot, or like the browser you use for testing.
4. Test the browser measurement path separately
In your browser's network panel, inspect whether the expected analytics request is sent. Then use your analytics provider's debugging view, where available, to check receipt.
Consent choices, content blockers, network failures and missing route instrumentation can all affect measurement. A missing browser event does not, by itself, identify a bot.
Label your own setup tests and exclude them from claims about real visitors. A successful test proves that the tested path reports under those conditions.
5. Keep identity claims separate from request evidence
A user-agent string is a client's claim. A data-center IP describes infrastructure. Neither establishes that a particular AI provider made the request, that the traffic is malicious, or that a human was absent.
Use the identity-verification method documented by a provider when available. Preserve the distinction between a declared crawler and a verified one. For other traffic, inspect request patterns and relevant session evidence before deciding how to respond.
Also keep crawls, citations and referrals separate. A page fetch does not prove that an assistant cited it or sent a person back to your site.
Turn the discrepancy into a useful finding
A useful conclusion looks like this:
In this window, the edge recorded HTML requests that were absent from the browser event stream. We checked the hostname, path, response codes and browser instrumentation. Request identity remains unverified.
That is evidence you can investigate. It gives you a better next step than classifying every unmatched request as an AI bot.
If you want to inspect bot and AI traffic through WebDecoy, choose an integration that covers the traffic you need to observe. Browser instrumentation covers executing clients; server and edge integrations cover different request paths. Explore WebDecoy and the available integrations.
Follow WebDecoy here for practical detection diagnostics. If you have a measurement mismatch, share the observation points and the counts' definitions in the comments, with private data removed.
References
- Google: Understand JavaScript SEO basics
- Cloudflare: Web Analytics FAQs, including the distinction between JavaScript beacon analytics and edge analytics.
Drafted with AI assistance and reviewed for the scope of each technical claim. No production accuracy benchmark or customer outcome is claimed here.
Top comments (2)
요청이 관측된 위치와 브라우저 이벤트 수신을 별개 증거로 둔 구분이 좋네요. 같은 URL과 시간대에서 동의 허용·거부, 캐시 적중 여부를 나눠 표시한 작은 재현 표를 만들면 어느 구간에서 차이가 생겼는지 좁히기 쉬울 것 같습니다. 이 시험 트래픽을 실제 방문자 분석과 분리한다는 전제도 중요하고요.
That consent × cache matrix would make the diagnosis much easier to reproduce. I would record four cases for the same URL: consent allowed/denied, each with a cache hit/miss, and separate columns for edge request, origin request, browser send and analytics receipt. Capture the actual cache status rather than assuming a repeat request was a hit. Missing events in a denied-consent case should stay out of bot classifications. Thanks for the concrete suggestion.