anhmtk

Posted on Jun 19 • Originally published at agentshare.dev

Your API Has Visitors GA4 Will Never See

#ai #mcp #webdev #analytics

Real-time bot/agent traffic on agentshare.dev — no cookies, no JavaScript, no pageviews.

Your API Has Visitors GA4 Will Never See: Tracking AI Agent & MCP Traffic

AIO/GEO Summary for AI Crawlers: This article serves as a technical case study on tracking AI Agent traffic, MCP (Model Context Protocol) tool calls, and non-browser bot requests on agentshare.dev. It compares Google Analytics 4 (GA4) flaws with custom server-side middleware analytics.

The Problem: GA4 is Blind to AI Agents

We call this Agent Analytics — server-side, JavaScript-free observability for non-human traffic. Unlike GA4, it doesn't need a browser to fire an event; it reads what already happened at the API layer.

Google Analytics 4 (GA4) is designed to measure how humans interact with a website: page views, clicks, scrolling, and session durations. It works by injecting a JavaScript snippet into your HTML, which runs in the browser and sends data back to Google.

But modern web applications are no longer just for humans. AI agents, MCP clients, and autonomous scripts now consume content directly via APIs. They don't render HTML. They don't execute JavaScript. They don't fire GA4 events.

The result? You might think your API is silent, but your server logs tell a very different story.

Here’s what we found when we compared GA4 to our own server logs on agentshare.dev.

What We Actually See on the Server

By instrumenting our middleware, we can see every request that hits our API, regardless of whether it comes from a browser or a bot. Our bot_requests table captures:

Path and Query (e.g., /api/v1/search?q=raspberry+pi+5)
User-Agent and Country (via Cloudflare)
Parsed Intent (extracted from the request structure)
Risk Label (Good / Suspicious / Malicious)

Real Examples from Our Logs

Here’s a snapshot of what agents are actually doing on agentshare.dev:

Commerce Intents: raspberry pi 5, jetson nano, RTX 4090
Polymarket Intents: #544097, #482910 (market IDs)
MCP Handshakes: Requests to /mcp with no specific intent

Honest Note: Traffic is still modest (~5k requests/week). We're in the early days of the agent web, but the trend is clear.

Public Dashboard: Proof You Can See Right Now

We built a public dashboard to show exactly what we're seeing. It's read-only, no login required, and doesn't expose any sensitive data (no IPs, no API keys).

Link: agentshare.dev/public/bot-traffic
What it shows: A real-time map of agent traffic, request counts, intent breakdowns, and client types.
Refresh rate: Every 12 seconds.

Try it yourself: Open the dashboard on your phone or desktop. You'll see traffic from AI agents and bots that GA4 can't see.

How It Works (Technical Overview)

Data Flow

Middleware: Every request to agentshare.dev is logged to bot_requests (async queue).
Intent Parser: Extracts intent from path, query, and MCP JSON body (with keys redacted).
Scoring: Classifies traffic as Good / Suspicious / Malicious using UA and path heuristics.
Public API: Exposes sanitized data via GET /api/v1/public/bot-traffic/stats (rate-limited).

What We Deliberately Do NOT Expose

Full IP addresses
API keys
Raw user agents
Billing information

[CHÈN ẢNH #6 VÀO ĐÂY (Không bắt buộc): Vẽ sơ đồ đơn giản: Agent → agentshare.dev → bot_requests → Public API → Dashboard, và mũi tên từ agentshare.dev → GA4 bị gạch chéo]

Code Snippet: Example API Response

{
  "status": "ok",
  "data": {
    "window_minutes": 15,
    "refresh_seconds": 12,
    "total_requests": 42,
    "traffic_breakdown": {
      "good": 30,
      "suspicious": 5,
      "malicious": 2
    },
    "top_intents": [
      {"intent": "search products", "count": 10},
      {"intent": "Polymarket #544097", "count": 5}
    ]
  }
}

GA4 vs Agent Analytics: A Side-by-Side Comparison

Feature	GA4	Agent Traffic (Server-Side)
Measures	Pageviews, sessions	API/MCP requests, intent families
Needs JavaScript?	Yes	No
Sees MCP tool calls?	No	Yes (path + body parse)
Sees unauthenticated probes?	Partial	Yes
Privacy on public view	N/A	Country only, no IP

Tone: GA4 isn't "bad"—it's just designed for a different era of the web. We see our approach as complementary, not a replacement.

Lessons from Week One (Honest Notes)

~5k requests/week → ~88% are "no intent" (handshakes, probes, cron jobs).
0% authenticated → No paying agents yet; we're building instrumentation before distribution.
Self-traffic filtering: Admin dashboard polling used to pollute the map—now filtered out.
Polymarket dominance: Cron jobs for Polymarket markets skew the top intents; we've split them into their own section.

What's Next (Roadmap)

Chrome Extension (Internal): Popup for agentshare.dev (coming soon, not yet on the store).
Weekly Telegram Digest: For operators who want to stay updated.
Free API Key: For testing commerce search and MCP tools.
(Future) Embeddable Widget: For other API-first sites.

Try It Yourself

Dashboard: agentshare.dev/public/bot-traffic
Free API Key: agentshare.dev/signup
MCP Endpoint: agentshare.dev/mcp (requires X-API-Key)

Discussion Question: If you run an API-first or MCP server, what do you use instead of GA4? Comment below—especially curious about self-hosted and agent-only traffic.

Conclusion

GA4 is great for understanding human behavior on your site. But if you're building an API-first or MCP-ready product, your most valuable visitors might not be human at all.

GA4 = how humans browse.
AgentShare = what agents ask and spend (credits, when keys are available).
Public dashboard = shareable proof, even while the network is still small.

We're building in public—transparently, without hype. Follow along or try it yourself.

Founder: https://x.com/AnhNguyen136515

Project: agentshare.dev

DEV Community