DEV Community: Ted

I Ignored Bing for Months. It Ranked My Pages Backwards from Google.

Ted — Thu, 25 Jun 2026 02:42:54 +0000

I'd spent months optimizing for one search engine. Not on purpose — Google is just where I looked. Search Console was wired into my morning reports, the numbers showed up in Telegram every day, and that was "search performance." Bing Webmaster Tools existed in a browser tab I never opened.

When I finally opened it, the data didn't agree with Google. It actively contradicted it.

On the site, two pages cover the same subject. One is a dated long-form article — the "everything about this topic, updated for 2026" piece. The other is an interactive map that renders the same underlying data geographically: pick a place, see its status. They target the same cluster of queries.

On Google, the article wins decisively — roughly five times the impressions of the map, and a better average position. I'd internalized that as settled: Google had picked the article as the answer, end of story.

On Bing, the map wins. The article is barely present. The map page alone accounts for 56% of every impression the site gets on Bing. The article that dominates Google sits near the bottom of the list.

Same two pages. Opposite order. Two engines looking at identical content and disagreeing about which one deserves to rank.

Page	Google	Bing
Long-form article	Wins	Loses
Interactive map	Loses	Wins

Why an engine "picks" a page

This isn't randomness, and it isn't a quirk of one crawl. The two engines reward different things for the same informational query.

Google rewards the article. Its helpful-content systems lean hard toward substantive, dated, written pages for informational intent. A long article with depth, a clear publish date, and a "2026" freshness signal reads to Google as the authoritative answer to "tell me about this topic." The map page — mostly an interactive widget with thin surrounding text — reads as a tool, not an answer. Google buries it.

Bing rewards the tool. Bing is more literal. When the query itself implies a lookup — anything phrased as a map or a by-location question — Bing matches the page whose entire purpose is that lookup. The map page's title, structure, and intent line up with the query word-for-word, so Bing puts it first. The article, to Bing, is just a wall of text adjacent to the thing the user actually asked for.

Neither is wrong. They're optimizing for different definitions of "best result," and my two pages happened to be perfect representatives of each definition. So each engine crowned a different king.

The inversion showed up at the sub-page level too, even more starkly: one regional reference page ranks on the first page of Bing and sits at position 69 on Google — effectively invisible. Same page. The gap there isn't content; it's authority. Those head terms on Google are owned by encyclopedias, government sites, and AI summaries, and a young page doesn't crack them. Bing's bar is lower, so the same page surfaces. Watching only Google, I'd have called that page a failure. It isn't — it's just winning on the engine I never checked.

The zero-click trap

Here's the part that nearly fooled me twice.

The map page's top Bing queries — the high-volume, head-of-cluster ones — rank around positions 6 to 9 and convert almost no clicks. My first read was "impressions without clicks, so this is worthless ranking."

That might be the wrong conclusion. One likely explanation — and I want to be careful to call it a hypothesis, not a proven cause — is that some of those impressions are being answered inline by an AI assistant: the model pulls the page into its grounding set, writes the answer, and the user never clicks because they already got what they came for. I've written before about why AI citation is becoming the metric that replaces the click, and a high-impression, zero-click, mid-position query is exactly the shape that pattern leaves behind.

Here's the honest limit, though: I can't confirm it from the data I have. The search engine's webmaster API reports impressions, clicks, and positions — it does not expose whether an AI surface cited the page. So "these zero-click impressions are AI citations" is an inference from the shape of the numbers, not something I measured. It's plausible, it fits, and it's the explanation I'd bet on — but it's a bet, and treating it as confirmed would be the exact confidence gap I keep warning about. What I can say for certain is narrower and still useful: zero clicks at position 6–9 does not mean zero value, and reading it as failure would be a mistake.

So "zero clicks" on those queries isn't dead weight. It's the citation surface. The clicks I do get come from longer-tail queries lower in the cluster, ranking higher, where no AI summary intercepts them. Two completely different value mechanisms inside the same page, and you can only tell them apart if you stop treating clicks as the only outcome that counts. (If the GEO/AEO/citation vocabulary is new, I untangled the three terms here.)

What watching one engine hid

I would not have seen any of this from Google alone. Not the inversion, not the page that's invisible on one engine and page-one on the other, not the citation pattern. Google's data is internally consistent and tells a clean story — the article is the answer — and that story is true on Google and false everywhere else.

That's the part that actually unsettled me, and it's bigger than Bing. For years my working definition was Google = search. One dashboard, one set of numbers, one verdict on whether a page worked. But search has quietly become plural — Google, Bing, AI Overviews, Perplexity, ChatGPT, Gemini — and they don't agree, because they don't rank the same way or even reward the same kind of page. My mental model wasn't wrong so much as incomplete: I was reading one instrument and calling it the weather. The inversion was just the first thing that didn't fit, because it was the first time I looked at a second instrument.

The fix was boring and is the actual point: pipe the second engine into the same place as the first. Bing exposes a Webmaster API with the same kind of search-performance data Google's does — totals, top pages, top queries, positions — behind a single API key you generate in the tool's settings. I wired Google Search Console into a daily Telegram briefing months ago; adding Bing was the same shape of work pointed at a different endpoint. Now both engines land in the same thread, weekly, side by side — a short "this week vs last week" and a six-month trend so I can watch the curve. (That curve, for the record, has compounded every month from a near-zero start — visibility I had no idea was accruing because I wasn't looking.)

One gotcha worth saving someone an hour: the related IndexNow endpoint — the protocol that lets you ping Bing and others the instant a URL changes — returned 403 Forbidden for every request until I added a User-Agent header. curl and the browser don't care; the API does. No error message explains it. If your IndexNow submissions are silently rejected, send a User-Agent.

The takeaway

"Rank" isn't one number. The same page can be a winner and a loser at the same moment depending on who's asking, and the two biggest engines can disagree completely about content they're both looking at.

The practical move once you see the split is not to force the pages to fight — I'm not stripping the article of the terms it wins on Google to help the map, or vice versa. They've each won an engine. The right move is to let them, cross-link them so they reinforce instead of compete, and keep both engines on the dashboard so the next divergence isn't invisible for months.

But the durable lesson isn't about Bing at all. It's that treating one engine as the whole of search means your picture is incomplete by definition — and you won't know what you're missing until you instrument the engines you've been ignoring. Mine had a story in it the whole time. I just wasn't looking.

Originally published on tedagentic.com.

Chrome Showed the Data. Firefox Showed Nothing. The API Was Being Blocked.

Ted — Wed, 17 Jun 2026 15:26:31 +0000

I was looking at the same page in two browsers side by side. In Chrome: a full list of results, images, working buttons. In Firefox: a different, smaller set of placeholder cards with no images and no links — and a banner that read "0 results found."

Same URL. Same deployed build. No "works on my machine" excuse, because it wasn't working on the other machine — it was the same machine, two windows.

That contradiction is the whole story, and the cause was a class of bug that is almost impossible to catch in development: the browser was blocking my own API call, and I'd been developing in the one browser that didn't.

How the page is built

The page renders a list of results that come from a database. The pattern is the common one for a single-page app: the page ships, then a bit of JavaScript runs and fetches the data from a hosted Postgres/API service (Supabase, in this case) at its own domain — something like xxxx.supabase.co. The results come back, React renders the cards.

If that fetch returns nothing, the component falls back to a hardcoded placeholder list so the page isn't blank. That fallback is what Firefox was showing. So the real question was narrow: why did the fetch return nothing in Firefox but everything in Chrome?

The diagnosis

The server was fine. I could hit the API directly from the command line and get the full result set, HTTP 200. The database, the credentials, the query — all correct. Chrome proved it too: it ran the exact same JavaScript against the exact same endpoint and got the data.

The difference was the browser, and specifically what the browser allowed. Firefox ships with Enhanced Tracking Protection, and a lot of people add uBlock Origin or Privacy Badger on top. Those tools block requests to domains they classify as third-party trackers. The relevant move is in the browser's network panel: the request to xxxx.supabase.co wasn't failing with an error from the server — it was being blocked before it ever left, by the privacy layer.

From my code's point of view, the fetch just throws. data comes back empty. The component does what I told it to do with no data: show the fallback. No crash, no red console error a casual glance would notice. The page "works." It just works with nothing in it.

The whole bug fits in two columns:

   Chrome                      Firefox + tracking protection

   Page                        Page
    ↓                           ↓
   Fetch supabase.co           Fetch supabase.co
    ↓                           ↓
   Data arrives                Blocked
    ↓                           ↓
   Render results              Fallback cards
                                ↓
                               0 results

Same code reaching the same fork, and the third step decides everything.

Why this is so easy to ship

Here's the uncomfortable part. I build in Chrome. Most developers build in Chrome. Chrome, with no aggressive privacy extensions, happily makes the cross-origin call, so the page is perfect every single time I look at it. The failure only appears in an environment I never use during development.

And it gets worse, because the measurement is blocked too. My click tracking — the thing that tells me whether people are using the page — went through the same API domain. So the users who couldn't see the content also couldn't be counted not using it. The bug hides in the browser I don't test, and it erases its own evidence in the analytics. It's invisible twice.

The only reason I found it is that someone opened the page in Firefox and said "this looks broken."

Fix one: stop the fallback being a dead end

The first thing I did was cheap and defensive. The placeholder fallback had no links — a blocked user landed on dead cards and had nowhere to go. So I made every fallback card and every "quick pick" link to the real directory page. Now, whatever blocks the fetch, the visitor still has a working path forward.

This is worth doing regardless, but be honest about what it is: a safety net, not a fix. The user is still being denied the real content. They're just no longer stranded. To actually fix it, the call has to stop being blockable.

Fix two: make the call first-party

It's worth being precise about what I'm fixing here, because it's easy to mislabel. This isn't a Supabase problem — Supabase did everything right. It's a browser trust-boundary problem. The browser draws a line between "the site I'm on" and "some other domain," and privacy filters police that line. My data was on the wrong side of it.

So the fix isn't to change the database. It's to move the request to the trusted side of the line — to change what the browser sees:

   Before                       After

   mysite.com                   mysite.com
    ↓                            ↓
   xxxx.supabase.co             mysite.com/sb-api
   (third-party — policed)      (first-party — trusted)

Same data, same backend. The only thing that changed is whose domain the browser thinks it's talking to. You do that with a reverse proxy: route the API through your own domain so the browser only ever sees a same-origin request to the site it's already on. On Vercel that's a rewrite in vercel.json:

{
  "source": "/sb-api/:path*",
  "destination": "https://xxxx.supabase.co/:path*"
}

Then point the client at the first-party path instead of the vendor's domain:

const SUPABASE_URL =
  import.meta.env.PROD && typeof window !== "undefined"
    ? `${window.location.origin}/sb-api`
    : RAW_SUPABASE_URL; // dev + build-time stay on the direct URL

Now the browser makes a request to mysite.com/sb-api/..., which Vercel quietly forwards to the real API. To Firefox's tracking protection it's a call to the site you're already visiting — first-party, not a tracker, not blocked. As a bonus, the browser no longer has to negotiate cross-origin access at all.

I verified it the same way I found the bug — by hitting the new first-party URL and watching the real data come back through it — then opened the page in Firefox and got the full list, images and all, identical to Chrome.

The gotchas, because there are always gotchas

A reverse proxy in front of your whole data layer is a real change. Three things to check before you ship it:

Websockets don't ride through a simple rewrite. If you use the service's realtime/subscription features, a path rewrite won't proxy the socket cleanly. I got to skip this because the app uses none — but check first, or you'll trade a Firefox bug for a realtime outage.
Pin your auth storage key. Client libraries often derive the key they store the session under from the API URL. Change the URL and the key changes, and everyone silently gets logged out. Set the storage key explicitly to the original value so swapping the URL is invisible to existing sessions.
Keep server-side and build-time calls on the direct URL. Anything running outside the browser — a build step, prerendering, server code — has no ad blocker and no same-origin to honor. Sending it through the proxy is pointless and can be circular. Gate the rewrite to the browser only.

The crawler is just another blocked client

There's a tidy way to think about all of this. A browser with tracking protection is a client that won't run part of your page. A search crawler is also a client that won't run part of your page — it may not execute your fetch at all. Both end up looking at the empty shell.

So the durable answer isn't only the proxy; it's not making the page's existence depend on a fetch that some clients will never complete. I also prerender the real results into the page's static HTML at build time, so the meaningful content is in the document before any JavaScript — for crawlers, for blocked browsers, for anyone whose script never runs. The proxy fixes the live experience; prerendering makes sure there's something real there even when nothing runs at all.

What I took from it

A page that's perfect in Chrome can be empty in Firefox. If your data comes from a client-side call to a third-party domain, privacy filters will block it for a real slice of users, and you'll never see it in the browser you develop in.
Silent failure is the dangerous kind. A blocked fetch doesn't crash — it just renders your empty state. Make the empty state recoverable, but don't mistake that for a fix.
The blocked users are uncounted users. If your analytics and your content share an API domain, the people who can't see the page also don't show up as not seeing it. Test in Firefox with tracking protection on, and with an ad blocker — that's your blind spot, on purpose.
First-party proxy beats third-party fetch. Route vendor APIs through your own domain so the browser sees same-origin. Mind websockets, pin the auth key, keep build/server calls direct.
Treat crawlers and blocked browsers as the same problem. Anything that won't run your JavaScript needs the real content in the HTML already.

Underneath the specifics, this was the same shape as most of the worst bugs I find:

Human validation passed
        ↓
System validation failed
        ↓
Find the hidden assumption
        ↓
Fix the assumption, not the symptom

The hidden assumption here was four words long — "if it works in Chrome" — and the real version was "it works in the browser I happened to develop in." The fix wasn't really a proxy. The proxy was just how I retired the assumption.

The page had been broken for months for an entire category of visitors. It looked flawless the whole time — in the one browser I happened to use.

My Redirects Worked in the Browser. Googlebot Saw Soft 404s.

Ted — Sun, 14 Jun 2026 11:06:53 +0000

Google Search Console flagged six URLs on a client site as Soft 404. Every one of them returned a clean 200 OK when I curled it. So how does a page that loads fine get reported as "not found"?

That contradiction is the whole story, and the answer turned out to be a category of bug I'd been shipping without realizing it.

What "Soft 404" actually means

A hard 404 is honest: the server returns a 404 status, Google drops the URL, everyone moves on. A soft 404 is when the server returns 200 OK but the content looks like an error or an empty page to Google. The status line says "here's your page," the body says "there's nothing here." Google trusts the body.

On a single-page app, there's a very common way to produce exactly that.

The setup

The site is a React/Vite SPA, deployed on Vercel, with a prerender layer that injects real HTML for SEO so the crawler doesn't have to run JavaScript. Routes that are prerendered get proper content. Routes that aren't fall through to Vercel's catch-all, which serves the app shell — effectively the homepage HTML — at whatever URL was requested.

Hold onto that last sentence. It's the trap.

The fingerprint

All six soft-404s shared a pattern: they were old, renamed blog slugs. A few were year-suffixed posts I'd renamed (think /blog/some-guide-2025 → /blog/some-guide-2026). Two were older duplicates of a post that now lives at a cleaner URL. One was the same path on the www host instead of the apex.

Old URLs that should redirect somewhere. And in the codebase, three of them did have redirects. That's what made this confusing — I had redirects, and Google was still calling the pages broken.

The trap: the redirects were client-side

The three "redirected" routes used the SPA router's redirect component — the React-Router <Navigate> element. In a browser, it works perfectly: the app boots, the router matches the old path, and the user is bounced to the new URL before they notice.

But look at the order of operations from Googlebot's point of view:

Request /blog/some-guide-2025.
The route isn't prerendered, so Vercel serves the app shell — homepage-ish HTML, status 200.
Then the JavaScript would run and redirect — but the crawler has already been handed a 200 with the wrong content.

The redirect lives inside the JavaScript. The crawler's verdict is formed before the JavaScript runs. So Google sees a 200 response whose body is the homepage, served at a URL that's supposed to be a specific blog post — a page that resolves to nothing meaningful. Soft 404.

The other two had no redirect at all and went straight to the shell fallback — same outcome by a more direct route.

A client-side redirect is invisible to a crawler that judges the first response. If you want Google to treat a URL as moved, the move has to happen in the response itself, before any JavaScript: a real server-side 3xx.

The fix

The fix is to do the redirect at the edge, not in the app. On Vercel that's a redirects entry in vercel.json:

{
  "source": "/blog/old-slug",
  "destination": "/blog/new-slug",
  "permanent": true
}

"permanent": true emits a 308 (Vercel's permanent redirect), which Google treats the same as a 301. Now the first response Googlebot gets is "this moved, permanently, here" — no shell, no JavaScript, no ambiguity. The www variant was already handled by an apex-redirect rule, so once the apex path resolved correctly, the www one chained into it.

I added server-side redirects for the two that had none, pointed both old duplicates at the canonical post, and the client-side <Navigate> routes became harmless fallbacks behind the real ones.

The twist: most of them were already fixed

Here's the part that saved me a pile of unnecessary work — and that I almost skipped.

Before "fixing" the three that already had server-side redirects, I checked when Google last crawled them. The URL Inspection API returns lastCrawlTime. Every one of those dates was weeks before the server-side redirects had shipped. The pages weren't broken anymore. Google's report was a snapshot from the last time it looked, and it simply hadn't looked again.

GSC statuses are not live. They're the result of the most recent crawl, which can be a month stale on a low-traffic site. Before you re-fix something the report calls broken, check lastCrawlTime — you may be debugging a problem that no longer exists.

Submission is not a fix

The instinct, once the redirects were in, was to "submit the pages to Google." But you don't submit a redirect source — it's not content, it's a signpost. The Indexing API is for telling Google a real page changed. For redirected URLs, the correct lever is GSC → "Validate Fix" on the soft-404 report, which re-queues the crawl. The only thing worth submitting is the redirect targets — the live pages — so Google freshens those.

Resubmitting a URL Google already crawled doesn't change Google's mind. It just asks the same question again.

The other bucket: "crawled, not indexed"

Separately, a larger batch of pages on the same site had been sitting in "Crawled – currently not indexed" — and around two dozen of them cleared into the index over the same period. It's tempting to lump that together with the soft-404 fix, but it's a different problem with a different cause.

Soft 404 is a technical verdict: wrong response, fix the response. "Crawled, not indexed" is a quality verdict: Google fetched a real page and chose not to index it. You don't move that with redirects or resubmission — you move it with better content (the prerender layer putting real HTML in front of the crawler) and internal links that give the page a reason to matter. Those recoveries were the delayed payoff of content and linking work from weeks earlier, not anything I did that day.

Keeping the two buckets separate matters, because the fixes don't transfer. A 308 will never rescue a thin page, and a content rewrite will never fix a client-side redirect.

What I took from it

A 200 status doesn't mean Google sees a real page. Soft 404 is the body contradicting the status line.
On an SPA, any route that isn't prerendered falls through to the app shell — a 200 full of the wrong content. That's a soft-404 factory.
Client-side redirects don't exist as far as a crawler is concerned. Anything Google should treat as moved needs a server-side 3xx, before the JavaScript.
GSC reports are stale snapshots. Check lastCrawlTime before re-fixing.
Submission re-asks the question; it doesn't change the answer. Use "Validate Fix" for redirects; submit only live targets.

The redirects had been working the entire time — in the one place that couldn't see them.

Migrating Off OpenClaw Without Downtime — and the Offset That Made Hermes Look Dead

Ted — Sun, 14 Jun 2026 10:19:10 +0000

A while back I wrote a comparison of OpenClaw and Hermes — two open-source, self-hosted AI agents I run on the same bare-metal box, both wired to Telegram. The verdict was that they're complementary: OpenClaw as the dependable gateway for scheduled delivery, Hermes as the agent that builds context over time. Run both, I said. Don't choose.

Then I actually lived with both for a few more weeks, and the verdict started to move.

OpenClaw is gateway-first: a messaging hub that runs plugins, models, and cron delivery on a schedule you define. Hermes is agent-first: the messaging is just how you reach it, and the point is an agent that accumulates knowledge and needs less steering each time. On paper that made OpenClaw the better runtime for my scheduled jobs.

The migration started for a plainer reason than "one is better": I'd come to trust Hermes more with anything long-running. When I handed my OpenClaw setup a task with real weight, it would acknowledge it and then go quiet — the agent loop hitting a timeout and dropping the work before it finished. That isn't a verdict on the project; it's a mismatch between how I had it configured and the load I was putting on it. But trust is trust, and mine had moved.

Hermes, on the same machine, on a free OpenRouter model, just finishes. It holds the task, retries, and delivers — because the persistence is in the harness, not the model. That's the whole thing: the model matters less than whether the system around it gives up.

So I decided to migrate. But not the way you'd think.

The rule: don't turn anything off

The constraint I set for myself was that OpenClaw keeps running, untouched, the entire time. No big-bang switch. Two reasons:

It's my fallback. If the new path has a blind spot I haven't seen, I don't want to discover it by losing notifications.
I want to compare. Running both in parallel is the only way to see where each one lags before I commit. You don't get that from a one-way cutover.

So the plan became: deliver everything through Hermes in parallel with OpenClaw for a couple of weeks, watch them side by side, then retire OpenClaw's delivery one job at a time.

The part I underestimated: there wasn't one delivery path. There were four.

My notification stack is a pile of cron jobs — site monitors, system-health checks, morning summaries, scheduled reports. The important thing about them is that the agent never touches them. They're pure notifiers: each one computes a result and pushes it straight to Telegram. The agent only gets involved when I actually ask it something.

I assumed they all delivered the same way. They didn't. When I went looking, there were four different mechanisms across ~39 delivery points:

Scripts shelling out to the OpenClaw CLI
Node monitors hitting the Telegram Bot API directly
A shared bash helper used by a few jobs
One Python script posting straight to the API

There was no single switch to flip. Which is exactly why a clean cutover would have been a bad idea.

The mirror pattern

Instead of rerouting anything, I made the new delivery purely additive. One control script — hermes_mirror.sh — and every delivery point gets one extra line that calls it in the background:

bash jobs: hermes_mirror.sh "$MSG" &
Python jobs: subprocess.Popen([...])
Node jobs: child_process.spawn(...)

The helper waits a moment, then sends the same message through Hermes. The original OpenClaw send is never modified, never wrapped, never moved. If the mirror errors or hangs, the existing delivery already happened — it cannot be affected. And because everything funnels through that one script, there's a single kill switch: set ENABLED=0 and all mirroring stops at once. One offset value, one log, one place to reason about.

That additive property is the whole safety story. I wasn't migrating the system. I was running a second, shadow system next to it and watching.

The offset that made it look broken

Here's the part that actually taught me something.

I didn't want the two copies of every notification landing in the same second — that's noisy and makes them hard to tell apart. So I set the mirror to wait three minutes before sending. Comfortable separation. Seemed obviously correct.

Then I ran the first real test. Fired a notification. It showed up on the old bot. The new one… nothing. I sat there watching an empty chat, fully convinced the Hermes path had failed.

It hadn't. It was running exactly as designed — three minutes behind. The copy landed right on schedule, 180 seconds later, while I was already digging through logs looking for a bug that didn't exist.

The lesson stuck: in a parallel migration, "late" reads as "broken." A safety margin I added to keep things tidy became a false failure signal, because the only way I had to judge the new system was did the message show up when I expected it. Three minutes of silence and your brain files it under "dead."

But the delay itself wasn't the mistake. The mistake was adding a delay without adding any signal that the delay was intentional. If a mirrored notification is meant to land three minutes later, something should say so — a log line, a status ping, anything that separates "received, waiting" from "never arrived." Without that, silence and failure are indistinguishable. A safety mechanism with no observability doesn't make a system safer; it makes it ambiguous — and ambiguity, in a migration, is its own kind of failure.

I cut the offset to 20 seconds. Long enough that the two copies never collide, short enough that the new one clearly arrives with the original instead of looking like it got lost. Same mechanism, completely different read on whether it's working. (The better fix is still on the list: have the mirror log an explicit "received, holding 20s" line, so the wait is something you can see, not infer.)

The principles that fell out of it

By the time the offset was sorted, the migration had quietly handed me its own rules:

Never modify the primary delivery path first. The thing that works keeps working, untouched.
Add the new path alongside the old — purely additive. It can fail without taking anything down with it.
Keep one global kill switch. A single place to stop everything the moment it misbehaves.
Make delayed delivery visibly delayed, not silently delayed. A wait with no signal is indistinguishable from a failure.
Remove the old path only after watching a full execution cycle — including the jobs that only fire weekly.

None of them are clever. They're just what's left after you stop assuming the replacement works and start making it prove it.

The gotcha in automating the edit

Wiring one line into ~20 Python scripts, I wrote a small injector instead of editing each by hand. It dropped the mirror call in as the first line of each delivery function. Clean — except one script imported its dependency inside the function, below where my line now sat. So the injected call referenced a module that wasn't imported yet. Instant NameError, only in that one file.

Blind codemods across a heterogeneous pile of scripts will find the one file that doesn't match your mental model. Compile-check everything after, not just the ones you expect to fail.

The thing I expected to be hard and wasn't

I assumed I'd have to port OpenClaw's scheduled agent jobs — the ones where it actually reasons, not just delivers. Turned out they were all already disabled, superseded long ago by plain deterministic scripts. The only thing OpenClaw was still genuinely doing for me on a schedule was acting as the mailman. The agentic work I thought I'd be migrating didn't exist anymore. The real dependency was just delivery.

Where it stands

Both agents now deliver the full stream in parallel, Hermes trailing each original by 20 seconds. OpenClaw is untouched and still my fallback. The plan is to watch a complete cycle — including the jobs that only fire weekly — and then start removing OpenClaw's send per-job, one at a time, only once I've seen each one deliver cleanly through Hermes.

The migration isn't "done" — and that's the point. The mirror pattern turned it from a decision into an observation exercise. I never had to ask whether Hermes was ready; I could watch it prove it, one notification at a time, with the old system holding the floor the whole way.

That's the part I'll carry to the next one. When you're replacing a live system you don't trust yet, the move isn't to cut over and hope. It's to run the replacement in the open, right next to the thing it's replacing, and let it earn the handoff — visibly, on a schedule you can watch, with nothing torn out until it has.

GEO: What Generative Engine Optimization Actually Means for Your Content

Ted — Thu, 11 Jun 2026 08:30:10 +0000

Search didn't break. It restructured. The page that ranked #1 for an informational query now sits below an AI-generated summary that pulls from it — and the user never scrolls down.

That's the GEO problem in one sentence.

What Changed in the SERP

Old SERP for most informational queries:

┌─────────────────────────────────────┐
│  [Ads]                              │
│  1. example.com/answer              │
│  2. another-site.com/guide          │
│  3. wikipedia.org/wiki/topic        │
│  4. ...                             │
└─────────────────────────────────────┘

New SERP for the same query:

┌─────────────────────────────────────┐
│  [AI Overview]                      │
│   └─ synthesized answer             │
│   └─ 3-5 citation links (collapsed) │
│                                     │
│  [People Also Ask]                  │
│   └─ expands inline                 │
│                                     │
│  1. example.com/answer  ← you       │
│  2. another-site.com/guide          │
└─────────────────────────────────────┘

The user gets an answer before they see your link. CTR drops. Impressions stay the same or grow. That gap is the GEO signal.

On one site I run, impressions on an informational page grew 4x over three months while CTR dropped below 0.5% — the query cluster started triggering AI Overviews during that period. The page didn't lose rankings. It lost the click.

SEO vs GEO — What Each Optimizes For

Dimension	SEO	GEO
Target	Ranking position	Citation in AI summary
Signal	Backlinks, authority, E-E-A-T	Structure, clarity, entity match
Win condition	User clicks your link	AI quotes your content
Metric	Clicks, CTR	Impressions → brand recall
Risk	Algorithm update	AI answers the query fully

GEO and SEO aren't competing strategies. GEO is what happens to SEO when the SERP adds a layer above organic.

Which Query Types Are Most Exposed

Not every query gets an AI Overview. The exposure depends on query type:

HIGH GEO EXPOSURE
─────────────────────────────────────────
  Informational   "what is X"
                  "how does X work"
                  "X explained"
                  "X rules / laws / limits"

MEDIUM GEO EXPOSURE
─────────────────────────────────────────
  Comparison      "X vs Y"
                  "best X for Y"
                  "X alternatives"

LOW GEO EXPOSURE
─────────────────────────────────────────
  Transactional   "buy X"
                  "X near me"
                  "book X in [city]"
                  "X price"

Transactional and local queries rarely trigger AI Overviews. Google still routes those to maps, commerce, and organic listings. That's where clicks live.

What GEO-Optimized Content Looks Like

AI systems appear to favor content that answers cleanly and early. The structure matters more than the word count.

Not this:

Introduction paragraph...
Background on the topic...
History and context...
What experts say...
[Actual answer buried at paragraph 6]

This:

[Direct answer in first 2 sentences]

## Key points
- Fact with number
- Fact with source signal ("as of 2026...")
- Specific limit, rule, or definition

[Supporting detail below]

The AI Overview pulls from the top of the page. If your answer is buried, a cleaner competitor gets cited instead — even if you outrank them.

Other signals that increase citation likelihood:

Defined terms in headings (##)
Numbered lists with specific values (not vague)
Dates and freshness markers
Tables for comparison content
No filler text between the H1 and the answer

The Downstream Intent Play

Here's the strategic shift GEO forces: stop trying to own queries where AI answers the whole thing. Own what comes next.

User query: "what are the cannabis limits in Colorado"
     │
     ▼
AI Overview answers it fully
     │
     ▼
User now knows the limit — and wants to act on it
     │
     ├──► "420-friendly hotels in Colorado"      ← transactional
     ├──► "cannabis tours in Denver"             ← transactional
     └──► "where to buy in [city]"               ← local

The informational query feeds intent. The downstream query converts. GEO-aware content strategy maps the full path — not just the definition page, but every action a user might take after getting the answer.

If your site only has the definition page, AI Overview cuts the journey short. If you have the definition page and the action pages linked from it, you capture both the citation signal and the conversion traffic.

Practical GEO Checklist

Content structure:

[ ] Direct answer in the first 2 sentences
[ ] No more than one paragraph before the first heading
[ ] Key facts in lists or tables, not buried in prose
[ ] Dates and specifics (not "recently" — use the year)
[ ] Headings match the exact phrasing of common questions

Internal linking:

[ ] Every informational page links to a downstream action page
[ ] Transactional pages don't depend on informational traffic alone
[ ] Related guides section at the bottom of every post

Monitoring:

[ ] Track impressions vs clicks separately — divergence signals AI capture
[ ] Flag pages where impressions grow but CTR drops below 1%
[ ] Check which queries trigger AI Overviews for your target keywords

The Honest Summary

GEO doesn't replace SEO work. Rankings still matter for non-AI-captured queries, and being cited in an AI Overview still requires ranking well enough to be in Google's index pull.

What GEO changes is the goal for different content types. Informational pages: optimize for citation and downstream linking, not for direct clicks. Transactional pages: optimize for clicks, those are still yours to win.

The operators who will get hurt are the ones with sites built entirely on informational content with no action layer underneath. The ones who built the full funnel — definition → comparison → booking — are fine.

Build the full funnel.

How I Wired Google Search Console to Telegram on Day One

Ted — Thu, 11 Jun 2026 08:30:03 +0000

Most people check Google Search Console by logging in, navigating to the property, clicking through the date filters, and reading off numbers. That works — but you only do it when you remember, and you only see what you think to look for.

This is a different approach. The GSC data comes to you, every morning, in the same Telegram thread as every other system alert. No browser, no login, no forgetting. And because it's API-driven, you pull exactly what you need — top queries, top pages, position changes — formatted the way you want it, not the way Google's UI presents it.

I wired this up on day three of launching tedagentic.com — before a single organic click. The Telegram delivery layer was already in place from the agent setup. Adding GSC to it was the natural next step: one more data source piped into the same channel.

Building it before any traffic exists means the baseline is captured from zero. When the first impression shows up, it's logged. When something drops later, there's something to measure against. Most monitoring gets built reactively — after something falls. This flips that.

This is what I built and how it works.

The stack

Google Search Console API
        ↓
  Node.js script (monitor.js)
        ↓
  cron (8:25am daily)
        ↓
  Telegram bot → phone

No dashboard. No third-party service. No browser required. The briefing arrives in the same Telegram thread as every other system alert — rankings, RAM, errors, competitor movements. One channel, everything piped in.

Step 1: GSC service account

I already had a Google service account from an earlier project. If you're starting from scratch, the setup is:

Go to console.cloud.google.com → create a project
Enable the Google Search Console API
IAM & Admin → Service Accounts → Create service account
Download the JSON credentials key

The credentials file looks like this:

{
  "type": "service_account",
  "project_id": "your-project-id",
  "client_email": "your-account@your-project.iam.gserviceaccount.com",
  "private_key": "..."
}

Save it somewhere secure. Mine lives at ~/.gsc-credentials.json.

Then go to Google Search Console, open your property, and add the client_email as a user with Full permission. The service account won't show in the autocomplete — paste the email directly and save.

One thing to watch: GSC doesn't always confirm the save visually. Go back to Users and permissions and verify the email is listed before moving on. If it's not there, add it again.

Step 2: The monitor script

The script pulls three things from the Search Console API: 7-day totals, top pages, and top queries. Then formats it and sends to Telegram.

const { google } = require('googleapis');
const https = require('https');

const GSC_SITE = 'sc-domain:yourdomain.com';
const CREDENTIALS_PATH = '/home/user/.gsc-credentials.json';
const TELEGRAM_TOKEN = 'YOUR_BOT_TOKEN';
const TELEGRAM_CHAT_ID = 'YOUR_CHAT_ID';

async function query(sc, requestBody) {
  const res = await sc.searchanalytics.query({ siteUrl: GSC_SITE, requestBody });
  return res.data.rows || [];
}

async function run() {
  const auth = await new google.auth.GoogleAuth({
    keyFile: CREDENTIALS_PATH,
    scopes: ['https://www.googleapis.com/auth/webmasters.readonly']
  }).getClient();

  const sc = google.searchconsole({ version: 'v1', auth });

  // GSC has a ~3-day data lag — pull days 3-9 ago as "last 7 days"
  const end = new Date(Date.now() - 3 * 86400000).toISOString().split('T')[0];
  const start = new Date(Date.now() - 9 * 86400000).toISOString().split('T')[0];

  const [totals, pages, queries] = await Promise.all([
    query(sc, { startDate: start, endDate: end, dimensions: ['date'], rowLimit: 7 }),
    query(sc, { startDate: start, endDate: end, dimensions: ['page'], rowLimit: 10,
      orderBy: [{ fieldName: 'impressions', sortOrder: 'DESCENDING' }] }),
    query(sc, { startDate: start, endDate: end, dimensions: ['query'], rowLimit: 10,
      orderBy: [{ fieldName: 'impressions', sortOrder: 'DESCENDING' }] }),
  ]);

  const totalClicks = totals.reduce((s, r) => s + r.clicks, 0);
  const totalImpr = totals.reduce((s, r) => s + r.impressions, 0);
  const avgPos = totals.length
    ? (totals.reduce((s, r) => s + r.position, 0) / totals.length).toFixed(1)
    : '—';

  let msg = `📡 tedagentic.com — ${new Date().toDateString()}\n\n`;
  msg += `Clicks: ${totalClicks} | Impressions: ${totalImpr} | Avg pos: ${avgPos}\n\n`;

  msg += queries.length
    ? `Top queries:\n` + queries.slice(0, 5).map(r =>
        `  "${r.keys[0]}" — ${r.impressions} impr | pos ${r.position.toFixed(1)}`
      ).join('\n')
    : `Queries: none yet`;

  await sendTelegram(msg);
}

The full script with error handling and page breakdown is at seo-monitor/tedagentic/monitor.js in the repo.

Two things worth noting:

The 3-day lag. GSC data isn't real-time — it runs 2-3 days behind. If you pull "yesterday," you'll get empty rows. The script accounts for this by offsetting: end = today - 3 days, start = end - 7 days. This is why the date window in the Telegram message will always look 3 days behind your actual calendar.

Empty rows aren't errors. A new site will return zero rows from the API for days or weeks. The script handles this cleanly — if rows is empty it reports "none yet" rather than crashing. Early on that's most of what you'll see.

Step 3: Cron

25 8 * * * cd /home/aiserver/seo-monitor && node tedagentic/monitor.js >> /home/aiserver/seo-monitor/tedagentic_monitor.log 2>&1

8:25am daily. Runs after the other site monitors (7am–8:20am) so alerts don't pile up at the same time. The log captures everything — errors, output, the full run — so if a briefing doesn't arrive in Telegram I can check what happened without re-running anything.

What day one looks like

Zero clicks. Zero impressions. No queries. No pages.

That's the point.

This isn't a failure state — it's the starting gun. The monitoring stack is live before the data exists. When the first impression shows up in GSC, it'll be logged. When the first query appears, I'll see it in the morning briefing the same day it lands. When something drops later, I'll have the baseline to measure against.

Most SEO monitoring setups are built reactively — something drops, you build monitoring to catch the next drop. Building it before any traffic exists flips that. You get the full picture from zero.

What's next

The current script is a daily briefing — totals and top items, no intelligence layer. The next step is adding an agent commentary pass: the raw GSC data goes to the LLM, which adds a short interpretation. Not a dashboard, not a report — a one-line read from the agent on whether the numbers look normal, surprising, or worth attention.

That's the difference between monitoring and an AI agent doing monitoring. The script already runs the former. Part 3 covers wiring in the latter.

The same agent that delivers this briefing can now post to X directly from Telegram — the infrastructure compounds as each piece connects.

For now: the baseline exists. The system is watching. The data will come.

The Prerender Was Running. The Pages Were Still Empty.

Ted — Wed, 10 Jun 2026 15:02:13 +0000

In Astro vs Lovable for SEO I wrote about a client site dropping out of Google's index because Googlebot was getting empty HTML shells from a CSR React app. The fix was a custom prerender layer — a build-time script that injects real content into the static HTML before the JS bundle runs.

I built it, deployed it, and moved on. The prerender was in the pipeline. I assumed it was working.

Six months later I checked GSC on the site's four main directory pages. Zero impressions. Not low — zero. Across the entire period.

What the pipeline was supposed to do

The build command in package.json:

"build": "vite build && node scripts/prerender.mjs"

After Vite builds the React bundle, the prerender script runs. It fetches live data from Supabase, then walks every route and injects real content — titles, canonicals, descriptions, and actual listings — directly into the HTML files in dist/.

┌─────────────────────────────────────────────────────────┐
│                   INTENDED PIPELINE                      │
├─────────────────────────────────────────────────────────┤
│                                                          │
│   vite build          →   dist/ HTML shells             │
│        │                                                 │
│        ▼                                                 │
│   prerender.mjs       →   fetch Supabase data           │
│        │                       │                        │
│        │              ┌────────▼────────┐               │
│        │              │  category A     │               │
│        │              │  category B     │               │
│        │              │  category C     │               │
│        │              │  category D     │               │
│        │              └────────┬────────┘               │
│        │                       │                        │
│        ▼                       ▼                        │
│   inject into dist/   →   real HTML content             │
│                                                          │
│   Googlebot sees: listings, titles, descriptions        │
└─────────────────────────────────────────────────────────┘

The theory was sound. The implementation had a silent failure mode I never tested for.

What was actually happening

The prerender script fetched data like this:

async function fetchListings() {
  const url = process.env.VITE_SUPABASE_URL;
  const key = process.env.VITE_SUPABASE_PUBLISHABLE_KEY;
  if (!url || !key) {
    console.warn('  ⚠ Supabase env vars missing — skipping live fetch');
    return [];
  }
  // ... fetch from Supabase
}

If the env vars weren't present, it logged a warning and returned an empty array. Then the HTML builder received an empty array and used its fallback:

function buildListingsHtml(items) {
  if (!items.length) return '<p>Browse our verified listings below.</p>';
  // ... build real listing HTML
}

The prerender ran. It produced output. Every page got a title, a canonical URL, and a meta description. But the body content was a single generic sentence.

┌─────────────────────────────────────────────────────────┐
│                  WHAT ACTUALLY HAPPENED                  │
├─────────────────────────────────────────────────────────┤
│                                                          │
│   Vercel build starts                                    │
│        │                                                 │
│        ▼                                                 │
│   prerender.mjs runs                                     │
│        │                                                 │
│        ▼                                                 │
│   process.env.VITE_SUPABASE_URL  →  undefined           │
│        │                                                 │
│        ▼                                                 │
│   ⚠ warning logged, empty array returned               │
│        │                                                 │
│        ▼                                                 │
│   buildListingsHtml([])  →  fallback placeholder        │
│        │                                                 │
│        ▼                                                 │
│   dist/category-a/index.html injected with:             │
│   <p>Browse our verified listings below.</p>            │
│                                                          │
│   Googlebot sees: one sentence, no listings             │
└─────────────────────────────────────────────────────────┘

The build logs showed the warning. I wasn't reading build logs. The deploy succeeded. Vercel reported green. The pages looked correct when I opened them in a browser — React hydrated immediately and replaced the placeholder with live data from Supabase. Only a Googlebot crawl or a raw curl would have caught it.

Finding it

One of the four directories was getting consistent clicks in GSC — not many, but steady. The others showed nothing at all. That partial signal was actually what made the problem easy to dismiss for months. One section performing feels like evidence the system is working. It isn't — it just means one placeholder happened to have enough content to be useful to Google, while the others didn't.

The directory that was ranking had a more descriptive static placeholder — a full sentence explaining what the section contained. The others had a single generic line. Google indexed what it could work with and ignored the rest.

Once I looked at the pattern across all four, the gap was obvious. I ran a direct check:

curl -s "https://example.com/category-a" | grep -i "ssr-prerender" -A 10

Output:

<div id="ssr-prerender" style="font-family:sans-serif;...">
  <h1>Category A Directory</h1>
  <p>Browse our verified listings below.</p>
  <p><a href="https://example.com">Site Name</a> — Directory</p>
</div>

One sentence. No listing names, no descriptions, no links to individual detail pages. Three of the four directories looked exactly like this.

The prerender had been injecting this placeholder on every single deploy since the site launched.

Root cause: why it works locally but fails on Vercel

Vercel does not read your local .env file during builds. It uses environment variables configured in the Vercel dashboard under Project Settings → Environment Variables.

The Supabase credentials existed locally. They were not in Vercel. The prerender script called process.env.VITE_SUPABASE_URL, got undefined, warned quietly, and continued.

┌────────────────────────────┐     ┌────────────────────────────┐
│      Local machine         │     │      Vercel build          │
├────────────────────────────┤     ├────────────────────────────┤
│                            │     │                            │
│  .env                      │     │  Environment Variables     │
│  VITE_SUPABASE_URL=...  ✓  │     │  (none configured)    ✗   │
│  VITE_SUPABASE_KEY=...  ✓  │     │                            │
│                            │     │  process.env.VITE_...      │
│  prerender works locally   │     │  → undefined               │
│                            │     │  → silent fallback         │
└────────────────────────────┘     └────────────────────────────┘

This is the part that made it easy to miss: the prerender worked perfectly in local builds. vite build && node scripts/prerender.mjs locally produced real content because the .env file was present. The failure only happened in the Vercel build environment, which I never tested directly.

The fix

Two parts.

1. Add env vars to Vercel. In the Vercel dashboard: Settings → Environment Variables → add VITE_SUPABASE_URL and VITE_SUPABASE_PUBLISHABLE_KEY for all environments (Production, Preview, Development).

VITE_SUPABASE_PUBLISHABLE_KEY is the Supabase anon key — designed to be public. Vercel warns that keys prefixed with VITE_ may be exposed to the browser. This is intentional; the anon key is already in every user's browser via the JS bundle. Row Level Security policies handle actual access control.

2. Extend the prerender to cover all directories. The original script only fetched one category. I refactored the fetch into a shared helper:

async function supabaseFetch(table, select, order = 'name') {
  const url = process.env.VITE_SUPABASE_URL;
  const key = process.env.VITE_SUPABASE_PUBLISHABLE_KEY;
  if (!url || !key) {
    console.warn('  ⚠ Supabase env vars missing — skipping live fetch');
    return [];
  }
  try {
    const res = await fetch(
      `${url}/rest/v1/${table}?select=${encodeURIComponent(select)}&order=${order}&limit=200`,
      { headers: { apikey: key, Authorization: `Bearer ${key}` } }
    );
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    return await res.json();
  } catch (e) {
    console.warn(`  ⚠ ${table} fetch failed: ${e.message}`);
    return [];
  }
}

Each directory now has its own HTML builder that receives live data and produces real listings for Googlebot:

┌─────────────────────────────────────────────────────────┐
│                    FIXED PIPELINE                        │
├─────────────────────────────────────────────────────────┤
│                                                          │
│   Vercel build                                           │
│        │                                                 │
│        ▼                                                 │
│   process.env.VITE_SUPABASE_URL  →  set ✓              │
│        │                                                 │
│        ▼                                                 │
│   supabaseFetch('table_a')  →  N listings               │
│   supabaseFetch('table_b')  →  N listings               │
│   supabaseFetch('table_c')  →  N listings               │
│   supabaseFetch('table_d')  →  N listings               │
│        │                                                 │
│        ▼                                                 │
│   each builder produces <ul> of real listings           │
│   with names, descriptions, prices, links               │
│        │                                                 │
│        ▼                                                 │
│   Googlebot sees: real content on first request         │
└─────────────────────────────────────────────────────────┘

Verifying prerender output

After any prerender change, verify the output before assuming it worked:

# Check a directory page for real content
curl -s "https://yoursite.com/your-directory" | grep -i "ssr-prerender" -A 20

# If you see a single generic sentence — the fetch failed
# If you see <h3> listing names — it's working

Locally, after a build:

npm run build
grep -A 20 'ssr-prerender' dist/your-directory/index.html

If the output is a placeholder, check whether env vars are available in the build context. Don't assume the warning in build logs is acceptable — treat it as a failure.

What this cost

Six months of Googlebot crawling directory pages with no real content. Every crawl budget spent on those pages returned nothing indexable. The pages weren't penalised — they just didn't exist in Google's index.

The framer-motion incident taught me to add synthetic monitoring for broken UI. This one taught me to verify build output directly, not just check that the script ran.

┌─────────────────────────────────────────────────────────┐
│               MONITORING GAPS, UPDATED                   │
├─────────────────────────────────────────────────────────┤
│                                                          │
│  ✓ UI monitoring      →  Playwright, every 2h          │
│  ✓ GSC monitoring     →  rankings, index status         │
│  ✓ Build verification →  curl prerendered pages         │
│                            after every pipeline change   │
│                                                          │
│  Still blind:                                            │
│  ✗ Build log scanning  →  automated warning detection   │
│  ✗ Data freshness      →  confirm live data not stale   │
└─────────────────────────────────────────────────────────┘

The fix is running. The build log now shows row counts for every table fetch — real numbers, not zero. That's what it should have shown from day one.

framer-motion v12 Broke My UI — And My Monitoring Never Saw It

Ted — Wed, 10 Jun 2026 15:02:07 +0000

I had 22 cron jobs running — ranking checks, error guards, page watchers, RAM monitors, Telegram briefings. None of them fired.

A framer-motion v12 upgrade had been silently blocking navigation on all browsers and crashing the app entirely on Safari. I found it by accident.

What broke

Two symptoms, discovered in the same session:

1. Safari crash. One page wouldn't load at all. Users saw: "Something went wrong. Please refresh the page to continue." Refreshing didn't help.

2. Click navigation broken on all browsers. Interactive cards did nothing when clicked — they hovered correctly, scaled visually, but the click never fired.

The error message came from the React error boundary in App.tsx:

render() {
  if (this.state.hasError) {
    return (
      <div className="min-h-screen flex items-center justify-center">
        <h2>Something went wrong</h2>
        <p>Please refresh the page to continue.</p>
      </div>
    );
  }
}

Error boundaries are good practice — they prevent blank screens. But they swallow the actual error. Nothing in any log showed what crashed or when it started.

Root cause: framer-motion v12 + WAAPI

framer-motion v12 switched its animation engine to the Web Animations API (WAAPI) by default. This broke things in two distinct ways:

Safari crash — Safari's WAAPI implementation is incomplete. Certain element.animate() calls throw in Safari where Chrome and Firefox handle them fine. The unhandled error cascades into the error boundary, which hides the page behind the generic message.

Click blocking on all browsers — When motion.div runs a whileHover animation via WAAPI, the animation layer sits in the pointer event path and intercepts clicks before they reach the underlying <Link>. The hover works. The click doesn't.

Both problems came from the same pattern:

<Link to="/some/route">
  <motion.div whileHover={{ scale: 1.02 }}>
    ...card content...
  </motion.div>
</Link>

The fix

Replace whileHover on elements inside <Link> with CSS-only hover. Same visual result, no JS animation in the pointer event path:

// Before
<motion.div whileHover={{ scale: 1.02 }}>

// After
<div className="transition-all duration-200 hover:scale-[1.02]">

Entrance animations (initial / animate) were left in place — fade and slide transitions work correctly via WAAPI on all browsers including Safari. Only whileHover inside interactive elements is the problem.

Two files changed, ~10 lines. Navigation worked on first deploy.

Why monitoring missed it

Here's what my stack was watching:

┌─────────────────────────────────────────────────────┐
│              WHAT I WAS MONITORING                  │
├─────────────────────────────────────────────────────┤
│  GSC Rankings      impressions, CTR, position drops │
│  Index Status      crawl errors, 4xx, deindex risk  │
│  Page Speed        Core Web Vitals, load time       │
│  Server Health     RAM, disk, process uptime        │
│  Telegram alerts   morning briefing, threshold hits │
└─────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────┐
│              WHAT I WASN'T MONITORING               │
├─────────────────────────────────────────────────────┤
│  Does the page actually render without JS errors?   │
│  Do navigation links work when clicked?             │
│  Is the error boundary showing instead of content?  │
└─────────────────────────────────────────────────────┘

GSC data lags 48–72 hours minimum. A broken page can still show stable impressions for days because crawlers hit the prerendered HTML shell — the static content is fine. The JS crash only happens when React hydrates in a real browser, which Googlebot doesn't fully replicate.

The gap is synthetic monitoring — an automated browser that visits pages, clicks things, and asserts on the result. I had zero of that.

Closing the gap with Playwright

Playwright is a headless browser automation library. Script it to visit URLs, interact with elements, and assert on what it finds.

The new monitoring layer looks like this:

┌──────────────┐     every 2h      ┌─────────────────────┐
│   cron job   │ ───────────────▶  │  ui_monitor.js      │
└──────────────┘                   │                     │
                                   │  Playwright opens   │
                                   │  real Chromium      │
                                   │                     │
                                   │  7 checks:          │
                                   │  • page loads       │
                                   │  • no error boundary│
                                   │  • clicks navigate  │
                                   └────────┬────────────┘
                                            │
                                   pass     │     fail
                                            │
                                   ─────────┼──────────▶  Telegram alert
                                            │
                                          log

The 7 checks:

PASS: Homepage loads
PASS: Key blog page loads
PASS: Interactive guide loads
PASS: Navigation card click — route A
PASS: Navigation card click — route B
PASS: Laws page loads
PASS: Blog index loads

Each check opens a fresh Chromium context, loads the page, and verifies behaviour — not just HTTP 200, but that the error boundary isn't visible and that clicking a card navigates to the correct route.

Telegram alert on failure:

🚨 UI Monitor — 1 check(s) failed

❌ Navigation card click — route A
   Error boundary visible after click

⏰ 2026-05-12T14:00:00.000Z

One practical detail: the app had an age-verification modal that blocks all pointer events on first visit. The monitor bypasses it by injecting a localStorage key before page load via addInitScript:

await page.addInitScript(() => {
  window.localStorage.setItem('age-verified', 'true');
});

Don't try to click through first-visit gates in tests — just set the state the app checks.

Cron entry:

0 */2 * * * node /path/to/ui_monitor.js >> /path/to/ui_monitor.log 2>&1

What this doesn't cover yet

WebKit engine — Playwright Chromium doesn't replicate Safari's JS engine. A WebKit-specific crash would still go undetected. Playwright has a webkit browser option; that's next.
Authenticated flows — forms and gated pages aren't tested
Performance regressions — slow pages aren't flagged

Full coverage isn't the goal at this stage. The goal was to go from zero synthetic monitoring to something. Seven checks every 2 hours is a meaningful step change.

Takeaway

External monitoring  →  what crawlers see
Synthetic monitoring →  what users experience

They are not the same thing. A page can be indexed, ranking, and receiving impressions while being completely broken for every human who visits it.

Adding Playwright took an afternoon. The cron runs automatically. The next time a dependency ships a breaking change — and it will — I'll know within 2 hours instead of weeks.

This incident pushed me to think about monitoring gaps more broadly — if UI monitoring wasn't there, what else wasn't? The answer was search visibility, which led to wiring GSC directly to Telegram before the site had a single organic click.

How to Set Up a Local AI Agent on Your Own Server

Ted — Wed, 10 Jun 2026 15:02:01 +0000

I run SEO campaigns across multiple sites, monitor rankings daily, and manage automation scripts on a homeserver. For a long time the AI layer was completely disconnected from all of it — I'd open a browser, paste in context, get a response, close the tab. Every session started from zero. No memory of what I was working on, no access to my data, no continuity between conversations.

The problem isn't the models. The problem is the interface. A browser tab is a one-shot tool. You get a response and you leave. Nothing persists, nothing connects, and every time you come back you're rebuilding context from scratch.

The fix was OpenClaw — a free, open source AI orchestrator that runs as a background daemon on your server and connects your agent to Telegram. I already used Telegram for monitoring alerts — ranking drops, RAM spikes, cron failures. Having the AI in the same app meant going from an alert to asking the agent to investigate it without switching tools. One app, one workflow — and the agent is always on.

What OpenClaw Is

OpenClaw's official description is "multi-channel AI gateway with extensible messaging integrations." The mechanics are what matter.

The gateway runs on local loopback at port 18789 on your machine. Every message you send through Telegram routes through it to the agent. The agent has a persistent workspace — files that carry context between sessions: who you are, what your projects are, what it's learned about how you work. That workspace loads into every conversation automatically.

Most web AI interfaces are stateless — each session starts fresh. OpenClaw's agent accumulates context over time. The longer it runs, the more useful it gets.

It supports multiple LLM providers out of the box: Anthropic, Google, OpenRouter, Ollama, and OpenAI. You're not locked into one model or one bill. I run a free OpenRouter model as my primary and route heavier tasks to Claude or Gemini when needed.

Installation

I didn't run through this manually. I gave Claude Code the task over SSH — it handled the full install, configured the files, and flagged the two things that needed my input. Here's what the process looks like:

1. Install OpenClaw globally via npm

npm install -g openclaw@latest

Requires Node.js 22.16 or higher. Node 24 is recommended. I was on Node 22 and it worked fine.

2. Run onboarding with the daemon flag

openclaw onboard --install-daemon

The --install-daemon flag registers a systemd user service on Linux (launchd on macOS). After this, OpenClaw starts automatically on boot and stays running in the background.

3. Configure the main settings file

The config lives at ~/.openclaw/openclaw.json. This is where Claude did the most work — channel settings, model providers, plugin allowlist. The key sections:

{
  "channels": {
    "telegram": {
      "botToken": "YOUR_BOT_TOKEN_HERE",
      "enabled": true,
      "dmPolicy": "pairing",
      "allowFrom": ["YOUR_TELEGRAM_USER_ID"]
    }
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "openrouter/minimax/minimax-m2.5:free"
      }
    }
  },
  "models": {
    "providers": {
      "ollama": { "baseUrl": "http://127.0.0.1:11434" },
      "anthropic": { "baseUrl": "https://api.anthropic.com" },
      "openrouter": { "baseUrl": "https://openrouter.ai/api/v1" }
    }
  },
  "plugins": {
    "allow": ["ollama", "anthropic", "google", "openrouter", "browser", "telegram"]
  }
}

4. Create a Telegram bot via BotFather

This is the one step you do manually. Open Telegram, search @BotFather, send /newbot, follow the prompts. You get a bot token — paste it into openclaw.json under channels.telegram.botToken. Your Telegram user ID goes in allowFrom — message @userinfobot to find it.

5. Restart the daemon

openclaw daemon restart

6. Pair your Telegram account

Send your bot a message. OpenClaw's default security model (dmPolicy: "pairing") means unknown senders get a pairing code before the agent responds. Approve it with:

openclaw pairing approve telegram <code>

After that, the bot responds. Setup done.

Watch out: The plugins.allow list must explicitly include "telegram" or the channel silently fails to start — the daemon reports running but your bot won't respond. This catches people off guard because there's no error. The Ollama TUI inside OpenClaw also rewrites openclaw.json when you change models, and it drops telegram from the allowlist. If your bot goes quiet after a model change, that's why. Add "telegram" back to plugins.allow and restart the daemon.

Choosing a Model

My primary is openrouter/minimax/minimax-m2.5:free — handles general queries, completely free, good enough for most daily tasks. For anything that needs more reasoning I fall back to Claude Haiku 4.5 via the Anthropic API. For recurring automation like trend alerts I use kimi-k2.5:cloud via Ollama running locally — no API cost at all.

The free OpenRouter tier covers the majority of day-to-day use. You only route to paid models when the task demands it, which keeps the running cost close to zero most months.

One thing to avoid: Gemini via the OpenAI-compatible endpoint returns 400 errors on this setup. Route it through the native Google provider in openclaw.json instead.

How I Use It Day to Day

Setting up OpenClaw is what prompted me to build out my SEO monitoring scripts properly. Once I had Telegram as a reliable delivery layer, it made sense to build scripts that push data directly there.

The key design decision: scripts deliver raw GSC data straight to Telegram via the bot API — not through the OpenClaw agent. That's intentional. When I want accurate ranking numbers and click metrics, I don't want an AI in the middle summarizing or rounding figures. The script pulls from the API, formats it, and sends it. Exact data, no interpretation.

The agent comes in on top of that. Once raw data lands in Telegram, I ask the agent to interpret it, spot patterns, or flag what to act on. That's where AI adds value — commentary and analysis, not data retrieval. It also keeps cost down since the agent only runs when I actually need it.

What's Next

In Part 2: How I Wired Google Search Console to Telegram on Day One I show the full SEO monitoring workflow built on top of this — the scripts, the cron jobs, the Telegram alerts, and how the agent turns raw GSC data into actual decisions. That's the real case study.

The foundation is what this post covers: a self-hosted agent, always on, connected to your data, living in the app you already use. Everything else builds on top of that — including posting to X directly from Telegram without touching a browser.

Seven Days, Zero Impressions — One Wrong Sitemap URL

Ted — Wed, 10 Jun 2026 03:46:40 +0000

Seven days after launching this blog I had one page indexed: the homepage. Every post showed "URL is unknown to Google" in GSC's URL Inspection tool. Not crawled and rejected — unknown. Google had never seen them.

The stack was solid. Astro static output, 100 across all Core Web Vitals, real content on every post, sitemap submitted on day one. There was no rendering problem, no thin content problem, no technical issue I could point to.

I knew something was wrong. A site scoring 100 on CWV with real content should at minimum show impressions within a week even with no clicks. No signal at all meant Google wasn't reaching the posts.

What I checked

Pulled index status on every post via the GSC URL Inspection API:

verdict	last crawled	coverage state	url
PASS	2026-05-15	Submitted and indexed	/
NEUTRAL	never	URL is unknown to Google	/posts/automating-seo-monitoring-with-ai-agents/
NEUTRAL	never	URL is unknown to Google	/posts/claude-built-my-astro-blog/
NEUTRAL	never	URL is unknown to Google	/posts/connecting-x-api-to-server/
NEUTRAL	never	URL is unknown to Google	/posts/core-web-vitals-astro-case-study/
NEUTRAL	never	URL is unknown to Google	/posts/framer-motion-safari-synthetic-monitoring/
NEUTRAL	never	URL is unknown to Google	/posts/how-to-set-up-local-ai-agent/
NEUTRAL	never	URL is unknown to Google	/posts/lovable-ssr-update/
NEUTRAL	never	URL is unknown to Google	/posts/openclaw-x-skill/
NEUTRAL	never	URL is unknown to Google	/posts/prerender-silent-failure/
NEUTRAL	never	URL is unknown to Google	/posts/why-astro-over-lovable/

Homepage indexed, every post never crawled. That pattern points to one thing — Google found the homepage but couldn't discover the content. Either internal links were broken or the sitemap wasn't being processed.

Checked the sitemap submission in GSC:

URL:       https://www.tedagentic.com/sitemap.xml
Submitted: 2026-05-13
Downloaded: 2026-05-13
submitted: 37 | indexed: 0

There it is. The sitemap was submitted at https://www.tedagentic.com/sitemap.xml. The site canonicalises to https://tedagentic.com/ — no www. Google downloaded the sitemap once, saw 37 URLs, indexed zero of them, and stopped.

Why this breaks indexing

The sitemap URL and your canonical domain need to match exactly. If your canonical is tedagentic.com but your sitemap is submitted from www.tedagentic.com, Google sees an inconsistency. On an established domain with strong trust signals, it might work through it. On a brand-new domain, the mismatch appeared to stop Google from trusting or prioritising the sitemap URLs entirely.

It's not a warning, not a soft signal. The combination of canonical inconsistency, weak new-domain trust, and sitemap ambiguity silently stalled the entire submission.

The fix

Submit the sitemap at the exact URL the site serves on:

service.sitemaps().submit(
    siteUrl='sc-domain:tedagentic.com',
    feedpath='https://tedagentic.com/sitemap.xml'
).execute()

GSC downloaded it within minutes. 47 URLs detected. Posts will start getting crawled within days.

What to verify on any new site

Before assuming a crawl delay is just Google being slow, check three things in GSC:

Sitemap URL matches your canonical domain exactly — www vs non-www, http vs https, trailing slash vs none all count
Last downloaded date is recent — if Google downloaded your sitemap once on launch day and never again, something is wrong
Indexed count is not permanently zero — submitted: 37, indexed: 0 after a week is a hard signal, not a waiting game

A technically clean site should show impressions within a week. If it doesn't, the sitemap submission is the first place to look.

Posting to X From the Server — API Setup, the 402 Wall, and the First Agentic Tweet

Ted — Wed, 10 Jun 2026 03:46:03 +0000

The goal was simple: post to X from the server. No browser, no copy-paste, no manual step. The agent fires the tweet when new content goes live.

Here's what actually happened.

Step 1: X Developer Account

Apply at developer.x.com. Free to create. They ask you to describe your use cases — keep it specific and honest. Mine:

Personal automation on a self-hosted server. Posting content to @tedagentic when new posts go live. Reading my own timeline to avoid duplicates. Monitoring mentions. Single-user, no data storage beyond my own account activity.

Approved same day.

Step 2: App Permissions

This is where most people get stuck. The default app permission is Read only. You need to change it before generating tokens — tokens generated under Read-only stay Read-only even if you change permissions later.

Go to App Settings → User authentication settings and set:

App permissions: Read and Write
Type of App: Automated App or Bot
Callback URI: https://yourdomain.com/ (required field, won't be used)
Website URL: https://yourdomain.com

Save, then go to Keys and Tokens → Regenerate the Access Token and Access Token Secret. The regenerated tokens inherit the new Write permission.

Four keys total:

Consumer Key (API Key)
Consumer Secret (API Secret)
Access Token
Access Token Secret

Store them in ~/.env:

X_API_KEY=your_consumer_key
X_API_SECRET=your_consumer_secret
X_ACCESS_TOKEN=your_access_token
X_ACCESS_TOKEN_SECRET=your_access_token_secret

Step 3: The 402 Wall

First test post returned a 402 — Payment Required.

The X API free tier is read-only now. To write (post tweets), you need paid access. The old "1,500 tweets/month free" tier is gone. What X offers instead is pay-per-use credits — no monthly commitment, load whatever you need. $7 was enough to get started.

Go to the developer portal billing section, add credits, and the write endpoints unlock immediately.

Step 4: The Posting Script

Install the library. The script lives in ~/utils/ so create a package.json there if one doesn't exist, then install:

cd ~/utils && npm init -y && npm install twitter-api-v2

Then the script at ~/utils/tweet.cjs:

const { TwitterApi } = require('twitter-api-v2');
const fs = require('fs');

function loadEnv() {
    const lines = fs.readFileSync('/home/aiserver/.env', 'utf8').split('\n');
    for (const line of lines) {
        const [k, ...v] = line.split('=');
        if (k && v.length) process.env[k.trim()] = v.join('=').trim();
    }
}

async function main() {
    loadEnv();

    const client = new TwitterApi({
        appKey: process.env.X_API_KEY,
        appSecret: process.env.X_API_SECRET,
        accessToken: process.env.X_ACCESS_TOKEN,
        accessSecret: process.env.X_ACCESS_TOKEN_SECRET,
    });

    let text = process.argv[2];

    if (!text) {
        const chunks = [];
        for await (const chunk of process.stdin) chunks.push(chunk);
        text = Buffer.concat(chunks).toString('utf8').trim();
    }

    const tweet = await client.v2.tweet(text);
    console.log('Posted:', tweet.data.id);
    console.log('URL: https://x.com/tedagentic/status/' + tweet.data.id);
}

main().catch(err => { console.error('Error:', err.message); process.exit(1); });

The .cjs extension matters here — the server's root package.json has "type": "module" which breaks require(). Naming it .cjs forces CommonJS mode without touching any config.

Usage:

node ~/utils/tweet.cjs "your tweet text"
# or pipe in
echo "tweet text" | node ~/utils/tweet.cjs

Step 5: First Tweet From the Server

Test post went out. Then the real one — a blog announcement, posted directly from the terminal:

node ~/utils/tweet.cjs "your tweet text here"
# Posted: 2054142254147604629
# URL: https://x.com/tedagentic/status/2054142254147604629

No browser. No copy-paste. The server called the API and the tweet appeared.

What's Next

tweet.cjs is a standalone CLI right now. The next step is wiring it into OpenClaw as a skill — so the agent can post to X directly from a Telegram message. Tell the bot "post this," it calls the script, confirms the URL back to Telegram.

That's the agentic layer: not automation that runs on a schedule, but a tool the agent can invoke on demand. The script is the foundation. The skill is what makes it part of the workflow.

That integration is the next build.

Core Web Vitals on a New Astro Blog: What the Numbers Actually Mean

Ted — Wed, 10 Jun 2026 03:45:25 +0000

I added this blog to the PageSpeed monitoring pipeline last week. The script hits Google's PageSpeed Insights API, parses Core Web Vitals, and fires the results to Telegram.

The report came back:

🤖 tedagentic  99/100
  LCP 🟢 1.7s   CLS 🟢 0   TBT 🟢 0ms

A 99 on mobile for a site that's three weeks old and hasn't been manually optimised for performance. No image compression tuning, no font preloading, no custom cache headers. Just the default Astro build pushed to Vercel.

That number isn't luck. It's the direct consequence of how Astro renders pages. The rest of this post is the breakdown — what each metric is, why this site scores the way it does, and what's missing from the picture.

What Core Web Vitals are

Google uses three field metrics as ranking signals under the Core Web Vitals programme:

LCP — Largest Contentful Paint. How long until the largest visible element is rendered.
INP — Interaction to Next Paint. How long the page takes to respond to a user input.
CLS — Cumulative Layout Shift. How much the page layout moves unexpectedly during load.

These are field metrics — measured from real users in Chrome, aggregated in the Chrome User Experience Report (CrUX). PageSpeed Insights shows both field data (when it exists) and lab data from a simulated Lighthouse audit. For ranking purposes, Google uses field data.

The thresholds:

Metric	Good	Needs Improvement	Poor
LCP	< 2.5s	2.5s – 4.0s	> 4.0s
INP	< 200ms	200ms – 500ms	> 500ms
CLS	< 0.1	0.1 – 0.25	> 0.25

FCP (First Contentful Paint) and TBT (Total Blocking Time) are lab-only diagnostics. They don't feed directly into rankings, but they're useful proxies — TBT in particular is a strong predictor of INP.

LCP: 1.7s

The largest contentful element on most posts here is the post title — an <h1> in plain HTML. No hero image, no above-the-fold video, no JavaScript-rendered component.

When Astro builds a page, it produces a static HTML file. The browser receives a complete document on the first request. The title is in the HTML. There's no render cycle to wait for.

On a CSR React site, the sequence looks like this:

browser request
  → server returns HTML shell (empty <div id="root">)
  → browser downloads JS bundle
  → browser parses bundle
  → React hydrates, renders content
  → LCP element becomes visible

The JS bundle download and parse adds latency before anything is visible. On a fast connection with a small bundle it might be 400–800ms. On mobile, on a slow network, with a bundle that's grown over time, it can be several seconds. That's before any data fetching.

On Astro, the sequence is:

browser request
  → server returns complete HTML
  → browser paints immediately
  → LCP recorded

No intermediate steps. The browser gets the finished document and renders it. 1.7s on mobile is almost entirely network latency and time-to-first-byte from Vercel's CDN edge.

CLS: 0

Cumulative Layout Shift measures visual instability — elements moving around after initial paint. The classic causes: images without dimensions, late-loading fonts that swap, ads injecting above existing content, JavaScript inserting elements above the fold.

This site has no images above the fold on most pages. The fonts (Inter and JetBrains Mono) are self-hosted and loaded via @font-face — they're included in the Astro build and served from the same origin. There's no web font swap from a third-party CDN, no flash of unstyled text from a late-loading Google Fonts request.

There's also no JavaScript inserting DOM elements after paint. No React hydration reshuffling the layout, no analytics widget pushing content down, no cookie banner appearing above the fold.

CLS of 0 is the natural result of serving a page that looks the same the moment the HTML lands as it does after everything has loaded. When nothing changes after initial paint, nothing shifts.

TBT: 0ms

Total Blocking Time measures time spent blocking the main thread during page load — specifically, the sum of time any task takes longer than 50ms. It's a lab proxy for how responsive the page feels.

The main thread gets blocked by long JavaScript tasks. Parsing a large bundle, executing framework initialisation code, running hydration — these all create tasks that lock the main thread and prevent it from responding to user inputs.

This site ships zero JavaScript to the browser by default. There's no bundle to parse. The inline script in the layout is eleven lines for scroll-based header styles and a hamburger toggle — nowhere near 50ms to execute.

┌─────────────────────────────────────────────────────┐
│                 MAIN THREAD ACTIVITY                 │
├─────────────────────────────────────────────────────┤
│                                                      │
│  CSR React blog (typical)                           │
│  ──────────────────────────────────────────         │
│  [parse HTML] [████████████ parse bundle ████] ...  │
│               └── blocking task > 50ms              │
│               TBT: 200–600ms                        │
│                                                      │
│  Astro static (this site)                           │
│  ──────────────────────────────────────             │
│  [parse HTML] [inline script, 11 lines]             │
│                └── < 50ms, not blocking             │
│               TBT: 0ms                              │
│                                                      │
└─────────────────────────────────────────────────────┘

0ms TBT means the main thread is free from the moment the HTML is parsed. Any user input — a tap, a scroll, a click — is handled immediately.

INP: n/a

This is the one that needs explaining.

INP shows as n/a in PageSpeed Insights. That doesn't mean it's zero — it means there's no data. INP is a field metric. It requires real users to interact with the page in Chrome, and that interaction data to be collected in CrUX.

This domain is three weeks old. There isn't enough real-user interaction data for Google to report an INP score. Once the site accumulates CrUX data — typically after a few months of real traffic — INP will appear.

TBT is the lab proxy in the meantime. 0ms TBT is a strong signal that INP will be good when it does appear — the two metrics are closely correlated. A page with no long main-thread tasks during load is a page that responds instantly to inputs.

The distinction matters for how you interpret PSI reports on new sites. A missing INP isn't a gap in performance — it's a gap in data collection. The absence of field data is normal for any site with limited traffic. Lab scores (TBT, FCP, LCP in lab mode) are still meaningful and actionable.

The monitoring pipeline

The script that produced this report runs weekly via cron, checks all four sites, and fires to Telegram via OpenClaw:

SITES = {
    "tedagentic": ("https://tedagentic.com", "🤖"),
    # other sites omitted
}

Each site gets audited against Google's PageSpeed Insights API, scores are stored to a JSON history file for trend tracking, and the report is formatted and sent to Telegram.

┌──────────────────────────────────────────────────────┐
│                  WEEKLY PIPELINE                      │
├──────────────────────────────────────────────────────┤
│                                                       │
│  cron (Sunday 10:30am)                               │
│       │                                              │
│       ▼                                              │
│  speed_test.py                                       │
│       │                                              │
│       └─→ PSI API → tedagentic.com → score          │
│                           │                          │
│                           ▼                          │
│                    history.json (trend delta)        │
│                           │                          │
│                           ▼                          │
│                    OpenClaw → Telegram               │
│                                                      │
└──────────────────────────────────────────────────────┘

The trend delta is the part that matters over time. A score of 99 this week is useful context. A score that drops from 99 to 84 after a deploy is a signal to investigate immediately. The history file tracks the last score per site per strategy — the Telegram report shows the delta alongside the current number.

What the stack choice decides for you

The 99/100 on this blog required no performance work. No audit pass, no image optimisation sprint, no lazy loading configuration.

That's not because Astro is magic. It's because the framework's defaults eliminate the performance failure modes that require active remediation on other stacks.

Zero JavaScript by default means no bundle to block the main thread. Static HTML by default means no render cycle to delay LCP. Self-hosted assets by default means no third-party round trips to create layout shifts.

On a React CSR blog the starting point is roughly the opposite: a JS bundle is required, it blocks the main thread during parse, and LCP is gated behind hydration. Getting from that baseline to a 90+ score requires deliberate work — code splitting, lazy hydration, image optimisation, font strategy. It's achievable. It's also ongoing. Every new dependency added, every new component rendered above the fold, requires re-evaluation.

The Astro baseline is 99. The work is keeping it there, which mostly means not introducing the things that break it: large above-the-fold images without dimensions, third-party scripts in the <head>, client-side components that hydrate on load rather than on interaction.

The score you see in PSI is a consequence of the rendering decision you made when you chose the stack. Fix the stack, and most of the performance work is already done.