DEV Community: partick

My AI-Built Site's Internal Links Were Unmanageable — So I Mapped the Whole Graph Before Touching Anything

partick — Fri, 09 Oct 2026 10:02:02 +0000

My site is built largely by AI now — code and content. It's fast, and I'm not going back. But a few months in I noticed something nobody warns you about: internal linking is the one lever AI never managed, because it's site-wide state, not page state.

Every page was generated in isolation, with local context only. Each one looks well-linked — sensible related links, breadcrumbs, a nav menu. Nobody, me or the model, was keeping a ledger of the whole graph. Multiply that by hundreds of pages and you get what I got:

pages competing for the same queries, each hoarding a slice of the in-links that should have gone to one canonical page;
orphans with zero content in-links, invisible to ranking and to me;
dead ends that link nowhere, wasting their equity;
and no way to answer the basic question: which page should I strengthen, and where would those links even come from?

I'm planning a round of link consolidation — merging near-duplicate pages and re-pointing links toward the pages that already rank. Before editing anything, I wanted to see the actual graph: every from → to, from rendered HTML, not from what the templates claim they do.

Why the obvious tools didn't fit

I looked for an internal link visualization tool I could just point at a site. Commercial crawlers give you a report about your links; I wanted the raw graph — the kind you can diff after a change, join against Search Console data, or hand to an LLM. Homegrown scripts got one thing consistently wrong: navigation. Header/footer links appear on every page, so raw counts say "everything links to everything" and hide the ~5% of links that actually carry meaning.

So I built the instrument I needed, and open-sourced it: internal-links-export. It crawls a site (sitemap or BFS fallback), separates site-wide navigation from content links using a data-driven rule (any target linked from ≥ 90% of crawled pages is treated as chrome — no hardcoded nav lists), and compiles everything into a single offline HTML report plus exports. Zero dependencies, MIT. That's the whole pitch; the rest of this post is what the graph actually told me, because that's the part that transfers to your site.

What the graph showed

My own site — petguidecalc.com, an English pet-care guide and calculator site, 231 pages, AI-built like the rest — produced a sobering but manageable picture: the orphans, the topic twins splitting in-links, the uniform hubs. But I didn't want an instrument that only works at that scale, so I stress-tested it against a deliberately hostile target I don't own: a large third-party e-commerce site whose link graph resolves to 4,868 pages, 3,931 of them flat product URLs at /products/{slug} — no category in the path, nothing to group by. As one graph: hairball. As a table: meaningless.

The tool recovered the de-facto taxonomy from the link structure itself: every non-hub page grouped under its strongest content in-link source — whichever category page links to it most. That reconstructed 268 category groups covering 92% of products, each small enough to actually read. This was the moment the tool became useful: the site's real information architecture exists in its links, even when it exists nowhere else.

Three views carry most of the diagnostic weight:

In-degree distribution — which pages hoard equity (hubs linking in from everywhere) and which are starved. My orphans and near-orphans fell out of this list immediately.
The category group graph — double-click into any group and see the category page, its member pages, and exactly which members link back. The missing back-links are the cheapest wins in the whole audit.
Cross-cluster flows — where topic silos accidentally link to each other, and where they never do.

The consolidation plan this produces

With the graph exported (pages CSV + edge list CSV), the workflow I'm running is:

Join pages against Search Console — impressions and average position per URL. This splits every page into "earns traffic" / "exists but starved" / "competes with a sibling".
Find merge candidates — same-topic pages whose in-links are split between them. Merge the weaker one in, redirect it, and re-point every inbound link to the survivor. The edge list makes this mechanical: filter to == weak-page, and you have the exact edit list.
Re-point links toward the pages that rank — for each strong page, look at its topic neighborhood in the graph and find pages that link generically (nav-only) or to the wrong sibling. Specific links from relevant pages, not more links from everywhere.
Fix the floor first — every orphan gets at least one content in-link from its topic hub; every dead end links out to its siblings. These are one-line edits each, and they're the ones nothing else was going to catch.
Re-crawl and diff — run the crawler again after changes and diff the edge list CSV. The graph becomes a regression test for information architecture: consolidation should show in-links concentrating on fewer, stronger pages.

Step 5 is why I cared so much about exports rather than a pretty report. A graph you can't diff is a screenshot; a graph you can diff is an instrument.

Where AI comes back into it

The irony isn't lost on me: the mess came from page-level generation with no global view, and the cleanup is now AI-assisted too — but this time the model gets to see the whole graph. Export internal links as an edge list (one row per from, to, count, type), hand it to an LLM with the Search Console table, and ask for merge candidates and re-pointing proposals. It's the same trick as the generation failure, reversed: global state supplied as context. GEXF and GraphML exports are there for the days I want to look at it in Gephi instead.

Notes from the trenches

Sites behind Cloudflare-style challenges can't be fetched from Node (TLS fingerprint mismatch — a valid cf_clearance cookie doesn't help). The working path: pass the challenge in a real browser, then inject that page's fetch into the collector. That recovered 930/931 sitemap pages on the test site; the one failure was a login-only page, recorded honestly in the crawl output.
Dense category trees (one 104-node cluster had 1,930 breadcrumb links) need a deterministic layout, not force-directed — force layouts collapse them into a ball. The tool switches automatically and renders that cluster in ~265 ms.
The report UI's chrome text is currently Simplified Chinese; paths and graphs are language-neutral. An English UI is on the roadmap — PRs welcome.

The repo is partick33/internal-links-export — crawler, single-file report, CSV/GEXF/GraphML/JSON exports, 12 tests, MIT. It's a means, not a product: the point was never the tool, it was getting the graph on the table before restructuring a site that grew faster than its linking could keep up.

Jev Was Full, So My Agent Moved Into My Browser Instead

partick — Wed, 23 Sep 2026 09:42:13 +0000

"Whoops, we're full - check https://x.com/typesafeai for more information!"

too much demand, one week in

That's what TypeSafe's sign-up page told me on September 23. Jev Ultrafast itself is just an MIT-licensed repo — free to clone, and its first week was loud: zero to 18,392 stars, roughly 2,600 a day. But the agent runs on Jev, a TypeSafe model, and the API key for it only comes through TypeSafe's early access. That's what was full. I wanted to try it. I couldn't — and a week in, I still can't.

I had a real need for browser automation, though. So I let my coding agent move into my own browser instead, and that same week, it shipped real work — the first visual usage guide for one of my site's calculator pages, screenshots, annotations, and all.

The tool that moved in is ego lite, citrolabs' open-source, agent-native Chromium — the agent works in its own isolated workspace (ego calls them Spaces), so my browser stayed mine while it worked. The site is petguidecalc, a stack of pet-care guides and calculators; every tool page deserves a visual usage guide — annotated screenshots — and I was never going to hand-make those at that scale. Page one was this reptile UVB calculator: the agent drove the dev deployment, captured and annotated the three steps, and handed me a diff. One command, start to finish — the rest of this post is how that command works and why it's fast.

What Jev Ultrafast actually is

The model doing the deciding is Jev, TypeSafe's "System One Model"; the browser-use org wraps it in an open-source agent billed as the "fastest and cheapest web agent." The design, in their own words: "Think of Jev as a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out." Each cycle, the model emits exactly one typed decision — which operation, on which element — and only TYPE_TEXT leans on a small LLM. The full operation set: CLICK, TYPE_TEXT, SELECT, SCROLL_UP/SCROLL_DOWN, WAIT, DONE, BLOCKED.

The part I find genuinely interesting is the safety story: "Model output never becomes selectors, coordinates, shell commands, or executable JavaScript." Nothing the model says is directly executable. Their reported numbers: a Google Flights search completed in 7,073 ms, protocol calls down from 1,092 to 101.

The repo is also unusually honest about the edges — "Shadow roots, frames, canvas, uploads, pop-up tabs, nested scrolling, and arbitrary keyboard widgets remain outside this MVP" — and about pricing: input at $0.042 per million tokens, output free ("too cheap to meter"), with the admission that they can't prove that price isn't subsidized.

Fast brain, fast hands

Here's where the "Jev alternative" framing breaks down: strictly speaking, they're not competitors — they don't even live in the same layer. Jev is a decision model: every cycle, it picks the operation and the element. ego lite is an execution environment: a real Chromium that any coding agent — Claude Code, Codex, Cursor, your own — can move into and work from. Fast brain, fast hands. Jev decides where to click; ego lite is where the clicking happens.

Once I saw it that way, my problem sorted itself out. My task was "drive the dev build of a site I own through three steps I already know, and screenshot each one." The deciding is trivial — I wrote the page. What was actually scarce was the execution side: a browser the agent could treat as its own, isolated from mine, that I didn't have to babysit. I didn't need a faster brain. I needed fast hands.

Why it feels fast: code base, not CLI base

The first thing I noticed was the speed. My previous setup used a Chrome extension to drive the browser, and ego lite is clearly faster in day-to-day use. That's a felt sense, not a benchmark — what the project itself claims, I'll get to in a moment.

Their explanation is "Code base, not CLI base." In a CLI-driven loop, every round trip costs a model inference: generate an action, execute it, observe the result, generate the next. ego lite instead exposes the browser as in-page JavaScript tools — snapshot, fill, click, wait, navigate, capture — so the agent writes one whole program and the page runs it in a single pass. The multi-step loop collapses into one piece of code. As the README puts it, "the agent gets to do what it does best: write code" — and code is the cheapest, fastest thing an LLM produces.

What the project does benchmark against is Vercel's agent-browser: up to 2.5× faster on four complex tasks, with a large drop in token use. That's their self-reported number, not mine. The only claim I'll stand behind is the felt one above: my guide pipeline ran noticeably faster than the Chrome-extension setup it replaced.

From Live Calculator States to Publish-Ready Guides

The UVB calculator works like this: pick the species and the lamp type, enter the lamp's wattage and how far it sits from the basking spot, and the page judges the combination — a result badge that reads OK when the values land in the recommended range. That's why the screenshots were the hard part. No verdict exists on a freshly loaded page; the states worth capturing only exist mid-use, form filled in, result on screen. And the values can't be random ones — they have to be a combination the calculator actually judges OK, so the screenshot teaches the ideal setup. By hand that means clicking through, screenshotting, opening an image editor to draw highlight boxes over the right controls, exporting, and redoing all of it whenever the page's layout shifts. Multiply that by every tool page on the site and it simply never gets done.

So the capture went to the agent as code. The script it wrote drives the dev deployment through ego lite: load the page, walk the three steps, iterate the demo input values until the result badge reads "OK," then screenshot each state. Alongside each screenshot it records the controls' bounding boxes via getBoundingClientRect(), printed as JSON.

The capture had one genuinely tricky bit. ego lite's built-in screenshot() ignores the device-scale-factor override, so the first pass came out 1× and soft. The fix is to set the metrics yourself over CDP, and take the shot over CDP too:

await page.cdp('Emulation.setDeviceMetricsOverride', {
  width: 1280,
  height: 1200,
  deviceScaleFactor: 2,
  mobile: false,
  screenWidth: 1280,
  screenHeight: 1200,
  screenOrientation: { angle: 0, type: 'landscapePrimary' },
});
const r = await page.cdp('Page.captureScreenshot', { format: 'png' });

After that: crisp 2× captures, and enough resolution for the 1024-wide srcset variant.

A second script then runs each capture through sharp, drawing the highlight boxes from those recorded bounding boxes — where to circle is data, not eyeballing — in the site's brand colors, and exports three WebPs: 44 KB, 130 KB, and 127 KB.

The last piece is the one I care about most as a site owner: the guide's content lives in a single data object, and both renderings are generated from it — the HTML page (zero client-side JavaScript, images kept out of the LCP path) and a plain-Markdown twin for machines. One source, two renderings; they can't drift apart because neither is hand-written.

From one-off to standing SOP

The part I'd underline for anyone running agent workflows: the real deliverable wasn't the UVB guide — it was the process. Once the page shipped, the flow got distilled into a 99-line skill file — a template my agents load on demand — so "add a visual usage guide to page X" is a standing capability now, not a bespoke session. The next page is one sentence away.

The repo is private, so I can't link the file — but the skeleton is the transferable part. Here it is as a template:

1. **Triggers live in the frontmatter.** The skill lists its own trigger phrases — "usage guide", "how to use", 图文操作指南 — so when I next say the words, it takes over. No re-explaining the pipeline.
2. **"When to use it" states the SEO reason in one line.** Visible step text is the keyword carrier — Google doesn't index text inside screenshots — so the words carry the keywords and the screenshots are the differentiator. The same section rules out video and GIFs: payload weight, LCP, and no rich-result eligibility.
3. **Copy rules that survive without me.** H2s start with a verb (the "How to use" phrasing is explicitly banned), the intro carries the "free in your browser" value prop, exactly three imperative steps, and control names are copied verbatim from the page's data file — never from memory.
4. **Three landing spots, one source.** The guide fields in the data object, the Astro rendering, and the Markdown twin all come off the same data — the same-source rule from the pipeline above, encoded as a skill move.
5. **A capture runbook.** For a new page: copy the template script, change three parameters, aim it at the dev deployment, and wait for the interactive island to mount before touching anything.

The first run paid for the skill; every page after that rides on it. And the skeleton isn't pet-site-specific — if you build tool pages, borrow it, fit it to your stack, and "add a usage guide to page X" becomes a one-sentence job on your side too.

Where this goes next

My takeaway isn't "ego lite replaces Jev." It's that the barrier to this kind of automation has collapsed: one command did what I'd never have sustained by hand. The two tools live in different layers, but they point at the same future — agents that operate real software instead of describing it. I'd rather be learning that machinery now, on my own site, than reading about it while I sit on TypeSafe's waitlist.

I'm exploring AI-automated site operations next — more pages through the guide pipeline, and whatever else the agent can own end to end. If you're running similar experiments on your own site or product, I'd genuinely like to compare notes in the comments.

The site all of this runs on: petguidecalc.com