DEV Community: tomcate

Making a Tool Site Legible to AI Assistants: GEO in Practice

tomcate — Tue, 22 Sep 2026 10:38:22 +0000

There is a new front door on the web, and it does not have a search box. When someone asks ChatGPT, Perplexity, or Claude for "a tool to convert HEIC photos", the assistant picks from what it can see. If your site is invisible to that process, you are not in the consideration set — regardless of your Google ranking.

This post is the practical checklist we run on ToolVault (166 browser-local developer tools), with the actual crawler numbers from our logs and the mistakes that cost us rework.

First, the uncomfortable physics: AI crawlers do not run JavaScript

GPTBot, ClaudeBot, PerplexityBot and friends fetch HTML and read. They do not hydrate React, they do not execute your bundle, and they will not wait for your client-side rendering to finish. A JavaScript-only tool page arrives at the model as an empty shell.

Our site is statically exported (998 pre-rendered pages), which made this a non-issue — but if your content lives behind client rendering, that is item zero on this list. Test it the honest way: curl your page and look for the actual content in the HTML. Not "view source in DevTools" — the raw bytes a crawler gets.

llms.txt: useful convention, fatal if hand-maintained

llms.txt is the emerging convention for telling AI agents what your site offers. Ours covers the tool catalog, the privacy model, and per-category entry points, in both English and Chinese.

The lesson that cost us a month: generate it at build time from your single source of truth, never hand-edit it. Our first version was hand-written. It rotted in the exact ways hand-maintained files always rot: duplicate section titles after a copy-paste, tool counts drifting from reality ("130+ tools" when we had 144), a Chinese edition that never existed because nobody remembered to add one. An llms.txt that misdescribes your site is worse than none — it confidently teaches models the wrong thing.

Now a build step regenerates it from the same registry that renders the site. Counts are always exact; the file cannot drift because nobody edits it by hand.

robots.txt: decide, explicitly, who you let in

The default posture many sites discover they have (blocking everything unrecognized) or the opposite (allowing everything silently) are both decisions — just unmade ones. Ours is explicit:

User-agent: GPTBot
Allow: /

...for every AI crawler we know about, alongside the standard search bots. This is a values call: we want assistants to be able to recommend local-first tools accurately, because the privacy story is the product. A paywalled-content site might reasonably reach the opposite conclusion. Make the call on purpose.

The reward loop is real, and measurable

After the crawler allowances went live, our nginx logs filled up. The current steady state: 400-1000 AI crawler requests per day — GPTBot, PerplexityBot, ClaudeBot, Amazonbot, PetalBot — systematically walking the catalog. And then the loop closes: referral traffic from perplexity.ai and chatgpt.com starts appearing in the same logs. Small numbers yet, but the direction is established: assistants that can read your site will cite it, and their users click through.

We also push every new page through IndexNow (Bing's protocol, which ChatGPT Search partly sits on). It costs one POST request and removes the discovery delay for the engines that honor it.

What did not matter (for us, so far)

Structured data beyond the basics: we ship standard FAQ and tool schemas, but there is no evidence AI assistants consume JSON-LD preferentially over readable HTML.
Per-prompt optimization tricks: the content that models quote is the content that answers the question plainly. Our per-tool FAQ sections — written for humans — are what surfaces in AI answers.
Submitting to "AI search engines" directories: the crawlers found us through standard crawling plus IndexNow.

The checklist, condensed

Raw HTML contains your content (curl test) — client-rendered sites fix this first
llms.txt generated from the build, never hand-edited
robots.txt with explicit AI-crawler allowances (a decision, not an accident)
IndexNow for every new page
FAQ-shaped content that answers real questions in plain language
Measure: watch crawler UAs in your access logs, then watch for AI-domain referers

None of this is hard. All of it is the difference between being describable and being invisible when someone asks an assistant what to use.

Post #4 in the local-first series: #1 why 166 no-upload tools, #2 browser-side AI segmentation, #3 merging PDFs without a server. All tools: ToolVault.

Merging PDFs Without a Server: The Bytes Under a Structural Merge

tomcate — Mon, 21 Sep 2026 04:48:59 +0000

My last post covered running AI in the browser. This one is about the quieter member of the local-first family: PDF processing — merging, splitting, compressing — and what actually happens to the bytes when no server is involved.

The punchline first: a proper PDF merge does not "read" your documents the way a PDF viewer does. It rearranges serialized objects. Understanding that one sentence explains most of the behavior people find surprising in merge tools — and it shows why the whole operation fits comfortably in a browser tab.

A PDF is a database, not a document

Strip the format mystique away and a PDF file is a miniature object database: a header, a trail of objects (pages, fonts, images, streams), a cross-reference table pointing to each object byte offset, and a trailer. "Opening" a PDF means parsing that xref table, loading the page tree, and resolving references. Rendering is just one thing you can do after parsing.

Which means the three classic operations decompose like this:

Merge = copy the page objects of file B into file A's object space, remap their references, rebuild one xref table
Split = the mirror image: keep a subset of the page tree, carry over only the objects those pages reference
Compress (for scanned PDFs) = re-encode each page's embedded image stream (JPEG quality is your dial)

None of that requires rendering a single pixel. That is why our merge tool describes itself as "structural": pages come out byte-identical, text stays selectable, and a vector chart does not degrade, because it was never rasterized in the first place.

What pdf-lib gives you — and what it makes obvious

In the browser this is all pdf-lib, which is pure JavaScript with zero dependencies on native code or workers. The merge core is almost disappointingly small:

const out = await PDFDocument.create();
for (const file of files) {
  const src = await PDFDocument.load(await file.arrayBuffer());
  const pages = await out.copyPages(src, src.getPageIndices());
  pages.forEach(p => out.addPage(p));
}
const bytes = await out.save();

copyPages is where the object-remapping happens. Two things you learn from living with this API:

Page order is list order. There is no implicit sorting, no filename intelligence. If users expect page 10 to land after page 9, your UI has to enforce it — which is why reorder arrows exist and why "list order becomes page order" is the honest description.

What the tool does not preserve is as informative as what it does. Bookmarks that point to page positions shift. Form fields sharing a name across documents can collide — pdf-lib will not deduplicate them for you. Encrypted files need decryption before load or they throw. None of these are bugs in the library; they are the format showing its database nature.

Compression is where honesty matters

The interesting decision in a browser PDF compressor is what to do when compression makes the file bigger.

Scanned PDFs — one big image per page — compress beautifully: re-encode each page image as JPEG at quality 50-65 and 60-90% of the size evaporates, because the 300 DPI scan carries resolution nobody consumes. Vector PDFs generated by Word or LaTeX are already tiny; re-rendering them as images inflates them.

So the honest compressor has three behaviors: compress scans aggressively, and when the output grows, say so (+12% on the button, not a silent download) and point users toward splitting instead. Tools that always claim victory are lying to somebody.

The browser implementation detail that surprised me: canvas-mediated JPEG re-encoding of page images is where the quality dial actually lives, and quality-to-size is emphatically not linear — the useful cliff for scanned documents sits between 50 and 65, where halving the quality costs you single-digit percent of visual fidelity.

The privacy argument, restated for files people actually have

For portrait photos, local processing is a nice property. For PDFs it is closer to the whole point, because the documents that trigger PDF operations are disproportionately the sensitive ones — contracts under negotiation, visa applications, bank statements. The flow "I need to merge two files" should not have a mandatory step of "upload my signed contract to a random web service."

The verification is the same as always: DevTools, Network panel, watch zero requests leave while you work. Our complete guide to the PDF toolkit goes deeper on when to merge vs split vs compress — with the same no-upload guarantee.

What is hard in the browser (an honest list)

Encrypted PDFs: pdf-lib supports loading some variants but not decrypting all of them; the correct UX is a clear error, not a partial file
OCR: converting scanned pages back to selectable text needs a heavy model; we shipped without it rather than fake it
Very large merges: a few hundred pages works; ten thousand stresses mobile Safari memory — structural merging is cheap, but the file still has to exist in RAM once

None of these are server-shaped problems pretending to be local. They are the honest boundaries of the approach, and knowing them is the difference between a local tool and a local toy.

This is post #3 in the local-first series: #1 why we built 166 no-upload tools, #2 browser-side AI segmentation. All 166 tools live at ToolVault.

Running AI Portrait Segmentation Entirely in the Browser — Bugs Included

tomcate — Sun, 20 Sep 2026 04:06:40 +0000

My first post here mentioned that our ID-photo tools run portrait segmentation locally — a passport photo never leaves your machine. A few readers asked how that actually works, so here is the deep dive, including the three bugs that nearly shipped to production.

The stack is deliberately boring

MediaPipe Selfie Segmentation, compiled to WASM, loading a 244KB tflite model from our own origin (no CDN, no Google API key, works offline once the PWA is installed). The pipeline:

File → createImageBitmap → <canvas> → segmenter.segment(canvas)
     → categoryMask (Uint8Array) → custom mask pass → composite → download

Everything after the file picker is client-side JavaScript. The "server" is nginx serving static files — there is nothing to send data to.

Bug #1: The mask was inverted, and the specs were no help

The segmentation call returns a categoryMask. Intuition says: non-zero pixels = person, zero = background. MediaPipe's convention is the opposite: 0 is the person.

We shipped the intuitive version. The result: users uploaded a portrait, picked "blue background", and got a photo where the person was painted solid blue and the background stayed untouched. The bug report was a screenshot of a smurf.

The fix is one line — invert the predicate — but the lesson stuck: when a library hands you an opaque byte array, write a 10-line visualization dump before building anything on top of it. We now render the raw mask as grayscale before every mask-processing change.

Bug #2: The 1-pixel white halo that took three rounds to kill

After recoloring, every hair edge showed a white fringe. Classic cause: the mask boundary is a hard 0/1 cut, but anti-aliased pixels at the boundary are half-person, half-background.

Round 1: dilate the person region by 1px. Better, halo narrower.
Round 2: 2px. Better still, but now ears looked slightly "eaten".
Round 3 — the actual fix was two changes:

Dilate the background side by 3px (paint background color slightly into the person), and
Remove the feathering entirely.

That second one is counter-intuitive. We had added edge feathering (alpha-blend the mask boundary) for softness — but feathering blends the new background color at reduced opacity, which is exactly what a translucent white-ish fringe is. Hard edges beat pretty edges here; hair detail survives better with a slightly aggressive dilate than with a soft, diluted boundary.

Bug #3: Safari and `createImageBitmap`

createImageBitmap(file) — the clean, efficient entry point — is unavailable in older Safari. The fallback is an <img> element with objectURL, awaited through img.decode():

const source = typeof createImageBitmap === 'function'
  ? await createImageBitmap(file)
  : await new Promise((res, rej) => {
      const img = new Image();
      img.onload = () => res(img);
      img.onerror = rej;
      img.src = URL.createObjectURL(file);
    });

Both paths end in the same canvas.drawImage, so downstream code never knows the difference. Feature-detect at the edge, normalize immediately.

What "local AI" buys you in practice

A passport photo with face, ID number, and biometric pattern never touches a server — not because of a policy, but because there is no upload code path at all. F12 → Network proves it live.
It works on a plane, behind the great firewall, and on a locked-down corp machine.
Model inference at ~100-300ms per photo on a mid-range phone. Users cannot tell it is not a server call.
Our "server" cost for this feature is 244KB of static bandwidth.

None of this requires a special product. It requires rejecting the default architecture where "AI feature" means "API call".

Try it

The ID-photo background changer is live (pick any color, standard sizes for Chinese/US/visa formats, print layouts):
ToolVault ID Photo

If you are building client-side ML: what's the weirdest silent failure you have debugged in a byte array? That mask inversion took us a user screenshot to find — I am curious what yours took.

More local-first tools: 166 tools, zero uploads. First post in this series: why we built it.

I Built 166 Developer Tools That Never Send Your Files to a Server

tomcate — Tue, 15 Sep 2026 06:41:06 +0000

Every time I needed to convert a HEIC photo, merge two PDFs, or resize an ID photo, the search results led to the same kind of site: "Upload your file here." A passport photo. A contract PDF. A spreadsheet with real data. All shipped off to some unknown server, processed, and hopefully deleted.

I wanted the opposite. So I built ToolVault — 166 developer and productivity tools where every computation runs in your browser and zero bytes of your input leave your machine.

What's in the box

Data conversion: JSON ↔ YAML ↔ XML ↔ CSV ↔ Excel, Excel → Word, Markdown → Docx
Image tools: HEIC/WebP/AVIF/SVG conversions, batch resize, images → PDF
PDF suite: merge, split, PDF → JPG, compress — all via pdf-lib, all client-side
ID photo tools: background replacement, standard sizes, print layouts. Portrait segmentation runs a 244KB MediaPipe tflite model in the browser — your photo never touches a server
Dev staples: JWT decoder, regex tester, Cron parser, CIDR calculator, hashing, AES/RSA encryption, diff, code formatters
Workflows: chain multiple tools into a pipeline, so you stop copy-pasting between tabs

The architecture is the privacy policy

Most "online tools" are thin clients over a backend conversion service. ToolVault is statically exported Next.js — 998 pre-rendered pages served as plain files. There is no backend. No database. Nothing to leak, nothing to breach, nothing to subpoena.

That sounds like a slogan, so here's how to verify it yourself: open any tool, hit F12, watch the Network panel while you use it. Upload bytes: 0. Not "we promise" — observable.

A few implementation notes that made this work at scale:

Library choices matter. Everything runs on battle-tested client-side libraries: pdf-lib, pdf.js, SheetJS, heic2any, jszip, marked. The hard part is composing them (e.g., Safari's createImageBitmap gaps, which needed an <img> decode fallback).
Offline is a feature, not a gimmick. It's an installable PWA. Install it once, disconnect your network, and all 166 tools keep working. "Works on a plane" is the honest privacy demo.
Segmentation without the cloud. The ID-photo background remover runs MediaPipe Selfie Segmentation locally via WASM. The trickiest bug wasn't the model — it was realizing the category mask semantics were inverted from what I assumed (0 = person, not background), and a 3px background dilation needed to kill halo artifacts.

Why this matters more than it seems

When a tool uploads your file, you're making a trust decision — about their security, their employees, their acquisition by someone worse. When the computation is local, there is no trust decision. The browser sandbox is the policy.

This is also where tool sites are heading: WebAssembly and mature client-side libraries have removed most reasons for a conversion backend to exist. The remaining reason is usually ad-driven engagement ("watch our progress bar while we upload").

Try it

👉 https://www.wcytcn.com — no signup, no login, no upload. Your bookmarks are the account system.

If you have a tool you use weekly that doesn't have a "never uploads" version, tell me in the comments — that's my backlog.

DEV Community: tomcate

Making a Tool Site Legible to AI Assistants: GEO in Practice

First, the uncomfortable physics: AI crawlers do not run JavaScript

llms.txt: useful convention, fatal if hand-maintained

robots.txt: decide, explicitly, who you let in

The reward loop is real, and measurable

What did not matter (for us, so far)

The checklist, condensed

Merging PDFs Without a Server: The Bytes Under a Structural Merge

A PDF is a database, not a document

What pdf-lib gives you — and what it makes obvious

Compression is where honesty matters

The privacy argument, restated for files people actually have

What is hard in the browser (an honest list)

Running AI Portrait Segmentation Entirely in the Browser — Bugs Included

The stack is deliberately boring

Bug #1: The mask was inverted, and the specs were no help

Bug #2: The 1-pixel white halo that took three rounds to kill

Bug #3: Safari and createImageBitmap

What "local AI" buys you in practice

Try it

I Built 166 Developer Tools That Never Send Your Files to a Server

What's in the box

The architecture is the privacy policy

Why this matters more than it seems

Try it

Bug #3: Safari and `createImageBitmap`