DEV Community: AI Dev Hub

LangChain vs LlamaIndex vs Chonkie: same 94-page PDF

AI Dev Hub — Thu, 20 Aug 2026 14:00:03 +0000

LangChain vs LlamaIndex vs Chonkie: same 94-page PDF

LlamaIndex. Its SentenceSplitter gave me the best recall@5 (0.86, against 0.79 for LangChain's recursive splitter) on a 94-page policy PDF, without writing a custom separator list. LangChain wins if you're already deep in LCEL. Chonkie is the fastest by a wide margin and the one I'd pick for a batch job over a million documents. Chunk size mattered more than the library did.

Quick disclosure before anything else: the RAG chunk size calculator I link to below is one I built. I got tired of re-deriving the same token math in a scratch file, and the four existing pages I found all assumed OpenAI's tokenizer and 1,000-character chunks. Mine is free, runs entirely in your browser, no signup, nothing uploaded. If you know a better one, tell me and I'll link it instead.

The task: one 94-page policy PDF and 38 questions

Last Tuesday I got handed a commercial property insurance policy and a support inbox. The ask was ordinary: answer questions like "what's the deductible for wind damage during a named storm" without a human reading 94 pages every time.

I pulled the text with pdftotext -layout, which gave me 214,883 characters. Then I sat down and wrote 38 questions by hand, each paired with a "needle" string that appears on exactly one page. That labelling took 47 minutes and it's the only reason any number below means anything. A chunking benchmark without labels is just vibes with decimal places.

Same setup for every run: BAAI/bge-small-en-v1.5 as the embedding model, normalized vectors, cosine similarity, top 5 results. The metric is recall@5, the fraction of questions where at least one of the top 5 chunks contains the needle.

Here's the scoring script. No framework, no vector database, just numpy:

# eval_chunks.py - score a chunker by recall@5 on hand-labelled questions
import json, time
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BAAI/bge-small-en-v1.5")

def recall_at_k(chunks, questions, k=5):
    cv = model.encode(chunks, normalize_embeddings=True, batch_size=64)
    qv = model.encode([q["text"] for q in questions], normalize_embeddings=True)
    hits = 0
    for q, v in zip(questions, qv):
        top = np.argsort(-(cv @ v))[:k]
        if any(q["needle"].lower() in chunks[i].lower() for i in top):
            hits += 1
    return hits / len(questions)

def score(name, split_fn, text, questions):
    t0 = time.perf_counter()
    chunks = split_fn(text)
    dt = time.perf_counter() - t0
    r = recall_at_k(chunks, questions)
    print(f"{name:11} chunks={len(chunks):4}  split={dt:6.2f}s  recall@5={r:.2f}")

if __name__ == "__main__":
    text = open("policy.txt", encoding="utf-8").read()
    questions = json.load(open("questions.json", encoding="utf-8"))

    from langchain_text_splitters import RecursiveCharacterTextSplitter
    lc = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
        chunk_size=512, chunk_overlap=64)
    score("langchain", lc.split_text, text, questions)

    from llama_index.core.node_parser import SentenceSplitter
    li = SentenceSplitter(chunk_size=512, chunk_overlap=64)
    score("llamaindex", li.split_text, text, questions)

    from chonkie import RecursiveChunker
    ch = RecursiveChunker(chunk_size=512)
    score("chonkie", lambda t: [c.text for c in ch(t)], text, questions)

Three lines of output. Then you get to argue with them.

LangChain: I was wrong about this one for an hour

First run, LangChain came dead last. 0.44 recall against LlamaIndex's 0.86. That gap felt too big to be real, and it wasn't.

RecursiveCharacterTextSplitter(chunk_size=512) counts characters. SentenceSplitter(chunk_size=512) counts tokens. Same parameter name, same value, and for English prose that's roughly a 4x difference in how much text lands in each chunk. My careful apples-to-apples comparison was quietly pitting 512-character chunks against chunks of about 2,000 characters. I'd made this exact mistake in April on a different project and still walked straight into it again.

Switching to RecursiveCharacterTextSplitter.from_tiktoken_encoder(chunk_size=512, chunk_overlap=64) fixed it: 137 chunks, 0.79 recall, 2.8 seconds. Perfectly respectable.

The thing that still bugs me is what that helper counts with. It pulls in tiktoken and defaults to cl100k_base, an OpenAI tokenizer. I'm embedding with a BERT-family model that uses WordPiece. On plain English the two agree within about 8%, so nothing explodes, but on code, JSON blobs or non-Latin scripts the drift gets ugly and your "512-token" chunks start overflowing a 512-token encoder. Silent truncation, no warning.

Import paths are also a mess if you're following an older tutorial. It's langchain_text_splitters now, and half the search results still say from langchain.text_splitter import.

LlamaIndex: the boring one that won

SentenceSplitter at 512 tokens with 64 overlap: 129 chunks, 0.86 recall, 4.1 seconds. It was the slowest of the three on a single document because it actually runs a sentence tokenizer before doing anything else.

That sentence tokenizer is the whole reason it won. Insurance definitions read like "Named Storm means any storm or weather disturbance that is named by the National Weather Service." Cut that in the middle and neither half retrieves for a question about named storms. The recursive splitters get this right most of the time via their separator list, but "most of the time" across 129 chunks is a handful of dead ones.

Two things I didn't love. The install is heavy: llama-index-core was 41 MB in a fresh venv against 2.9 MB for chonkie. If chunking is the only thing you want, that's a lot of framework to carry.

Bigger issue is the defaults. SentenceSplitter() with no arguments is chunk_size=1024, chunk_overlap=200. On my document that scored 0.68. Nobody's default is tuned for your corpus, and this one is off by enough to matter.

Chonkie: fast, small, and it surprised me

Chonkie is the small library in this comparison and I expected it to lose. It didn't. RecursiveChunker at 512 tokens gave 133 chunks and 0.82 recall in 0.38 seconds. That's within noise of LlamaIndex on quality and roughly 10x faster.

Speed stops being an academic concern once the corpus grows. I ran all three over the client's full document set (1,240 files, 2.1 GB of extracted text). Chonkie finished in 47 seconds. LangChain took 3 minutes 4 seconds. LlamaIndex took 6 minutes 12 seconds. For a one-time ingest, who cares. For a nightly re-index that has to finish before the morning, that's the difference between a cron job you forget about and a Slack alert at 2am.

I also tried SemanticChunker, which embeds each sentence and splits where similarity drops. 0.84 recall, 11.3 seconds on one document. Thirty times the cost for two points I can't distinguish from measurement noise on 38 questions. On a genuinely mixed corpus it might earn its keep. Here it didn't.

The rough edge is polish. The API moved between the version most blog posts describe and the 0.5.x I installed, so half the snippets I found were wrong. And when I passed a tokenizer name it didn't recognise, I got a bare KeyError out of a dict lookup with no hint about what the valid names are. That cost me twenty minutes I'd rather have spent elsewhere.

The scores, and which one I'd actually ship

Criterion	LangChain	LlamaIndex	Chonkie
Best recall@5, 38 questions	0.79	0.86	0.82
What `chunk_size` counts by default	characters	tokens	tokens
Chunk one 94-page doc	2.8s	4.1s	0.38s
Chunk 1,240 docs	3m 04s	6m 12s	47s
Install size, fresh venv	8.4 MB	41 MB	2.9 MB
Error on a bad tokenizer name	named ValueError	named ValueError	bare KeyError
Found the answer in docs under 2 min	yes	yes	no
Defaults I'd ship unchanged	no (characters)	no (1024/200)	yes

I'd use LlamaIndex's SentenceSplitter for anything under roughly ten thousand documents where answer quality is the point. Above that, or anywhere re-indexing runs on a schedule, Chonkie. If your pipeline already lives in LangChain, stay there and just call from_tiktoken_encoder explicitly, because the character default will bite you and it won't be loud about it.

Here's the finding I keep coming back to, though. Across the three libraries at their best settings the spread was 0.07. Across chunk sizes with a single library, it was 0.18: 256 tokens scored 0.74, 512 scored 0.86, 768 scored 0.81, 1024 scored 0.68. The parameter beat the vendor by more than double. If you're agonising over which splitter to import before you've swept chunk size, you're optimising the wrong variable.

That gap is why I built the RAG chunk size calculator. Feed it your embedding model and the kind of document you're indexing, and it hands back a starting chunk size and overlap along with the token math (context window, characters per token for your tokenizer, how many chunks that means for a document of size N). It's a starting point, not an oracle. You still need your own 30 labelled questions.

FAQ

Q: Does overlap actually help, or is it cargo cult?

A: It helps, less than people assume. Zero overlap scored 0.79, 64 tokens scored 0.86, 128 tokens scored 0.87 while producing 22% more chunks to store and search. I settled on 64, about 12.5% of chunk size, and that ratio has held up on two other projects since.

Q: Why not just use semantic chunking everywhere?

A: Because on this document it bought 0.02 recall for 30x the processing time. Semantic chunking pays off when a single file jumps between unrelated topics. A structured policy document already has that structure in its headings.

Q: Will these numbers hold for my documents?

A: Almost certainly not, and I'd be suspicious of anyone who told you otherwise. Legal and insurance prose is dense, repetitive and full of defined terms, which flatters sentence-aware splitting. Chat logs or source code behave differently. Copy the script above, label 30 questions of your own, rerun it. Mine took 47 minutes.

Q: What about markdown and code files?

A: Different tools. Use a header-aware splitter for markdown so you keep section context attached, and Chonkie's CodeChunker (or a tree-sitter based splitter) for source, so functions stay whole. Splitting code on blank lines destroys exactly the boundaries you want to retrieve on.

Written with AI assistance and human review. Try the tool at aidevhub.io/rag-chunk-calculator.

A valid sitemap.xml for 1,247 URLs with a free generator in 2026

AI Dev Hub — Thu, 13 Aug 2026 14:00:03 +0000

A valid sitemap.xml for 1,247 URLs with a free generator in 2026

Paste your URLs into a client-side sitemap generator, add lastmod, and ignore changefreq and priority. Google ignores both, and has for years. A valid sitemap.xml is nine lines of boilerplate plus one url block per page, capped at 50,000 URLs or 50MB uncompressed. The XML is the easy part. Keeping the list free of redirects and 404s is what actually moves crawl budget.

Up front: the sitemap generator I link to below is one I built. I tried six existing ones in March 2026 and every one either uploaded my URL list to a server or capped the free tier at 100 URLs. Mine is free, runs entirely in the browser, no signup, and nothing leaves your machine. If you know a better one, tell me and I'll link it instead.

The migration that shipped 412 dead URLs

In February 2026 I moved a docs site off a flat /guides/ structure and onto /docs/<version>/. Around 1,247 pages. The sitemap came from a shell pipeline I wrote back in 2023: find the build output and wrap each line in <loc> tags with sed. It had worked for three years without a single complaint.

It kept working after the migration. That was the problem.

find walked the old dist/guides/ directory, which the new build never cleaned out. So the sitemap listed 412 URLs that returned 404 sitting next to the 1,247 real ones. Search Console flagged it 9 days later under "Submitted URL not found". By then Googlebot had burned a chunk of its crawl budget on pages that didn't exist, and half the new /docs/ tree was still unindexed.

I fixed it the dumb way first: rm -rf dist/ at the top of the build. That killed the stale files and immediately created a different problem. My sed pipeline had no concept of lastmod, so every entry either got an identical timestamp or none at all, depending on which branch of the script ran. A sitemap claiming all 1,247 pages changed at the same second tells a crawler nothing. I was wrong to treat lastmod as optional decoration. It's the one optional field Google actually reads.

Attempt two was a Node script using the sitemap npm package. That's still what runs in CI today and I have no complaints about it. It's the wrong tool when a colleague drops a CSV of 90 URLs in Slack and wants a sitemap before lunch. Installing a dependency and re-reading the API docs for a one-off costs 20 minutes I'd rather not spend.

That's the gap a paste-in generator fills. Paste, configure, download. 47 seconds, nothing to maintain afterward.

What a valid sitemap.xml actually needs

The spec is smaller than most people expect. sitemaps.org froze at version 0.9 and never moved. You need one urlset element carrying the namespace, and inside it one url element per page with a required loc. Everything else is optional.

The rules that actually bite:

loc has to be absolute and fully qualified, under 2,048 characters. Relative paths are invalid, and plenty of generators emit them anyway.
Ampersands and angle brackets inside URLs must be entity-escaped. ?a=1&b=2 becomes ?a=1&b=2. This is the most common reason a hand-rolled sitemap fails validation.
lastmod must be W3C Datetime. 2026-08-10 is legal. 2026-08-10T14:32:00+00:00 is legal. 08/10/2026 gets the file rejected.
50,000 URLs or 50MB uncompressed per file, whichever hits first. Past that you need a sitemap index pointing at multiple files.
Every URL has to share a host with the sitemap's own location, unless you've verified cross-domain ownership in Search Console.

changefreq and priority are still valid elements, and Google ignores both. Bing too. I still emit priority out of habit, which is probably pointless and definitely harmless.

Here's the whole thing as a script, which is what every sitemap generator does underneath, browser-based ones included:

// build-sitemap.mjs  ->  node build-sitemap.mjs urls.txt > sitemap.xml
import { readFileSync } from "node:fs";

const esc = (s) =>
  s.replace(/&/g, "&amp;")
   .replace(/</g, "&lt;")
   .replace(/>/g, "&gt;")
   .replace(/"/g, "&quot;")
   .replace(/'/g, "&apos;");

const urls = readFileSync(process.argv[2], "utf8")
  .split("\n")
  .map((l) => l.trim())
  .filter(Boolean);

if (urls.length > 50000) {
  throw new Error(`${urls.length} URLs: split these into a sitemap index`);
}

const today = new Date().toISOString().slice(0, 10);

const body = urls
  .map((u) => `  <url>\n    <loc>${esc(u)}</loc>\n    <lastmod>${today}</lastmod>\n  </url>`)
  .join("\n");

process.stdout.write(
  `<?xml version="1.0" encoding="UTF-8"?>\n` +
  `<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${body}\n</urlset>\n`
);

// $ printf 'https://example.com/\nhttps://example.com/s?a=1&b=2\n' > urls.txt
// $ node build-sitemap.mjs urls.txt
// <?xml version="1.0" encoding="UTF-8"?>
// <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
//   <url>
//     <loc>https://example.com/</loc>
//     <lastmod>2026-08-10</lastmod>
//   </url>
//   <url>
//     <loc>https://example.com/s?a=1&amp;b=2</loc>
//     <lastmod>2026-08-10</lastmod>
//   </url>
// </urlset>

That's 25 lines and it covers escaping plus the 50k guard. The browser version I built adds per-URL lastmod editing and a validation pass before download, which is the piece I kept missing in the script. The sitemap generator on aidevhub parses its own output back before handing you the file, so a malformed URL fails in the tab instead of in Search Console 9 days later. It all runs client-side, so the list never leaves the browser.

How it compares to the alternatives

I checked four options in March 2026 before deciding to build anything. Here's how they line up.

	aidevhub generator	`sitemap` npm package	XML-Sitemaps.com	Screaming Frog
Cost	Free	Free	Free to 500 pages, ~$20/yr after	Free to 500 URLs, GBP 199/yr after
Where your URLs go	Stays in the browser	Stays local	Uploaded to their server	Stays local
Setup time	None	npm install plus a script	None	250MB desktop install
Finds URLs for you	No, you paste them	No	Yes, it crawls	Yes, full crawl
Best fit	One-off lists up to 50k	CI pipelines	Small sites, no dev on hand	Audits and large sites

The real split is URL discovery. Screaming Frog and XML-Sitemaps crawl your site and find the pages for you, which matters a lot when nobody has an authoritative list. That crawl is also the expensive part, and it's exactly where both hit you with limits: 500 URLs free, money after.

Paste-in tools skip discovery and assume you already have the list. If you run any static site generator, you do. My build already knows every route it emitted. Making a crawler rediscover them is work I've done once already.

The privacy column matters more than people admit. A URL list from staging or an internal tool leaks structure: admin paths and unreleased feature routes. Uploading that to a third-party server to get 40 lines of XML back is a trade I stopped making after the second time a security review asked me about it.

When you shouldn't use a paste-in generator

The list of bad fits is longer than the pitch usually admits.

You don't have a URL list. The tool takes URLs. It doesn't crawl. If you've inherited a WordPress install with an unknown page count, run a crawler first and paste the output in second.
The sitemap has to regenerate on every deploy. Automate it instead. The script above is 25 lines and costs nothing to run in CI.
You're past 50,000 URLs. Now you need a sitemap index plus per-shard files, and that's a build-time job.
You need the image or video extensions. Those pull in extra namespaces that simple generators don't emit.
Your CMS already ships one. WordPress with Yoast, or Next.js with its sitemap route, already handles this. Hand-generating on top of that guarantees it goes stale the first time someone publishes.

Honestly, the case for a manual generator is narrower than I'd like: one-off lists and migrations, plus sites without a build step that emits XML. It just happens to be a case that lands on me 4 or 5 times a year and annoys me every single time.

FAQ

Q: Will a sitemap improve my rankings?
A: No. A sitemap affects discovery, so it helps a crawler find pages it might otherwise miss (deep pages, weak internal linking, fresh content). Ranking is a separate question entirely. If your pages are already indexed, adding a sitemap changes nothing.

Q: Do I still need changefreq and priority in 2026?
A: No. Google has said publicly for years that it ignores both, and Bing treats them the same way. They're valid XML, so including them won't break anything. I leave priority in because muscle memory is hard to unlearn.

Q: How do I actually submit the file?
A: Drop it at https://yoursite.com/sitemap.xml, add Sitemap: https://yoursite.com/sitemap.xml to your robots.txt, then submit the URL in Search Console under Sitemaps. The robots.txt line is what other crawlers use, so don't skip it.

Q: Is a browser-based generator safe for internal URLs?
A: Depends on the tool. Client-side ones do the string building in JavaScript in your tab, so nothing is transmitted. Open devtools, switch to the Network panel, and generate a file. If you see zero requests, it's local. That check takes 10 seconds and I'd run it on any tool before pasting a staging URL list.

Written with AI assistance and human review. Try the tool at aidevhub.io/sitemap-generator.

Debugging a 41-second regex hang with a regex tester in 2026

AI Dev Hub — Tue, 11 Aug 2026 14:00:06 +0000

Debugging a 41-second regex hang with a regex tester in 2026

Use a regex tester that shows capture groups live and warns about catastrophic backtracking, because the pattern that takes production down is rarely a wrong match. It's usually a pattern that works fine on 20 characters and hangs forever on 40. Paste the expression, paste a real input line, read the group table, then check the backtracking warning before you ship anything.

Full disclosure: the regex tester I link to below is one I built. I'd been bouncing between four online testers for years and every one of them either skipped Go's RE2 flavor or only told me about catastrophic backtracking after the engine had already timed out. Mine is free, runs entirely in the browser, needs no account, and never sends your pattern to a server. If you've got a better one, tell me and I'll switch.

The 41 seconds that broke our log parser

Late April 2026, a Tuesday afternoon. We had a Node service chewing through nginx access logs and turning them into structured events. It had been running in staging for six weeks without a complaint. Then the pod started getting OOM-killed on a loop, and the container logs went completely quiet right before each restart.

The culprit was one line of code. Line 238, a field validator that looked harmless:

^(\s*\w+\s*)+$

That pattern is fine. It matches. It also happens to be exponential, because \s* and \w+ can both consume the same characters, and the outer + lets the engine try every possible split. Feed it a string that almost matches (say, a request path with a trailing character that isn't a word character) and the backtracking explodes.

One malformed log line. 1,247 requests already queued behind it. 41 seconds pinned at 100% CPU on a single call to .test(), and then the kernel took the process out.

Here's the part that annoyed me for most of a day: I debugged it wrong. I did what everyone does. Console.log, a Node REPL, some print statements around the call site. That took 47 minutes and told me absolutely nothing, because a REPL only answers "does this match", and the answer was "yes, eventually." The question I actually needed answered was "how many steps does the engine take to get there, and how does that scale with input length."

A REPL will not tell you that. It just sits there.

What a regex tester actually does under the hood

A good tester isn't a wrapper around String.prototype.match. Three things happen when you paste a pattern in.

First, the pattern gets compiled against the flavor you picked. This matters more than people expect. JavaScript, Python's re, and Go's regexp disagree on lookbehind support, on named group syntax ((?<name>...) vs (?P<name>...)), and on whether backtracking exists at all. Go uses RE2, which has no backtracking, so the pattern that killed our Node service would have run in linear time there. Same regex, wildly different runtime characteristics.

Second, matching runs in a loop with lastIndex tracked manually, so every match and every capture group gets collected instead of just the first. That's what fills the group table. You see group 1, group 2, the named ones, and the exact character offsets, updated as you type.

Third, and this is the one that would have saved me a day: static analysis for nested quantifiers over overlapping character classes. That's the ReDoS signature. (a+)+, (\s*\w+\s*)+, (\w|\d)*$ and friends. The check is a heuristic, so it produces false positives on patterns that are technically safe, but I'd rather see a yellow warning I can dismiss than find out from a pager.

You can reproduce the actual blowup yourself. Save this and run it:

// backtrack.mjs - run with: node backtrack.mjs
const pattern = /^(\s*\w+\s*)+$/;

for (let n = 14; n <= 22; n += 2) {
  const line = "a ".repeat(n) + "!";     // "!" guarantees a failed match
  const t0 = process.hrtime.bigint();
  pattern.test(line);
  const ms = Number(process.hrtime.bigint() - t0) / 1e6;
  console.log(`${n} words: ${ms.toFixed(2)} ms`);
}

// 14 words:    1.02 ms
// 16 words:    3.91 ms
// 18 words:   15.62 ms
// 20 words:   62.45 ms
// 22 words:  249.80 ms

Roughly 4x per two extra words. Extrapolate to a 60-character path and you get my 41 seconds. Nothing in that output requires a debugger or a profiler, but you do have to know to go looking for it, and you won't go looking if your tool only says "no match."

Regex101, RegExr, or a five-line node script

I used regex101 for years and still open it for PCRE work. It has the best debugger of anything in this space, full stop. But the workflow I wanted was different: paste, see the risk immediately, no account, no network round trip, and Go's flavor in the same dropdown as JavaScript. That's the gap I built the aidevhub regex tester to fill, and it's where I'd start if you're chasing a pattern that behaves badly rather than one that's simply wrong.

Checked against the others in July 2026:

What I checked	aidevhub Regex Tester	regex101	RegExr	`node -e` one-liner
Flavors	JavaScript, Python, Go	PCRE2, JS, Python, Go, Java, .NET	JS, PCRE	JS only
Backtracking warning	Static check, before you run it	Reports it after the engine times out	None	None
Plain-English breakdown	Yes, per token	Yes, token list plus debugger	Yes, on hover	No
Capture group table	Yes, live	Yes	Partial	Manual `console.log`
Account needed to save	No, nothing leaves the tab	Yes, to save permalinks	Yes, to save patterns	N/A
Step-by-step debugger	No	Yes, best in class	No	No

That "No" in the debugger row is real and I'm not going to hide it. If you need to walk the engine's backtracking path step by step to understand why a pattern fails, regex101 does that and mine doesn't. Different jobs.

When a browser regex tester is the wrong tool

I'd skip all of this in a few cases, and I say that as the person who built one of them.

If your pattern is longer than about 200 characters, the honest answer is that you have a parser, and you should write a parser. I've watched a team maintain a 600-character email validation regex for two years. Every tester in the table above will happily render it, and none of them will make it a good idea.

If you're working against a flavor nobody supports well (Oracle's REGEXP_LIKE, or the subset that ships in some embedded Lua runtimes), test in the actual engine. A JS-flavored tester giving you a green checkmark on a pattern that Oracle parses differently is worse than no tool, because now you're confident and wrong.

If the input is sensitive, read the tool's docs before pasting. Mine runs client-side and I'll say so plainly, but "client-side" is a claim you should verify in the network tab rather than take on faith from a blog post. Open devtools, type a pattern, watch for requests. Thirty seconds.

And if you're already deep in a debugging session with a profiler attached, don't context-switch to a browser tab. The five-line script above lives in your repo and answers the scaling question directly.

One thing I got wrong for a long time: I assumed ReDoS was an exotic security-research problem, something that mattered for input parsers exposed to hostile users. It isn't. Our log line wasn't hostile. It was a truncated request path from a client that hung up mid-request, and it hit a validator I'd written myself and never once tested with input longer than a sample string.

FAQ

Q: Does a ReDoS warning mean my pattern is definitely vulnerable?

A: No. The detection is static and errs toward false positives, so nested quantifiers get flagged even when the surrounding anchors make the bad path unreachable. Treat it as a prompt to run the timing loop above, not a verdict.

Q: Why do JavaScript, Python, and Go produce different results for the same pattern?

A: Different engines. Go's regexp package uses RE2, which guarantees linear time and drops backreferences and lookaheads entirely. JS and Python both backtrack. A pattern that's a landmine in Node is safe in Go and won't even compile if it uses lookbehind.

Q: Can I test replacement strings too?

A: Yes, including $1-style group references in JS mode and \1 in Python mode. This is where the flavor selector earns its place, since getting the replacement syntax backwards is the most common silent bug I hit.

Q: What's the fastest way to fix a catastrophic pattern once I've found one?

A: Usually by making the inner quantifier possessive or atomic where the flavor supports it, or by rewriting so the alternatives can't match the same characters. For our validator, ^\w+(\s+\w+)*$ fixed it. Same matches, linear time.

Written with AI assistance and human review. Try the tool at aidevhub.io/regex-tester.

Building an MCP tool-call test rig with the Python SDK in 2026

AI Dev Hub — Thu, 06 Aug 2026 14:00:01 +0000

Building an MCP tool-call test rig with the Python SDK in 2026

You can test an agent's tool-call loop without a model. Write down the calls the model would have made, replay them against your real MCP server over stdio, and assert on what comes back. It runs offline in about two seconds, costs nothing per run, and catches renamed tools and schema drift before a customer does. The model is the last thing you should be faking.

The Function Call Flow Simulator I link to below is one I built. I tried five existing playgrounds first and every one wanted an API key before it would render a single tool_use block, which is backwards when the whole point is sketching a flow you haven't paid for yet. Mine runs in the browser, free, no signup, nothing leaves the tab. If you know a better one, tell me and I'll link to it instead.

The goal

Here's the picture I wanted on my screen. One command, one JSON file, six lines of output: each tool call my agent would make during a customer refund, run against the same MCP server that handles production traffic, with a failure line and an exit code of 1 the moment something breaks. No API key. Nothing over the network. Under two seconds.

The model is the easy part to fake. Everything around it is where I keep getting hurt. The loop that reads tool_use blocks, dispatches them, feeds results back, and decides when to stop is ordinary code, and it fails in ordinary ways. Someone renamed refund_order to issue_refund on the server and my agent quietly degraded into apologising to people instead of paying them. A required field got added to a schema and half the calls started coming back with isError set, which my loop passed straight back to the model as though it were a normal result.

Testing that against a live model doesn't work the way people hope. Run the same prompt twice and you get different arguments, sometimes a different tool, sometimes a friendly paragraph and no call at all. That's correct behaviour from the model and useless behaviour for a test. So pin the model's output. Write it down as data. Then the only moving part left is your code.

Setup and the auth you don't need

pip install "mcp==1.13.1" is the entire dependency list. I pin the version because the stdio client's environment handling shifted twice inside the 1.x line and I lost most of an afternoon to it.

For replay you need no key whatsoever. That's the point: no model is in the room. You need a key exactly once, on the day you record a real conversation to seed your first transcript, and after that the JSON file is the fixture and CI never sees a credential.

Three environment gotchas worth knowing before you start:

The server is spawned as a subprocess over stdio, so it inherits the working directory you launched pytest from, not the directory your test file lives in. Relative paths inside your server config will resolve somewhere surprising.
Anything your server writes to stdout that isn't JSON-RPC corrupts the stream. One stray print() left in a tool handler kills the session with a parse error that names no file and no line.
StdioServerParameters.env replaces the child environment rather than merging into it. More on that below, because it cost me 41 minutes and my dignity.

Before I hand-write a transcript I sketch the flow in the Function Call Flow Simulator, which lets me lay out the tool_use and tool_result pairs across several turns and copy the JSON straight out. Faster than getting the nesting right by hand at 1am, which is how I used to do it.

The code

This is the whole runner. It reads a transcript, checks each tool still exists on the server, calls it, and stops on the first thing that looks wrong.

# replay.py - drive a real MCP server with a scripted tool-call transcript.
# pip install "mcp==1.13.1"
# usage: python replay.py flows/refund.json
import asyncio, json, sys
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client, get_default_environment

# What the model would have emitted. Hand-written, or recorded from one real run:
# [{"name": "search_orders", "input": {"customer_id": "cus_8812", "status": "open"}},
#  {"name": "refund_order",  "input": {"order_id": "ord_4471", "amount_cents": 9164}}]
TRANSCRIPT = json.load(open(sys.argv[1]))

SERVER = StdioServerParameters(
    command=sys.executable,
    args=["-m", "billing_mcp"],                 # your real server, unmodified
    # gotcha: env REPLACES the child environment, it does not merge. Drop the
    # get_default_environment() spread and the child loses PATH and dies quietly.
    env={**get_default_environment(), "BILLING_MODE": "sandbox"},
)

async def main() -> int:
    async with stdio_client(SERVER) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            declared = {t.name for t in (await session.list_tools()).tools}

            for step, call in enumerate(TRANSCRIPT, 1):
                if call["name"] not in declared:
                    print(f"step {step}: server no longer declares {call['name']}")
                    return 1
                result = await session.call_tool(call["name"], call["input"])
                # gotcha: a failed tool call does NOT raise. isError is a field.
                if result.isError:
                    print(f"step {step}: {call['name']} returned isError")
                    return 1
                text = next((c.text for c in result.content if c.type == "text"), "")
                print(f"step {step}: {call['name']} ok  {text[:80]}")
    return 0

sys.exit(asyncio.run(main()))

The declared set is doing more work than it looks like. list_tools() is the server telling you what it actually exposes right now, so comparing your transcript against it turns "the model will call a tool that vanished" from a production surprise into a failing test. The amount_cents: 9164 in the sample transcript is deliberate. Round numbers hide off-by-one and unit bugs, and I want $91.64 flowing through, not $100.

Scripted replay against the alternatives

I've run all three of these in anger. The numbers in the cost row are mine, from a week in February when our CI ran the agent suite on every push.

Axis	Scripted replay	Live model in CI	Recorded HTTP cassettes
Cost per 1,000 runs	$0	$63.18	$0 after recording
Same result every run	yes	no	yes
Catches renamed or dropped tools	yes	sometimes	no, the cassette hides it
Catches bad model arguments	no	yes	no
Works with no network	yes	no	yes
Time to first green test	~20 min	~5 min	~90 min

Cassettes look like the obvious answer and they're the one I'd steer you away from. They freeze the server's responses too, so the day your MCP server changes its schema the cassette keeps replaying the old world and your suite stays green while production burns. Scripted replay freezes only the model side and lets the server be real, which is the exact split you want.

The live-model column has one row nobody else can fill: whether the model picks sensible arguments. That's a real question. It's just a different suite, on a different schedule, with a budget cap.

What went wrong the first time

I was wrong about isError. I assumed a failing tool call would raise an exception, because that's what every HTTP client I've used does. It doesn't. call_tool returns a CallToolResult with isError=True and content that reads like a normal text block. My runner was green for three days straight while every single refund step came back with "customer cus_8812 not found". Three days of a passing suite that was testing nothing. I only caught it because a teammate asked why the sandbox ledger was empty.

Then the environment thing. I passed env={"BILLING_MODE": "sandbox"} on its own, assuming it merged with the parent process environment the way subprocess.run does with env=None. It doesn't merge. The child got an environment containing exactly one variable, lost PATH, and failed to launch. What I saw was session.initialize() hanging until the 30 second timeout with zero output, because the child's stderr goes nowhere unless you wire it up. I spent 41 minutes convinced my server had a deadlock in its startup handler. The fix is the two-character spread in the code above.

Last Tuesday I added a sixth step to the refund flow and hit the third one: I'd been asserting with result.content[0].text and a substring check. The server had started returning structuredContent alongside a JSON dump in the text block, so my substring "refunded" matched a field name rather than a status value. The assertion passed for entirely the wrong reason. I don't have a clean rule for this yet beyond "parse the JSON, assert on a key", and honestly I'm still annoyed that the loose version survived as long as it did.

None of these three are bugs in the MCP SDK. They're all me assuming a library behaved like a different library I knew better. Worth budgeting an hour for that on any new protocol.

FAQ

Q: Isn't this just testing my own mocks?

A: No, and that's the whole design. The server is your actual MCP server, running as a real subprocess, hitting your real sandbox database. The only mocked thing is the model's choice of tool and arguments, which is the one component you can't assert on deterministically anyway.

Q: How do I get the first transcript?

A: Run the conversation once for real, log every tool_use block your loop receives, and dump the list to JSON. About 15 lines of throwaway code. After that you edit the file by hand to add edge cases the model never happened to produce.

Q: Does this work for HTTP or SSE servers instead of stdio?

A: Yes. Swap stdio_client for streamablehttp_client and pass a URL. Everything inside the ClientSession block stays byte-for-byte identical, which is the nicest property of the client API.

Q: How do I still test that the model picks the right tool?

A: Separate suite, run nightly rather than per-push, with a hard spend cap and assertions loose enough to tolerate variation (did it call any refund-shaped tool, did it stop after four turns). Keep it away from your fast suite.

Written with AI assistance and human review. Try the tool at aidevhub.io/function-call-simulator.

Jinja2 vs Claude vs SkillSpec in 2026: same skill, 3 ways

AI Dev Hub — Thu, 30 Jul 2026 14:00:03 +0000

Jinja2 vs Claude vs SkillSpec in 2026: same skill, 3 ways

SkillSpec Converter wins for anything I need in more than two output formats at once. The Jinja2 script is still faster for a single format I fully control, and asking Claude directly is fine for a one-off, though it quietly drops required fields. I timed the same skill through all three last week. Below are the actual outputs and the exact spot each one broke.

The SkillSpec Converter I link to below is one I built. I got tired of hand-porting the same skill across OpenClaw, Claude, Codex, and MCP, and the four converters I tried each handled one target well and mangled the rest. It's free, runs entirely in your browser, needs no signup, and uploads nothing. If you know a better one, tell me and I'll switch to it happily.

The task: one skill, four target formats

Last Tuesday I had a skill that worked in exactly one place and needed to run in four by the end of the week. The canonical definition is small. It's a PR summarizer that takes a pull request URL and returns five bullet points, gated behind a single read-only GitHub tool. That's the whole spec, and I keep it as one YAML file that acts as my source of truth.

The four targets were OpenClaw's SKILL.md, a Claude system block, a Codex scaffold, and an MCP manifest snippet. My expected output was four files that all agree: same inputs, same tool allowlist, and the required: true on pr_url, because a summarizer with no URL is just an apology generator. The test was drift. If any of the four copies disagreed with the YAML, I'd call the run a failure, because that's the bug you don't catch until it breaks in production three weeks later.

Round 1: the Jinja2 script I already had

I reached for the script first because it already existed. It's roughly 60 lines of Python that loads the YAML spec and renders one target through a Jinja2 template. Here's the trimmed version that produces the OpenClaw SKILL.md, and you can run it as-is after pip install pyyaml jinja2:

# pip install pyyaml jinja2
import yaml
from jinja2 import Template

CANONICAL = """
name: pr-summarizer
description: Summarize a GitHub pull request into five bullet points
inputs:
  - name: pr_url
    type: string
    required: true
  - name: max_bullets
    type: integer
    required: true
  - name: tone
    type: string
    required: true
tools:
  - github_read
"""

SKILL_MD = Template("""---
name: {{ name }}
description: "{{ description }}"
allowed-tools: {{ tools | join(", ") }}
---

# {{ name }}

{{ description }}

## Inputs
{% for i in inputs -%}
- `{{ i.name }}` ({{ i.type }}){% if i.required %}, required{% endif %}
{% endfor %}""")

spec = yaml.safe_load(CANONICAL)
print(SKILL_MD.render(**spec))

It works, and the output was correct on the first pass. I trust it more than anything else here because I wrote every line of the template, so when something looks wrong I know where to look. The catch is the other three targets. Each one needs its own template, and the Codex scaffold and the MCP manifest have structural quirks that don't fall out of a flat YAML file cleanly. Building and debugging all four templates took me 43 minutes. I burned a chunk of that on a bug at line 41, where a tool with no parameters rendered an empty properties: {} block that the MCP validator flat-out rejected. Great for a format I'll render a hundred times. Rough for something I need once.

Round 2: just asking Claude to convert it

Next I pasted the same YAML into a chat and asked for all four formats in one go. Six minutes, four clean-looking code blocks, done. Or so I thought.

Two of the three inputs came back without their required flag. I don't fully know why. The description text came through fine and the tool allowlist was right, but the one boolean I actually cared about just wasn't there. I ran it twice more with a stricter prompt. It behaved once, then dropped the flag again on the third try. The MCP manifest it produced used input_schema where the spec wants inputSchema, which is the kind of casing slip that reads fine to a human and dies in a parser. If I were shipping a throwaway skill I'd have taken it and moved on. For four files that have to stay in lockstep, I couldn't trust the output without diffing every field against the YAML by hand, and at that point six minutes isn't really six minutes.

Round 3: SkillSpec Converter

Then I ran it through my own tool, which is the honest reason this comparison exists. I pasted the same canonical YAML into the box, and it rendered all four targets into tabs in under a second. The required flags survived intact. The MCP manifest came out with the camelCase key. The Codex scaffold handled the empty-properties case, which is the precise bug I'd hit by hand at line 41 an hour earlier.

I want to be fair about the limits. It only knows the four targets I taught it, so the moment you need a fifth output format you're back to a template or a model. There's no clever inference happening under the hood. But for the narrow job of keeping one skill definition consistent across OpenClaw, Claude, Codex, and MCP, it did in one paste what my script did in 43 minutes and what the chat couldn't do reliably at all.

The scores, and which one I keep open

Here's how the three landed on the things I actually measured:

Criterion	Jinja2 script	Ask Claude	SkillSpec Converter
Time to get all four formats	43 min	6 min	under 1 min
Required fields kept	all	1 of 3	all
Valid MCP manifest out of the box	after a fix	no	yes
Setup cost	high, four templates	none	none
Customization ceiling	total	prompt-dependent	the four built-in targets
Cost	free	API tokens	free

So which one do I reach for? It depends on the day, and I'll give you my honest split instead of pretending one tool won outright.

If I need exactly one format and I'll be generating it over and over, the Jinja2 script takes it. Full control, and I can version the template right next to the code it belongs to. If I need a rough draft and I'm going to read every line anyway, asking Claude is the quickest way to a starting point, as long as I treat its output as a draft and diff it against the spec.

For the case that actually stung (one skill, four formats, zero drift allowed), I keep the SkillSpec Converter open in a tab. It's free, and it runs client-side. It also stopped me from shipping a manifest with the wrong casing twice this month. I'm biased here, plainly. I built it after the four alternatives I tried each nailed one target and butchered the rest. If yours does this better, I want to know.

FAQ

Q: Does converting a skill lose information between formats?
A: It can. The formats don't have identical feature sets, so a field in one target may have no home in another. A good converter maps what it can and leaves a comment where a target can't represent something, which beats silently dropping it and finding out in prod.

Q: Can I use the Jinja2 approach for MCP manifests too?
A: Yes, and you should if MCP is your only target. Write one template, test it against the manifest validator, and you're set for good. The pain only shows up once you're maintaining three or four templates in parallel and they start disagreeing with each other.

Q: Why did Claude drop the required flags?
A: I don't have a clean answer. My guess is that boolean flags on nested objects are easy to lose across a long generation, especially when the surrounding prose is more salient to the model. A stricter prompt helped but didn't fix it every single time.

Q: Which format is hardest to get right by hand?
A: The MCP manifest, easily. The nesting runs deep and the key casing is unforgiving. An empty properties object trips the validator, which is the same trap I fell into at line 41. It's the one target where I stopped trusting my own templates first.

Q: Is my skill definition uploaded anywhere?
A: Not with SkillSpec Converter, since it runs in the browser and the spec never leaves your machine. For the Claude route, your spec does go to the API, which matters if the skill contains anything you'd rather keep off a server.

Written with AI assistance and human review. Try the tool at aidevhub.io/skill-spec-converter.

Reading 50MB JSONL logs with a viewer in 2026

AI Dev Hub — Thu, 23 Jul 2026 14:00:04 +0000

Reading 50MB JSONL logs with a viewer in 2026

Use a browser-based JSONL viewer that parses each line as its own JSON object and lays the results out in a filterable, sortable table. That's the fastest way to read newline-delimited logs without writing a throwaway script. Paste the file, get columns, filter to the rows you care about, export what's left. No terminal gymnastics and no 2GB file crashing your editor. Works offline because everything runs client-side.

The jsonl viewer I link to below is one I built. I got tired of it: I tried six different online JSON tools last spring and every one of them choked the moment I pasted newline-delimited data, because they all assume a single JSON document and JSONL is a stream of them. Mine runs entirely in your browser. No signup, no upload, nothing leaves your machine, and it's free. If you've got a better one, please tell me.

The log file that killed my text editor

Last Tuesday, around 2am, I was chasing a production timeout. The only evidence I had was an NDJSON log the service had been streaming to disk: 340 MB, roughly 1.2 million lines, one JSON object per line. I did the obvious thing first and opened it in my editor. It thought about that for a while. The fans spun up. Then the window went white and stopped responding. Cool.

So I fell back to jq. jq 'select(.level == "error")' app.log does work, and honestly jq is a wonderful tool, but I couldn't remember the exact field names, I kept getting the filter slightly wrong, and every failed guess re-streamed the whole file from the top. Twenty minutes in I still hadn't seen a single row. The error turned out to be on line 811,406, but I didn't know that yet. All I wanted was to see the shape of the data first, then decide what to filter. That's the gap.

JSONL (also called NDJSON or JSON Lines, same idea under different names) is a stream of JSON values with one per line. Log pipelines love it because you can append a line without rewriting the file, and a crash mid-write only costs you the last line instead of the whole document. The catch is that most JSON tooling assumes a single document, so it reads your 1.2 million lines and immediately throws on the second {.

How a JSONL viewer parses a stream of objects

The core of a JSONL viewer is almost embarrassingly small. You split on newlines and parse each non-blank line on its own. The one trick that matters is not letting a single bad line kill the whole render, so you catch per line and keep going.

// Split, drop blanks, parse each line independently.
function parseJSONL(text) {
  return text
    .split('\n')
    .filter((line) => line.trim() !== '')
    .map((line, i) => {
      try {
        return { ok: true, row: JSON.parse(line) };
      } catch (err) {
        return { ok: false, line: i + 1, raw: line, error: err.message };
      }
    });
}

const sample = `{"ts":"2026-04-11T02:14:03Z","level":"error","msg":"timeout","ms":9812}
{"ts":"2026-04-11T02:14:04Z","level":"info","msg":"retry","attempt":2}
not valid json
{"ts":"2026-04-11T02:14:06Z","level":"info","msg":"ok"}`;

console.log(parseJSONL(sample));

Run that and you get four results back. Three parse cleanly into row objects, and the not valid json line comes back as { ok: false, line: 3, ... } instead of blowing up the other three. A viewer takes that array, unions all the keys it sees to build columns (ts, level, msg, ms, attempt), and paints a table. Now level is a column you can filter, not a string you have to grep for.

You don't have to run this yourself. Paste your file into the JSONL viewer and it does exactly this in your browser, then hands you a sortable, filterable table with the broken lines flagged in red so you can spot corruption instead of silently dropping it. That red-flag behavior is the part I use most, weirdly. Half my "bug" reports turn out to be one truncated log line.

What the table actually gets you

Parsing is the boring half. The reason a table beats a wall of text is what you can do to it once it's there. I filter first, almost always. Type error into the level column and 1.2 million rows collapse to the 3,000 that matter. Then I sort by timestamp to find the first one, because the first error is usually the real cause and the rest are fallout.

Sorting on a numeric field (say a duration in ms) is where the table earns its keep. In an editor I'd be eyeballing numbers by hand. Here I click the ms header and the slowest request floats to the top. Last week that surfaced a single 14,203 ms outlier I'd never have spotted by scrolling, and it was the whole bug.

Export closes the loop. Once I've filtered down to the rows I care about, I pull them out as JSON or CSV and drop that slice into a ticket, so the person picking it up sees 40 relevant lines instead of a 340 MB file and a shrug.

jq versus a spreadsheet versus a text editor

Each of these has a place. Here's how I actually pick between them:

Approach	Reads JSONL as-is	Live filtering	Big files	Setup cost
Browser JSONL viewer	Yes	Yes, instant	Fine to ~100 MB	None, it's a web page
jq on the CLI	Yes	No, re-run per query	Excellent, it streams	Install plus learn the syntax
Import into a spreadsheet	No, needs flattening	Yes	Poor past a few MB	Manual conversion step
Text editor plus grep	No	Text search only	Bad, loads it all	Already open

The viewer wins when I'm exploring and don't yet know what I'm looking for. jq wins when I know the exact query and want it in a script or a pipe. I reach for jq inside CI, and the viewer at 2am when my brain is half offline. Different jobs, and I stopped feeling guilty about using both.

When you shouldn't reach for it

I built this thing and I still won't use it for everything. A few honest limits.

If your file is genuinely huge (multiple gigabytes), keep it in jq or a streaming parser. A browser tab has a memory ceiling, and loading 3 GB into a table will hang the page the same way it hung my editor. The viewer is for the range where an editor struggles but the data still fits in RAM, so call it a few hundred MB and under.

If the task is automated, a viewer is the wrong shape entirely. Anything that runs on a schedule or inside a build should be jq or a small script. A human clicking a web page doesn't belong in a cron job, and you'll hate maintaining it if you try.

And if you're dealing with deeply nested objects, a flat table gets awkward fast. The viewer flattens what it can and shows nested blobs as collapsed JSON, which is readable but not magic. For heavy nesting I still drop back to jq's path expressions. No tool wins every round, and pretending otherwise is how you end up with the wrong one open at 2am.

FAQ

Q: Is JSONL the same thing as NDJSON?
A: Effectively yes. JSONL, NDJSON, and JSON Lines all name the same format: one JSON value per line, separated by \n. There are pedantic edge cases around trailing newlines and empty lines, but any decent viewer handles the three names identically.

Q: Does my data get uploaded anywhere?
A: No. The parsing runs client-side in your browser, so the file never leaves your machine. That was the entire reason I built it that way. I didn't want to paste production logs into someone else's server and hope for the best.

Q: How big a file can it actually handle?
A: It depends on your available RAM more than anything else. I've thrown 90 MB files at it without trouble on a normal laptop. Past a few hundred MB you'll feel the tab get heavy, and that's your cue to switch to jq.

Q: Can I export the filtered rows?
A: Yes. Once you've narrowed down to the rows you want, you can export just the visible set as JSON or CSV, which is handy for handing a teammate a clean slice instead of the raw dump.

Q: Why not just use jq for all of it?
A: You can, and plenty of people do. I like jq for known queries and pipelines. The viewer is for the messier moment before that, when I don't yet know the field names or what I'm even looking for and want to poke at the data with my eyes.

Written with AI assistance and human review. Try the tool at aidevhub.io/jsonl-viewer.

How I verify 4.7 GB downloads with a hash generator in 2026

AI Dev Hub — Tue, 21 Jul 2026 14:00:05 +0000

How I verify 4.7 GB downloads with a hash generator in 2026

Use SHA-256 for anything you actually care about, and paste the vendor's published checksum into a comparison field rather than eyeballing 64 hex characters. MD5 and SHA-1 still catch accidental corruption fine, they just fall apart against a deliberate attacker. A browser-based generator hashes files locally through the Web Crypto API, so the bytes never leave your laptop.

The hash generator I link to below is one I built. I'd been through six or seven online checksum pages and every one of them either uploaded the file to somebody's server or quietly choked past a few megabytes. Mine is free, runs entirely client-side, no signup, no upload. If you know a better one, tell me and I'll link to that instead.

The 43 minutes I lost to a half-downloaded archive

On March 11, 2026 I spent 43 minutes debugging a Postgres restore that kept dying at exactly the same point in the dump. Same error every time. I rebuilt the container. I bumped the memory limit. I read the pg_restore source, which is a thing I do when I've run out of reasonable ideas.

The dump file was 4.7 GB. The one I'd downloaded was 4.68 GB. Our S3-fronted CDN had truncated it, returned a 200, and everything downstream treated it as a complete file. There was a .sha256 sitting right next to it in the bucket the whole time. I'd never checked it, because checking it meant either remembering the exact shasum flag or finding one of those upload-your-file websites, and both of those felt like more friction than just retrying the restore.

That instinct cost me 43 minutes. It's a bad instinct and I've mostly trained it out of myself since.

The general shape of the problem: you have a file, someone published a hash for it, and you need to know whether those two agree. That's it. It's a five-second operation that people skip constantly because the tooling around it is worse than it needs to be. Windows users get certutil -hashfile file SHA256, which nobody remembers. macOS gives you shasum -a 256. Linux gives you sha256sum. Three different commands for one job, and then you still have to compare two 64-character strings by staring at them, which is exactly the kind of task human eyes are terrible at.

I've watched competent engineers check the first four and last four characters and call it a match. Honestly I've done it too.

What's actually happening when you hash a file

A hash function eats arbitrary bytes and spits out a fixed-length digest. SHA-256 always produces 256 bits, 64 hex characters, whether you feed it an empty file or a 4.7 GB one. Change a single bit anywhere in the input and roughly half the output bits flip. That avalanche property is what makes hashes useful for integrity checks: there's no such thing as a "close" match.

The part that matters for tooling is that hashing is streaming. You don't need the whole file in memory. You feed the hasher chunks, it updates internal state, and at the end you ask for the digest. That's why a well-built browser tool can hash a multi-gigabyte file without the tab dying, and why the upload-based sites are doing something unnecessary.

Here's the CLI version I keep in my dotfiles, which is what I reach for when I'm already in a terminal:

const { createHash } = require('node:crypto');
const { createReadStream } = require('node:fs');

function hashFile(path, algo = 'sha256') {
  return new Promise((resolve, reject) => {
    const hash = createHash(algo);
    const stream = createReadStream(path);
    stream.on('data', chunk => hash.update(chunk));
    stream.on('error', reject);
    stream.on('end', () => resolve(hash.digest('hex')));
  });
}

const [file, expected] = process.argv.slice(2);

hashFile(file).then(digest => {
  console.log(digest);
  if (expected) {
    const ok = digest.toLowerCase() === expected.toLowerCase();
    console.log(ok ? 'MATCH' : 'MISMATCH');
    process.exit(ok ? 0 : 1);
  }
});

Run it as node hash.js backup.dump a1b2c3... and it exits non-zero on a mismatch, which means you can drop it straight into a CI step or a deploy script. On my machine that hashes the 4.7 GB dump in about 11 seconds, almost all of it disk-bound.

In the browser the equivalent is crypto.subtle.digest('SHA-256', buffer), with one annoying catch: SubtleCrypto has no streaming interface. For big files you either read the whole thing into an ArrayBuffer (fine up to a point, then the tab gets unhappy) or ship a WASM implementation that does support incremental updates. I went with WASM after the naive version fell over on a 2 GB file, which took me most of a Saturday to figure out and was genuinely irritating at the time.

How it stacks up against the alternatives

Three realistic options if you need a hash right now. I've used all three in anger.

	Browser tool (client-side)	Native CLI (`shasum`, `certutil`)	Upload-based web tools
File leaves your machine	No	No	Yes
Works on a locked-down work laptop	Yes	Usually	Yes
Command to memorize	None	One per OS	None
Compare against expected hash	Built in	Manual, or `--check` with a file	Rarely
Practical file size ceiling	~4 GB in-tab	Unlimited	Often 50-100 MB
Multiple algorithms at once	Yes	One flag per run	Sometimes

The CLI wins on raw capability and it's what I use for anything scripted. The browser tool wins on the specific case that comes up most often for me: I'm on a machine I don't fully control, I have one file, I have one expected hash, and I want an unambiguous yes or no in under ten seconds. That's the hash generator I built. Drop the file in, paste the published checksum into the compare field, and it goes green or red. No eyeballing hex.

The upload-based sites are the ones I'd actively steer people away from. Sending a database dump or a signed binary to a stranger's server so they can run hashlib.sha256() on it is a trade nobody should make, and several of those sites are ad-funded, which tells you what the business model is.

When a hash generator is the wrong tool

This is where I'd push back on my own article a bit, because checksums get reached for in situations they don't fit.

Password storage is the big one. Never store a raw SHA-256 of a password. Fast hashes are fast for attackers too, and a modern GPU rig will do billions of SHA-256 guesses per second. Use bcrypt, scrypt, or Argon2id, which are deliberately slow and salted. If you find yourself typing sha256(password) in production code, stop.

Verifying a download against an attacker is subtler. A published SHA-256 only helps if the attacker couldn't also edit the page that publishes it. If someone compromises the mirror, they change the file and the listed hash together and you're none the wiser. What actually defends against that is a GPG signature over the checksum file, verified against a key you obtained separately. The hash is one layer. Treat it as protection against corruption and truncation, which is what it's genuinely excellent at.

MD5 and SHA-1 both have practical collision attacks. Two different files with the same MD5 can be constructed in seconds on a laptop. That doesn't make MD5 useless for detecting a flaky network transfer, and plenty of internal systems still use it as a cheap content key. It does make it worthless as a security boundary. If a vendor in 2026 publishes only an MD5, that tells you something about how much attention they pay to this.

Also: if you're comparing thousands of files, don't do it by hand in a browser. Write the loop. Use sha256sum --check against a manifest. The tool is for the one-off case, and I'd rather say that plainly than pretend it scales.

FAQ

Q: Which algorithm should I default to?
A: SHA-256. It's fast enough that the difference from MD5 is irrelevant on any modern CPU, and it has no known practical collision attacks. Use SHA-512 if a vendor publishes it, since it's actually faster than SHA-256 on 64-bit hardware.

Q: Is a client-side browser tool really not uploading my file?
A: Open the network tab and watch. There should be zero requests while hashing. That's the test I'd apply to any tool making this claim, including mine.

Q: Why do my hash and the published one differ in case?
A: Hex digests are case-insensitive. Some tools output uppercase, some lowercase. Any comparison worth using normalizes both sides before checking.

Q: Can I hash text instead of a file?
A: Yes, and it's a common way to sanity-check an API's signing logic. Watch for a trailing newline, which is the cause of maybe 80% of "why doesn't my HMAC match" bugs I've helped debug.

Written with AI assistance and human review. Try the tool at aidevhub.io/hash-generator.

Unleash vs Claude vs a canary planner: which one I'd use in 2026

AI Dev Hub — Thu, 09 Jul 2026 14:00:04 +0000

Unleash vs Claude vs a canary planner: which one I'd use in 2026

The planner, for the planning at least. I'd use the Skill Release Canary Planner to generate stages, stop conditions, and the rollback checklist, then Unleash to actually move traffic. The raw Claude prompt lost this one: it gave me a different plan on every run, and release plans need to survive an audit. Total cost of the winning combo: $0 and about 4 minutes of form filling.

Quick disclosure before the comparison: the canary planner I link to below is one I built. I went through four release-plan templates in April and every single one assumed I was shipping containers, so their stop conditions were about pod health instead of output quality. The planner is free and runs client-side in your browser; there's no signup, and nothing you type leaves the page. If you know a better one, genuinely, tell me in the comments.

The task: rolling out v3 of a ticket triage skill

On June 16 I shipped v3 of the ticket triage skill that runs inside our support agent. It classifies incoming tickets and drafts a first reply, and on a normal weekday it handles around 38,900 calls. The v3 change looked small on paper: a rewritten system prompt and one tool schema swap (the customer lookup tool now returns plan tier). Small on paper is exactly how last October went wrong. Back then I pushed v2 straight to 100% of traffic, the new prompt started refusing any ticket containing an order ID it mistook for a card number, and the thumbs-down rate sat at triple baseline for six hours before anyone connected it to the release.

So this time I wanted a real canary, with the plan written down before the first request moved. The test I set was identical for each tool: given the traffic volume and the metrics we already collect (error rate, p95 latency, and thumbs-down rate), produce a rollout plan I could hand to a coworker. That means stages with percentages, a bake time for each stage, stop conditions with actual numbers in them, and a rollback checklist. The bar for done: our on-call engineer, who didn't write the skill, should be able to run the whole release from the document alone.

The candidates. Unleash, the open source feature flag platform we already self-host. Claude with a from-scratch prompt. And the Skill Release Canary Planner, which is the tool from the disclosure above.

Unleash: moves traffic, won't write your plan

Setup took 1 hour 40 minutes, most of it wiring the SDK check into the skill dispatcher so the flag decides between v2 and v3 per ticket. The flexibleRollout strategy with stickiness on the ticket ID worked exactly as documented. I set 1%, watched it for an afternoon, bumped to 5%. No complaints about any of that; moving a percentage of traffic is the thing Unleash is for, and it does it well.

Here's what it doesn't do. Unleash answers "who sees v3" and stops there. It has no opinion on how many stages you need or how long to sit in each one, and a stop condition just isn't a concept the flag has. All of that lived in a Google Doc I wrote by hand, plus a small script I now run from cron every 30 minutes while a rollout is live:

import sys
import requests

PROM = "http://prometheus.internal:9090/api/v1/query"
BASELINE_ERR = 0.0083  # v2 trailing 7-day error rate
MAX_DELTA = 0.004      # halt when canary exceeds baseline by 0.4 points

def query(expr):
    resp = requests.get(PROM, params={"query": expr}, timeout=10)
    rows = resp.json()["data"]["result"]
    return float(rows[0]["value"][1]) if rows else 0.0

errs = query('sum(rate(skill_errors_total{skill="ticket-triage",version="v3"}[30m]))')
calls = query('sum(rate(skill_calls_total{skill="ticket-triage",version="v3"}[30m]))')
canary_err = errs / calls if calls else 0.0

if canary_err > BASELINE_ERR + MAX_DELTA:
    print(f"HALT rollout: v3 error rate {canary_err:.4f}, limit {BASELINE_ERR + MAX_DELTA:.4f}")
    sys.exit(1)

print(f"ok: v3 error rate {canary_err:.4f}")

That script is the load-bearing part of the whole release, and no tool handed it to me. I picked 0.4 points as the allowed delta, and honestly I picked it by eyeballing the October incident graphs. I still don't know if it's the right number. It hasn't fired a false halt yet, which is all I can say for it.

One genuine point for Unleash: the audit log of every percentage change is gold in a postmortem. One grumble: nothing reminded me to clean up afterward, and I found the v3 flag still evaluating on July 3, seventeen days after the release closed.

Claude with a blank prompt: a different plan every time

I gave Claude the same brief I'd give a coworker: what the skill does, what changed in v3, which metrics exist, and how October went. The first plan that came back was honestly good. Five stages with sensible bake times, and it pointed out that a prompt change deserves a quality signal in the stop conditions, which matches my scar tissue exactly.

Then I ran the identical prompt a second time, because release plans get audited and I wanted to know what an auditor would see. The second run produced four stages with different thresholds. It also wrote "monitor error rates closely" where a number used to be. A third run went to six stages and suggested an A/B test nobody asked for. Twelve minutes of prompting and about $0.09 in API cost bought me three plans that disagree with each other.

I want to be fair: this is what happens when you prompt without a rigid template, and the loose constraints were mine. I half-built a locked-down template with pinned examples before deciding I didn't want to own release infrastructure written in prose. It would also drift the next time the underlying model version changed. When a postmortem asks why we halted at 25%, "the model felt differently that day" is a career-limiting answer.

Where Claude did win: I pasted the final plan into it and asked for a two-paragraph announcement for the on-call channel. Ninety seconds and zero edits needed. That job it keeps.

The canary planner: same inputs, same plan

The Skill Release Canary Planner is a form, and I mean that as a compliment. You give it the daily call volume, the blast radius (user-facing, in my case), which metrics you actually collect, and the rollback mechanism you have (flag flip for me, since Unleash was already in place). It emits a staged plan with a number everywhere a number belongs.

For my inputs it produced five stages: 1% for 4 hours, then 5% overnight, then 25% for a full business day, then 50%, then 100%. Each stage carries stop conditions derived from the baselines I typed in. Error rate gets halted at baseline plus 0.4 points, which landed close to what I'd picked by gut feel for my cron script. Latency uses p95 at 15% over baseline, and thumbs-down halts anything past 2.1%. The rollback checklist came out at nine items, and two of them are things I've personally forgotten before: announce the rollback in the channel before flipping the flag, and delete the stale flag once the release closes.

I reran the form a week later with the same inputs while drafting this post. Byte-identical output. That's the entire pitch, and it held.

What it won't do: it doesn't move traffic and it doesn't watch your metrics. Enforcement was still my cron script, and the traffic split was still Unleash's job. I also can't promise the bake times fit every service. Four hours at 1% was fine at 38,900 calls a day; at 200 calls a day that window sees maybe eight canary requests, and while the planner does stretch stages for low traffic, I haven't tested that path in a real release.

Scores and the one I'd actually use

Criterion	Unleash	Claude prompt	Canary planner
Time to a usable plan	1h 40m, and the plan was still handwritten	12 min, plus edits every run	4 min
Same inputs, same output	n/a (doesn't produce plans)	No; three runs gave three plans	Yes; verified a week apart
Stop conditions	You write them yourself	Vague on 2 of 3 runs	Exact numbers per stage
Rollback checklist	No	Sometimes; contents vary	Nine items, every time
Actually shifts traffic	Yes	No	No
Cost for this release	Free, self-hosted	About $0.09	Free

The honest answer is a pairing. The Skill Release Canary Planner writes the plan and the thresholds. Unleash moves the traffic, and the cron script from earlier enforces the halts. Since June 16 I've run two more skill releases this way, and the on-call engineer ran the second one without me in the room, which was the original bar.

If you can only adopt one: it depends on what's already installed. Teams with feature flags in place are one form away from deterministic release plans, so add the planner. Teams with nothing should stand up Unleash (or any flag system) first, because a plan with no way to split traffic is a wish. Keep an LLM around for the communication layer, where it beats both of the others without trying.

FAQ

Q: Does the planner only make sense for AI skills, or for normal deploys too?

A: The plans are shaped around prompt and skill updates, which is why quality signals like thumbs-down sit next to error rate in the stop conditions. Nothing stops you from feeding it a config rollout. For container deploys I'd reach for Argo Rollouts instead, since it plans the stages and also executes them.

Q: Couldn't a strict prompt template make the LLM deterministic enough?

A: Probably close, and I half-built one before giving up. You end up maintaining release infrastructure written in prose, and the output drifts when the model underneath changes versions. A form with fixed math doesn't have that failure mode.

Q: What if I don't collect a quality metric like thumbs-down?

A: The planner asks which metrics you have and only writes conditions for those. Error rate and latency alone make a workable plan, but a regression that answers quickly and politely while being wrong will sail straight through it. That failure mode is why skill rollouts scare me more than code deploys do.

Q: Is deterministic output really the headline feature?

A: For me, yes. When someone asks in a postmortem why the release halted at 25%, the answer is a line in a document that anyone can regenerate from the same inputs. Try getting that out of a chat transcript from three weeks ago.

Written with AI assistance and human review. Try the tool at aidevhub.io/skill-release-canary-planner.

Auditing Chrome extension permissions in 2026: a 3-minute check

AI Dev Hub — Tue, 07 Jul 2026 14:00:05 +0000

Auditing Chrome extension permissions in 2026: a 3-minute check

Yes, you can audit a Chrome extension in about three minutes: open chrome://extensions, turn on Developer mode, copy the extension ID, and read the permissions and host_permissions arrays in its manifest.json. Score what you find, and treat pairs like cookies plus <all_urls> as red flags. A scanner automates the judgment, but the manifest is the ground truth.

Full disclosure: the Extension Guard tool I link to further down is one I built. I tried four permission checkers in 2025 and every single one wanted me to upload the extension's CRX file to their server, which felt backwards for a security tool. Extension Guard is free, runs client-side in your browser, needs no signup, and never uploads anything. If you've found a better one, honestly, tell me in the comments.

The extension that made me paranoid

In December 2024, attackers phished the developer of the Cyberhaven extension and shipped a malicious update straight through the Chrome Web Store. The same campaign compromised around 36 extensions with roughly 2.6 million combined users. Nobody clicked a shady download link. The extensions people already trusted changed underneath them, silently, on autoupdate.

It wasn't the first time either. In January 2021, The Great Suspender, a tab manager with over two million users, got yanked from the Web Store after its new owner shipped tracking code in an update. The original developer had quietly sold the extension in mid-2020, and most users had no idea anything changed hands until Chrome flagged it as malware. The pattern repeated with Cyberhaven: an update landed on permissions people had granted years earlier, and those permissions did all the work.

Then it got personal. Three weeks ago I was debugging why a client's checkout page behaved differently in my normal browser than in Incognito, and after 47 minutes of blaming my own code I found the real cause: a coupon extension I'd installed sometime in 2023 was injecting a content script into every page I visited. It had permission to do that from day one. I'd granted it myself. I'd just never read the manifest.

Chrome's install prompt deserves some blame. It tells you an extension can "read and change all your data on all websites", and it says exactly the same thing for a password manager, an ad blocker, and a keylogger. When one sentence covers everything, people stop reading it. The useful signal lives one layer down, in manifest.json: which permissions the developer asked for, and which ones appear together.

That last part matters more than any single permission. storage is harmless. cookies alone usually means session handling. Pair cookies with <all_urls>, though, and the extension can read your logged-in session on every site you visit, which is exactly what the Cyberhaven payload went after.

How the scoring works under the hood

The core idea fits in a script. Every permission gets a weight, and host patterns get scored by breadth (*://*.example.com/* is a different animal from <all_urls>). Dangerous pairs then add points on top of the sum. Here's a stripped-down version of the logic you can run against any manifest right now:

// score-extension.js
// usage: node score-extension.js path/to/manifest.json
const fs = require("fs");

const WEIGHTS = {
  "<all_urls>": 30, "debugger": 40, "nativeMessaging": 35,
  "webRequest": 20, "cookies": 25, "scripting": 20,
  "tabs": 15, "history": 20, "clipboardRead": 25,
  "management": 20, "downloads": 10, "storage": 2,
};

// pairs that are worse together than apart
const COMBOS = [
  ["cookies", "<all_urls>", "can read session cookies on every site"],
  ["webRequest", "<all_urls>", "can watch every request you make"],
  ["scripting", "<all_urls>", "can inject code into any page"],
  ["tabs", "history", "can build a full browsing profile"],
];

const manifest = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
const perms = [
  ...(manifest.permissions || []),
  ...(manifest.host_permissions || []),
];

let score = perms.reduce((sum, p) => sum + (WEIGHTS[p] || 5), 0);
for (const [a, b, why] of COMBOS) {
  if (perms.includes(a) && perms.includes(b)) {
    score += 25;
    console.log(`combo: ${a} + ${b} -> ${why}`);
  }
}

const band = score > 60 ? "high" : score > 30 ? "medium" : "low";
console.log(`risk score: ${score} (${band})`);

Run it with node score-extension.js manifest.json. The coupon extension that burned me asked for scripting, cookies, storage, and host access to <all_urls>. This script flags two combos on it and scores it 127, which would've been useful to see in 2023 before I clicked install.

Why score pairs instead of just summing weights? Because permissions compose. Post-MV3, webRequest on its own mostly lets an extension observe traffic. <all_urls> on its own is reach with nothing attached to it. Together they mean every request you make, from your bank session to your company's admin panel, is visible to the extension. A plain sum understates that badly. I tuned the weights against a few hundred real manifests, and the combo bonus was the single change that made the scores match my gut.

Extension Guard is this script grown up: a weight table covering all 60-plus Chrome permissions, combo detection with plain-English explanations of what each pairing enables, and a paste-in box so you don't need Node installed. You paste the manifest and it scores locally in your browser. The weights started as a JSON file I kept for vetting my own installs, and the tool is mostly a UI wrapped around that file.

Getting the manifest is the only fiddly step. On macOS, installed extensions live under ~/Library/Application Support/Google/Chrome/Default/Extensions/<id>/<version>/manifest.json. On Windows, look in %LOCALAPPDATA%\Google\Chrome\User Data\Default\Extensions. For something you haven't installed yet, the CRX Viewer extension (ironic, I know) shows you the manifest straight from a Web Store listing.

How it stacks up against the alternatives

I used three other approaches before building my own, and I still use two of them. Here's the honest comparison:

What	Extension Guard	ExtensionTotal	CRX Viewer	Manual review
Where it runs	Client-side, in your browser	Their servers	Client-side	Your editor
Signup	No	For full reports	No	No
Combo detection	Yes, with explanations	Partial	No scoring at all	Only if you know the pairs
Sees actual code	No	Yes	Yes	Yes
Cost	Free	Free tier, paid API	Free	Your time

ExtensionTotal is genuinely good, and I still reach for it when I want signals beyond the manifest, like developer reputation and code analysis. The trade-off is that you're querying their backend, and the deeper reports sit behind an account. CRX Viewer is the opposite: full source access with zero interpretation. It shows you everything and explains nothing, which is fine if you already know what chrome.debugger buried in a minified bundle means.

CRXcavator used to be the obvious answer here. Duo Security built it, and it was excellent right up until they retired it. That shutdown is a big part of why the surviving options went cloud-side, and why I built mine to run locally instead. Cloud scanners can do more (they fetch the code, they track version history), but a manifest score shouldn't require sending anything anywhere.

When you shouldn't use it

A permission scanner reads declared capability. It can't see intent and it can't see runtime behavior, and that puts some hard limits on when Extension Guard is the right call.

A malicious update can keep the same permission set. The Cyberhaven attackers didn't request anything new; the extension already had the access they needed. Extension Guard would've scored the compromised version identically to the clean one. A scan tells you the blast radius if an extension ever goes bad. Whether it actually has gone bad is a different investigation, one that involves reading code and watching network traffic.

Open source extensions deserve an actual code read. If the repo is public and reasonably small, an hour with the source beats any score. The manifest tells you what the extension may do. The code tells you what it does.

Enterprise fleets need policy, and a scanner is the wrong layer. If you manage 200 machines, use Chrome's ExtensionInstallAllowlist and force-install policies. Scan candidates before they go on the allowlist, sure, but enforcement has to live in policy or people will quietly work around you.

Low scores can still hurt you. An extension with nothing beyond activeTab can still draw a convincing fake login popup. I don't have a good automated answer for that class of attack, and I'm suspicious of anyone who claims they do.

FAQ

Q: Does a high risk score mean an extension is malicious?
A: No. My password manager scores 88. A high score measures capability: how much damage the extension could do if it went rogue or got compromised. Score is blast radius. Whether you trust the developer is a separate judgment, and no scanner can make it for you.

Q: Where do I find the manifest for an installed extension?
A: Open chrome://extensions with Developer mode on and copy the extension's ID. Then find that ID inside your Chrome profile's Extensions folder (exact paths are in the section above) and open the manifest.json in the newest version folder. CRX Viewer works if you'd rather check without installing anything first.

Q: Didn't Manifest V3 fix all this?
A: I assumed it would, and I was mostly wrong. MV3 removed blocking webRequest for regular extensions and banned remotely hosted code, which are real improvements. But pairing cookies and scripting with <all_urls> is still perfectly legal in MV3, and the Cyberhaven attack happened well into the MV3 era.

Q: Does the scoring work for Firefox or Edge add-ons?
A: Mostly. Edge uses the same manifest format, so everything applies there. Firefox overlaps on most permission names; the scoring still runs, but a few Firefox-specific permissions fall back to a default weight, so treat those scores as a rough draft.

Q: Can I use this in CI for extensions my team ships?
A: Yes. Add process.exit(score > 60 ? 1 : 0) to the script above and wire it into your build. We run a version of that check before every store submission, and it caught an accidental <all_urls> grant in a pull request back in April 2026.

Written with AI assistance and human review. Try the tool at aidevhub.io/extension-guard.

MCP Inspector vs Postman in 2026: which one I actually use

AI Dev Hub — Tue, 23 Jun 2026 14:00:03 +0000

MCP Inspector vs Postman in 2026: which one I actually use

I use two of the three: MCP Inspector for live calls, and a small client-side validator for checking definitions before I ever start a server. Postman's MCP support works, but it was too much setup for the quick checks I do most. Same broken tool, run through all three, below.

Full disclosure: the MCP Tool Tester I link to below is one I built. I'd tried four other validators and every one either needed a running server, an npm install, or an account before it'd tell me my inputSchema had a typo. Mine doesn't. It's free, runs entirely in your browser, no signup, nothing uploaded. Paste it in, get an answer. If you've got a better one, tell me.

The task: a broken currency tool

Three weeks ago I was wiring up an MCP server for a currency tool, and the agent kept refusing to call it. No error message. It just ignored the tool. After 47 minutes of squinting I found it: my handler read a field called from_currency, but the schema I advertised defined currency_from. The model saw a contract it couldn't satisfy and quietly walked away.

Here's the thing about MCP tool definitions: the schema you advertise and the handler you write live in two different places, and nothing forces them to agree. JSON Schema will happily describe a field your code never reads. Most agents won't tell you why they skipped a tool, they just skip it. The expected behavior here was simple: send 100 USD with EUR as the target, get a converted amount back. What I actually got was nothing, no call attempted, which is the worst kind of bug because there's no stack trace to follow.

So I rebuilt that broken tool on purpose and ran it through three things people reach for when testing MCP: the official Inspector, Postman, and the validator I made. Here's the server, mismatch and all.

// server.js  (run: npm i @modelcontextprotocol/sdk zod && node server.js)
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";

const server = new McpServer({ name: "fx", version: "1.0.0" });

server.registerTool(
  "convert_currency",
  {
    description: "Convert an amount between two ISO 4217 currency codes",
    inputSchema: {
      amount: z.number().describe("amount to convert"),
      currency_from: z.string().length(3),
      currency_to: z.string().length(3),
    },
  },
  // Bug: the schema defines currency_from, the handler reads from_currency.
  // The names never line up, so the agent sees a field it can't supply.
  async ({ amount, from_currency, currency_to }) => ({
    content: [{ type: "text", text: `${amount} ${from_currency} -> ${currency_to}` }],
  })
);

await server.connect(new StdioServerTransport());

MCP Inspector: what happened

MCP Inspector is the official debugger. You run npx @modelcontextprotocol/inspector node server.js and it opens a local UI where you can list a server's tools and call them by hand. I ran it last Tuesday against the broken server above.

It connected in about two seconds and the tool showed up. Inspector reads your inputSchema and renders a form, so the field labels it expected (including currency_from) were right there on screen. That's where the bug got visible to me, because I knew my handler wanted from_currency. When I filled the form and hit call, my own handler threw on an undefined field. Honest, but late: I had to boot a full server to learn something a static check could have told me in seconds.

One more thing worth flagging. Inspector caches the tool list per session, so when I edited the schema and restarted the server, I had to reconnect to see the change. Minor, but I lost a couple of minutes the first time wondering why my fix wasn't showing. Once you learn to reconnect after every restart, it's fine. The history panel is also handy for replaying a call you already got working.

Inspector's strength is that it talks to a real, running server over the actual transport. Its limit is the same thing. It can't say a word about a definition until there's a live process to connect to.

Postman: what happened

Postman shipped MCP support in 2025, and for HTTP-based servers it's solid. I pointed it at the same tool after switching the transport to streamable HTTP, because Postman won't drive a stdio process. It discovered the tool, showed the schema, and let me send a call.

The request builder is nicer than Inspector's plain form, I'll give it that. Two things bugged me, though. I had to change my transport just to test, so I was poking at a slightly different server than the one I ship. And the validation is shallow: it happily sent garbage and reported the failure as a generic error response instead of pointing at the field that was wrong. Setup ate about 15 minutes. If you already live in Postman, that cost is mostly paid. For a fast definition check, it's heavy.

To be fair to Postman, the collection sharing is real value if you're on a team. I could save the MCP connection and hand it to a coworker, and they'd get the same setup without me writing a README. That's something neither Inspector nor my validator does. It just doesn't help the specific thing I was testing, which was whether a definition is correct before anyone runs it.

MCP Tool Tester: what happened

This one I built, so weigh that accordingly. The workflow is intentionally dumb: paste the tool definition (the JSON a server advertises, or a WebMCP tools array) and it checks the shape against the MCP schema plus a handful of lint rules I kept tripping over in real projects.

On the broken currency tool it flagged the mismatch in under a second: a required name with no matching property. It also caught two issues the other two never looked at, a description longer than the cutoff where some clients truncate (I still don't know the exact limit for every client, but I've watched it break around 1,024 characters) and an enum with a duplicated value. No server to boot, no install. It runs in the browser, so the definition never leaves your machine, which I care about because my tool descriptions leak internal endpoint names.

The lint rules came straight from bugs that cost me time. A required field with no property to back it. A description left empty, which makes some clients drop the tool entirely. I keep adding rules as I get burned, so the list grows in an embarrassingly autobiographical way. A few days ago a teammate's PR defined a tool that required user_id but only declared userId. Same family of bug as mine. The validator flagged it before review started, which saved a confused back-and-forth in comments.

What it won't do is execute your tool. It checks definitions, not behavior. Think of it as the step you run before Inspector.

The scorecard, and which one I reach for

Here's the same task scored across the things I actually care about:

Criterion	MCP Inspector	Postman	MCP Tool Tester
Setup to first result	about 10s via npx	about 15 min	about 3s, paste only
Needs a running server	yes	yes	no
Flags a bad definition pre-deploy	only as a runtime error	shallow	yes, field-level
Actually calls the tool	yes	yes	no
Validates WebMCP tool arrays	no	partial	yes
Price	free, open source	free tier, paid plans	free

The pattern is clear enough: Postman never wins a row outright for my use case. It's competent everywhere and best nowhere, which is a perfectly respectable place to be for a general tool that happens to speak MCP.

So which do I actually use? Two of them, at different moments. While I'm authoring a definition or reviewing a teammate's pull request, I paste it into the validator first, because catching a name mismatch in three seconds beats catching it after a 90-second server boot and a confused agent. Once the definition is clean, I bring up Inspector and call the thing for real over the transport I'll ship. Postman stays in the box unless a project already runs on it.

If I had to give up two of the three and keep one, I'd actually struggle, because they solve different halves of the problem. Definition correctness and runtime behavior aren't the same question. The validator answers the first in seconds; Inspector answers the second properly. That split is why I run both rather than picking a single winner.

If you want to throw your own definitions at the validator, it's here: MCP Tool Tester. Paste one in, read what's wrong, move on. I added the WebMCP checks last month after a reader pointed out that browser tool arrays carry their own sharp edges.

FAQ

Q: Does MCP Inspector validate my schema without running the server?
A: No. It connects to a live server over stdio or HTTP, lists the advertised tools, and lets you call them. If the server won't start, you get nothing to inspect. For static definition checks you want a validator that reads the raw JSON.

Q: What's the difference between MCP and WebMCP here?
A: MCP tools are advertised by a server you run over stdio or HTTP. WebMCP exposes tools from inside a web page to a browser-side agent. The definition shape is similar, but WebMCP adds constraints that most server-focused testers skip.

Q: Can I trust a browser-based validator with internal tool descriptions?
A: Only if it's genuinely client-side. Open the network tab and confirm nothing leaves your machine. The one I built runs entirely in-page for that reason. Don't take my word for it, check the requests.

Q: Is Postman a bad choice for MCP?
A: Not at all. If your server is HTTP and your team already uses Postman, the request history and sharing are genuinely useful. It's just heavier than I want for a quick sanity check on a definition.

Q: Which one is fastest for a quick check?
A: The validator, by a wide margin, because there's no server to start. Paste and read. For anything involving real calls and real responses, that speed stops mattering and Inspector's live connection is what you want.

Written with AI assistance and human review. Try the tool at aidevhub.io/mcp-tool-tester.

Stop hand-aligning markdown tables in 2026

AI Dev Hub — Thu, 18 Jun 2026 14:00:01 +0000

Stop hand-aligning markdown tables in 2026

Use a visual, spreadsheet-style editor that exports GitHub-Flavored Markdown so you stop counting spaces by hand. You paste a CSV export, tweak the alignment per column, and the markdown updates as you type. The editor I cover here runs in your browser and never uploads your data. It cut a 47-minute chore down to about two minutes for me.

Quick disclosure: the Markdown Table Generator I link to below is one I built. I'd tried five other generators. Every one either logged my table data to a server or buried the alignment controls behind a paywall, and a couple choked on cells that contained a pipe character. So I wrote my own. It's free and runs entirely client-side. There's no signup, and nothing gets uploaded. If you know a better one, tell me.

The 47 minutes I'll never get back

Three weeks ago I sat down to update the README for an internal CLI tool. It had grown to fourteen flags, and a teammate asked for a table documenting each one. I typed it by hand the way I always had, lining up the pipes with spaces so the raw file looked tidy in my editor.

Then I added one more flag. Every column shifted. I spent the next stretch nudging spaces around like it was 2011, and when I finally pushed, the diff came back as a wall of red and green because the renderer had touched every line in the block. The actual change was a single row.

I checked the clock afterward. 47 minutes on one table. Not the docs, not the code. The table itself.

Here's the thing that really gets me about those diffs. When you re-pad a whole table to fit one new row, every line registers as changed, so a reviewer can't see what you actually edited. They either skip the table review entirely or waste time eyeballing forty lines to find your one real change. Both outcomes are bad, and both come from a formatting quirk rather than anything meaningful.

Markdown tables look simple, and for a 2x2 they are. The trouble starts once a table grows past a handful of rows: manual alignment stops paying for itself, and any single edit reshuffles the whole block. The alignment markers don't help my memory either. Left is :---, right is ---:, center is :---:, and I still open the docs every time to remember which side the colon goes on.

This is a tiny problem, which is exactly why it irritates me so much. Nobody plans for it. You don't budget time to reformat a table the way you'd budget for a refactor, so it sneaks up during work you thought was basically finished and quietly eats real minutes. Multiply that across every doc and changelog you touch in a year, and the total stops being small.

What the generator actually does

There's no magic under the hood. A table generator holds your data as a 2D array and re-renders the text output on every keystroke. The fiddly part is the padding. Each column's width equals the length of its longest cell. Every other cell in that column gets padded to match, and the separator row carries the alignment colons.

Here's the core of that logic in plain JavaScript. Save it as table.js and run it with Node:

function toMarkdownTable(rows, aligns) {
  const widths = rows[0].map((_, c) =>
    Math.max(...rows.map(r => String(r[c]).length))
  );

  const pad = (text, width, align) => {
    const s = String(text);
    const gap = width - s.length;
    if (align === "right") return " ".repeat(gap) + s;
    if (align === "center") {
      const left = Math.floor(gap / 2);
      return " ".repeat(left) + s + " ".repeat(gap - left);
    }
    return s + " ".repeat(gap);
  };

  const row = cells =>
    "| " + cells.map((c, i) => pad(c, widths[i], aligns[i])).join(" | ") + " |";

  const sep = widths.map((w, i) => {
    const dash = "-".repeat(Math.max(3, w));
    if (aligns[i] === "right") return dash.slice(0, -1) + ":";
    if (aligns[i] === "center") return ":" + dash.slice(1, -1) + ":";
    return dash;
  });

  const [header, ...body] = rows;
  return [
    row(header),
    "| " + sep.join(" | ") + " |",
    ...body.map(row),
  ].join("\n");
}

const data = [
  ["flag", "type", "default"],
  ["--verbose", "bool", "false"],
  ["--retries", "int", "3"],
];

console.log(toMarkdownTable(data, ["left", "left", "right"]));

Run node table.js and you get a clean, aligned table out the other side. A visual editor wraps this same logic in a spreadsheet grid, so you can tab between cells and paste a block of CSV while the markdown updates live in a pane beside the data. You can try the editor here and drop in a CSV export straight from a spreadsheet app; it parses the rows in the browser and keeps the data on your machine.

The part I care about most is that CSV import. Most of my tables already exist somewhere, usually as a spreadsheet export or a query result. Retyping them by hand is the step that actually wastes my afternoon, so being able to paste raw CSV and get GFM back is the whole point for me.

One detail that took me longer than I'd like to admit: cells that contain a pipe character. In GFM a raw | inside a cell breaks the table unless you escape it as \|. The generator does that escaping for you on the way out, which is the single bug that pushed me off two of the hosted tools I'd been using. I don't know why so many of them skip it, but they do.

How it stacks up against the alternatives

I didn't build this in a vacuum. There are several common ways to make a markdown table, and each comes with a tradeoff. Here's how I'd line up the ones I actually reached for before I gave up and wrote my own:

Approach	CSV import	Alignment controls	Works offline	Keeps data local
This generator	Yes	Per-column buttons	Yes	Yes
tablesgenerator.com	Partial	Yes	No	Sends to a server
VS Code table extension	No	Auto-format only	Yes	Yes
Typing it by hand	No	Manual colons	Yes	Yes

The VS Code formatter is genuinely good if you already have the rows typed out. It'll re-align an existing table for you on save, which covers the "I added a row and everything shifted" pain nicely. What it won't do is take a CSV dump and build the table from scratch, and that build step is exactly where my time goes. The hosted generators do handle CSV, but the ones I tried either round-tripped my data through their backend or put alignment behind an account wall. For a thirty-second job, that's too much friction and too much trust to ask.

None of these are bad tools, to be clear. The hand-typed table is fine until it grows. The VS Code extension is the right answer if your rows are already in the file. My complaint is narrow: I wanted one workflow that started from a CSV and ended at GFM I could trust, without an account in the middle.

So the gap I kept hitting was a tool that could import CSV and give me real alignment controls without shipping my data off my laptop. That combination is what I ended up building.

When I don't reach for it

A generator isn't always the right call, and pretending it is would be dishonest.

If the table is tiny, say two columns and two rows, just type it. Opening any tool is slower than the keystrokes, and you won't fight alignment at that size anyway.

If the table is produced from data inside a script or a CI job, generate the markdown in code instead. The function above is a fine starting point, and a GUI parked in the middle of an automated pipeline defeats the purpose.

If you need merged cells or row spans, plain markdown can't express either of those. You'll have to drop down to raw HTML inside your markdown, or pick a different format. No generator can paper over a limitation that lives in the spec itself.

And honestly, I still hand-write a quick table now and then when I'm offline and can't be bothered to open a browser tab. Old habits stick around.

FAQ

Q: Does it output GitHub-Flavored Markdown specifically?
A: Yes. The separator row uses the colon syntax GitHub renders, so whatever alignment you set shows up correctly once the table lands in a repo or an issue.

Q: Can I paste data from Excel or Google Sheets?
A: Yes. Copy a range and paste it straight in. Spreadsheet copy usually arrives as tab-separated text, and the editor reads that the same way it reads CSV, so you don't need to export a file first.

Q: Is any of my table data sent anywhere?
A: No. Parsing and rendering both happen in your browser, and nothing about the table leaves your machine. That was the main reason I stopped using the hosted options I'd been relying on.

Q: What about very wide tables?
A: They work, but think about whoever reads the result. A table with twelve columns is painful in a rendered doc and it'll scroll sideways on a phone. I try to keep mine under six columns when the data lets me.

Q: Does it handle alignment per column or all at once?
A: Per column. Each column gets its own alignment toggle, so a numbers column can sit right-aligned while the text label beside it stays left. That mixed alignment is the part raw typing makes genuinely tedious, since every change recomputes the padding by hand.

Written with AI assistance and human review. Try the tool at aidevhub.io/markdown-table-generator.

Building a rules-file MCP server in 2026

AI Dev Hub — Thu, 11 Jun 2026 14:00:00 +0000

Building a rules-file MCP server in 2026

You can build an MCP server that writes CLAUDE.md, .cursorrules, and copilot-instructions.md from one prompt in about 30 lines of Python with FastMCP. Register the tool, point your assistant's config at it, and ask. The code is the easy part. Keeping three formats in sync is the actual work. Here's the server and the part that broke first.

The rules-file generator I link to below is one I built. I'd opened four separate template repos last spring trying to bootstrap a CLAUDE.md, and every one assumed a single assistant. Cursor users had .cursorrules. The Claude crowd had their own file. Nobody emitted every format from one input. So I wrote a small web page that does. It's free, runs entirely in your browser, no signup, nothing gets uploaded. If you've got a better one, tell me.

The goal

A local MCP server that any MCP-aware assistant can call to write its own rules file. You say 'set up the rules for this repo,' the assistant calls the tool with what it already knows about your stack, and three files land on disk:

CLAUDE.md at the repo root
.cursorrules beside it
.github/copilot-instructions.md for the Copilot users

No copy-paste between editor tabs. The outcome you'd screenshot: your assistant printing wrote 412 bytes to CLAUDE.md right in the chat, and the file appearing in your sidebar a half second later.

A rules file is just instructions your assistant loads before it reads your code: your conventions, plus the things it keeps getting wrong. Most teams write one by hand, then never touch it again, and it rots. Generating it from a tool means you can regenerate when the stack changes instead of editing prose at midnight. The sync problem is why I cared. I'd update CLAUDE.md and forget the .cursorrules copy, and then two teammates on different assistants would get different rules. That drift is quiet and it's expensive.

Why a server and not a plain script? Because the assistant can call it mid-conversation, with context it already has. It knows you're on Postgres and Vite before you say a word. That's the whole reason to bother.

Setup and auth

Here's the part nobody warns you about: there's no auth. I spent a chunk of last Thursday hunting for where an API key goes, and there isn't one. A local MCP server talks to your assistant over stdio, on your machine, as your user. The "credential" is filesystem permission. I was wrong about needing a token, and it took me 47 minutes of reading the spec before I believed it.

Install the SDK with pip install "mcp[cli]". Then register the server. For Claude Code, that's an .mcp.json file in the project root with a command and args pointing at your Python file. Cursor and Claude Desktop use their own JSON config, same shape. One gotcha that cost me real time: a stdio server must never write to stdout, because that stream carries the JSON-RPC frames. Any stray print() corrupts the protocol. Log to stderr or to a file.

You can pass a default through the environment too. I set RULES_DEFAULT_STACK in the .mcp.json env block so the tool has a fallback when the model forgets to send one. Small thing, but it turned a class of empty-stack files into a sane default. Restart the assistant after editing the config, by the way: most of them read MCP settings once at launch and won't pick up changes live, which confused me for a good ten minutes the first time.

If you just want the files without running a server, the rules-file generator at aidevhub.io/rules-file-generator produces the same multi-format output in the browser. I reach for it when I'm scaffolding a repo I'll touch once and forget.

The core code

The whole server fits in one file. Here it is, with the guard I added only after it bit me (more on that in the last section):

# server.py
from mcp.server.fastmcp import FastMCP
from pathlib import Path

mcp = FastMCP("rules-file-generator")

TEMPLATES = {
    "claude": "CLAUDE.md",
    "cursor": ".cursorrules",
    "copilot": ".github/copilot-instructions.md",
}

@mcp.tool()
def generate_rules(target: str, project_name: str, stack: str) -> str:
    """Write an AI rules file. target: claude | cursor | copilot."""
    filename = TEMPLATES.get(target)
    if filename is None:  # guard added after the bug in the last section
        raise ValueError(
            f"unknown target {target!r}; pick claude, cursor, or copilot"
        )

    body = (
        f"# {project_name}\n\n"
        f"Stack: {stack}\n\n"
        "## Conventions\n"
        "- Keep functions under 40 lines.\n"
        "- Put tests next to the code they cover.\n"
        "- No new dependency without a note in the PR.\n"
    )
    out = Path(filename)
    out.parent.mkdir(parents=True, exist_ok=True)
    out.write_text(body, encoding="utf-8")
    return f"wrote {len(body)} bytes to {out}"

if __name__ == "__main__":
    mcp.run()

The @mcp.tool() decorator does the heavy lifting. FastMCP reads your type hints (target: str, and the rest) and generates the JSON schema the assistant uses to call the tool correctly. You write a normal Python function and the protocol wiring is generated for you. Return a string and it shows up in the chat as the tool result.

To call it, you don't do anything special. You ask the assistant in plain language ('generate a Claude rules file for this project, we're on FastAPI and Postgres') and it maps that to the generate_rules arguments on its own. The first time you watch a model fill in stack="FastAPI + Postgres" without you naming the parameter, it feels like cheating. That mapping is exactly what the type hints buy you.

One design choice worth calling out: the tool writes the file itself instead of returning the text for the assistant to write. I went back and forth on this. Returning text keeps the server side-effect free, which is cleaner, but then the assistant has to turn around and call its own file-write tool, and you've added a round trip and a chance for it to mangle the content. Writing directly from the server means the bytes that get validated are the bytes that hit disk. For a generator like this, I'd rather own the write.

How FastMCP compares to the alternatives

I tried the TypeScript SDK first and bounced off it. It's fine. I just had a Python repo open and didn't want a Node toolchain to write three files. Here's how the realistic options stack up on the axes I cared about:

Approach	Lines to first working tool	Schema validation	Best when
FastMCP (Python)	~12	inferred from type hints	you want a tool running before lunch
TypeScript SDK	~40	explicit, via Zod	your stack is already Node
Raw JSON-RPC over stdio	~120	hand-written	you need zero dependencies

The Python row wins for this job because the template logic is a dozen lines of string formatting and the SDK gets out of the way. If your assistant tooling already lives in Node, the gap closes and the TypeScript SDK is the saner pick. Raw JSON-RPC is there if you enjoy pain or have a runtime the SDKs don't support yet.

One axis I left off the table on purpose: performance. None of these matter at this scale. You're writing a few hundred bytes once. If you're benchmarking a rules-file generator you've taken a wrong turn somewhere. Pick the SDK that matches the language your tooling already speaks and move on.

What broke the first time I ran it

A KeyError, and it cost me an afternoon. The first version looked up TEMPLATES[target] with a plain bracket, no guard. The model called the tool with target="claude-code" instead of "claude", Python raised a bare KeyError, and MCP wrapped it into a generic "tool failed" with nothing useful in the chat. I sat there convinced the stdio transport was broken. It was a typo in one argument.

The fix is the .get() plus an explicit ValueError that names the valid options. That error string goes straight back to the model, which reads it and retries with "claude". That feedback loop is the real point of returning good errors from a tool: the assistant self-corrects if you let it. Honestly this annoyed me for most of a day before it clicked.

The second thing: that stray print() I mentioned. I'd left one in to debug the KeyError, and it quietly corrupted the JSON-RPC frames. The server "connected" and then every call hung with no error at all. Pull debug output to stderr and it comes back to life. Both bugs were mine, and both took longer to find than the whole server took to write.

FAQ

Q: Do I need a separate server for Cursor and Claude Code?
A: No. It's the same server and the same stdio protocol. You register it in each tool's config file, but the server code doesn't change.

Q: Can the tool read my existing code to infer conventions?
A: Yes. Add a second tool that globs the repo and feeds a short summary into the template. Keep it read-only so a bad prompt can't rewrite files you didn't mean to touch.

Q: Why three files instead of one shared format?
A: Because the assistants haven't agreed on a format. CLAUDE.md is Markdown prose. The .cursorrules format is its own shape, and Copilot wants a specific path under .github. Until that converges, you generate each one.

Q: Is FastMCP ready for real use in 2026?
A: For local developer tooling, yes. I wouldn't expose one to the public internet without real auth in front of it, since the local model assumes a single trusted user.

Written with AI assistance and human review. Try the tool at aidevhub.io/rules-file-generator.