Every one of our amazon-product tests passed. The tool returned null for the price currency, null for the rating, null for every image, and the string "Brand: Amazon" where the brand should have been.
Nothing looked broken in CI, because the fixtures had been written to match the selectors instead of the site. The tests were checking that our code agreed with itself.
So over two days we ran all 28 tools of CrawlForge MCP against real sites — Wikipedia, Hacker News, Amazon, Shopify storefronts, npm, YouTube, GitHub, Reddit, an IRS PDF — and shipped six releases: v5.2.0 through v5.2.5, plus four releases of the shared crawlforge-extractors package.
This is the whole set, organised by what it does rather than by version number.
Table of contents
- Why six passing tests meant nothing
- Product data from the source
- Reddit-wide search after PullPush went dark
- Change tracking that actually fires
- Content that was being deleted
- Local LLMs finally work
- Browser automation and the stealth wedge
- Language detection and PDF tables
- One copy of the extractors
- How to upgrade
Why six passing tests meant nothing
The amazon-product failure is the clearest case. Six unit tests covered that template and all six passed, because every selector they exercised — a priceCurrency meta tag, #acrPopover .a-size-base, img.a-thumbnail-image — exists nowhere on Amazon today, and the fixture HTML had been written to contain them.
The pattern repeated:
-
scrape_with_actionshad seven Playwright defects invisible to its suite, because its fake pages implemented whatever the executor called — including APIs Playwright does not have. - The
youtube-videotemplate read an attribute that appears nowhere on a watch page, while its fixture kept the test green. -
process_documentread a field the PDF library declares and never writes.
So the new suites were written to fail against the pre-fix code first: 15 of 24 for amazon-product, 8 of 10 for the crawl_deep cache scope, 7 for the scrape_with_actions Playwright APIs. The suite is now 1,140 tests with MCP protocol compliance at 100% across all 28 tools — but the number that matters is that every fix below was found by a real page.
Product data from the source
Four scrape_template templates changed, and three moved the same direction: stop parsing the rendered page, read the structured data the site already publishes.
Shopify's Dawn theme ships every price badge in the markup unconditionally and hides the inapplicable ones with component CSS. A scraper reading the DOM sees "Sold out" on a product with a hundred units in stock. Ask an LLM for a compare-at price on a product that has none and it invents one — in our testing, 27.99.
Shopify serves the same data as JSON at /products/<handle>.json on every storefront including custom domains, so the new template reads that:
{
"tool": "scrape_template",
"params": {
"template": "shopify-product",
"url": "https://shop.example.com/products/kelpie-bandana"
}
}
Exact price, compare-at price, on_sale, currency, price range across variants, per-variant stock, options, images and tags — no HTML parsing, no LLM in the path.
Making that possible needed a change one level down: TemplateRegistry gained resolveUrl(url) to redirect the tool's single fetch and extractRaw(body, url) to parse a non-HTML response. The tool still owns the SSRF-guarded fetch.
npm-package moved the same way — npmjs.com answers plain fetches with 403, so it now reads the registry document. Weekly downloads are omitted rather than returned as null: they live on a separate endpoint, and the registry endpoint that carries them alongside package data is a search endpoint — ask it for left-pad and it answers pad-left. Returning another package's download count is worse than returning none.
And amazon-product was rebuilt from live captures of three pages. Currency comes from the hidden add-to-cart field, not a meta tag. Rating parses to a number from #acrPopover's title. Review count handles both "(198,594)" and "198,594 global ratings". Images drop Amazon's size token — the tokened URL is a 1 KB thumbnail, the same URL without it is the 16 KB original.
Reddit-wide search after PullPush went dark
If your Reddit pipeline started returning 429s this month, the fix is not a longer sleep, a rotating user agent or a proxy. We tested all three. Every request now returns:
{ "error": "Rate limit exceeded. This website does not provide free scraping resources for agents..." }
This is policy, not throttling. And it matters more than one dead endpoint, because Arctic Shift — the other public Reddit archive — cannot keyword-search across all of Reddit by design. Ask it for a query naming no subreddit and no author and it returns HTTP 400:
'query' query parameter requires one of: author, subreddit
So scoped searches still worked, and unscoped ones — where you know the phrase but not the community — had no backend at all.
reddit_search now serves that in two steps. Discovery: a site-restricted web search returns post URLs in relevance order, and post IDs come out of the /comments/<id>/ segment. Hydration: those IDs go to Arctic Shift in one batch lookup, returning the real post rows — score, comment count, subreddit, author, timestamp, full selftext.
These are archive rows, not search snippets. Discovery only supplies the addresses. Responses carry source: "web_discovery".
Two limits, both reported rather than hidden: after/before cannot apply on the discovery route, and the response says the filter was ignored instead of returning results that look filtered; and an unscoped comment search has no backend at all, so it asks for a scope rather than failing generically.
Change tracking that actually fires
track_changes had seven defects across this release, and the first meant price monitoring — the headline use case — did not work.
Price moves were scored by how much of the page they occupy. Significance was purely volumetric, so $19.99 → $29.99 and $19.99 → $99.99 both scored "minor". With notificationThreshold defaulting to "moderate", a monitor set up the obvious way never fired. Monetary amounts are now compared directly, and relative magnitude raises significance to at least "moderate", or "major" at 20%+. Only currency-tagged numbers count, so view counters do not fire.
customSelectors never scoped anything. It was read only inside section-level analysis, where it added hashes — so scoping made it worse. On an Amazon product page, scoping to the price block raised modified elements 456 → 3204 and payload 5.35 MB → 6.18 MB, on a page whose price had not moved.
structuralSimilarity lied in both directions — reporting 0 when it had not measured (a real score meaning "completely changed"), and never able to fall below 0.5, because the hierarchy half of the average compared an object that was initialised empty and never written.
Then 5.2.5 added three more, found while testing price tracking on Zillow and Newegg. Element-level analysis indexed a fixed tag allowlist — h1-h6, p, div, span, a — so a monitor scoped to address, td, li, tr or dd built a baseline of zero elements and could never report a change. A Zillow page scoped to ['address'] built a baseline of 0 elements from 9 matching nodes. A scoped compare also silently ran unscoped, and "Text content changed" appeared on compares that found no changes.
Content that was being deleted
Three scrape defects shared a root: the cleanup passes that strip invisible content were removing content that was visible.
Framework-streamed content was deleted. The hidden-content strip removed <div id="S:0" hidden> — where the Next.js App Router streams the rendered page. On a pricing page that wrapper is the whole visible page, so markdown came back empty and every price with it.
Script payload counted as page text. The bulk-removal guard sized elements with $('body').text(), which includes every inline <script>. On a Shopify storefront the denominator was 62,269 characters of which 4,295 was visible copy — so a wrapper holding the entire product section measured under the threshold and was deleted along with the price. The json path then had no price to extract, and the model invented one.
An LLM's schema echo was returned as data. The json format read only success from extract_with_llm and discarded the rest, so a schema document returned instead of page data reached callers looking like a clean extraction — well-formed JSON, and with no required fields it passed validation too.
And in 5.2.2, the markdown format stopped returning raw HTML. turndown-plugin-gfm converts a table only when its first row is entirely <th>; every other table hits the plugin's keep filter and is emitted verbatim. Real pages are full of layout tables — Hacker News is built out of them.
Local LLMs finally work
Ollama was never registered as an LLM provider. LLMManager registered only OpenAI and Anthropic, both gated on an API key. So on a machine running Ollama with no cloud keys, extract_structured skipped LLM extraction entirely and reported css_fallback — producing values like "$79.99$79.99" — and deep_research silently disabled query expansion, semantic ranking and synthesis. extract_with_llm has its own private client, which is why it worked and masked the gap.
Model routing now picks the best installed model instead of always llama3.2. Benchmarked against three live product pages with verified ground truth, gemma3:4b scored 18/18 at 1040 ms while llama3.2 scored 16/18 — and the failures are systematic: over five runs llama3.2 invented a compare-at price all five times. Parameter count did not predict accuracy; the 4B model beat both a 12B and a 20B.
Browser automation and the stealth wedge
Seven Playwright defects in scrape_with_actions, all invisible to the suite:
-
scroll toElementcalledscrollIntoView(), which does not exist on a handle or locator, so the branch threw every time it ran. - The
waitaction advertisedenabled/disabled/stablebut passed them to an API that rejects them. - The per-action
Promise.raceshared a deadline with the work it raced and won, replacing Playwright's real error with a bare "Action timeout" — and left a live timer per action. - Every recovery strategy sat behind
retries > 0while the schema defaultedretriesto 0, so none could ever run.
Separately, stealth_mode create_page never closed its page, leaking one Chromium renderer per call until the instance ran out of memory. A wedged browser was then reused forever behind truthiness-only checks, and cleanup hung on protocol calls to the dead browser. There is now isConnected() corpse detection, cleanup racing closes against 5 s deadlines with a SIGKILL fallback, and an in-flight launch mutex.
Language detection and PDF tables
Five languages were undetectable. analyze_content filters franc's output against a map of language names, and five keys were ISO 639-2/B codes — chi, gre, ara, nor, msa — which franc, emitting ISO 639-3, never produces. A page written entirely in Chinese returned null. So did Greek, Arabic, Norwegian and Malay.
A second bug sat behind it: franc scores whichever script is most common, and a CJK page carries the usual run of English product names and code samples. Detection now short-circuits on a Han/kana/hangul share of at least 10% of letters — measured across live pages, Latin-script pages sit at 0% and genuine CJK pages at 24–51%.
PDF table extraction returned [] for every document ever passed to it. The route read TableResult.mergedTables, a field the PDF library declares but never writes; the tables live on result.pages[].tables. Against the arXiv "Attention Is All You Need" paper, the IRS W-9 and the IRS 1040, the field being read held 0 tables in every case while the per-page arrays held 7, 6 and 11.
One copy of the extractors
The MCP server and our REST API each carried their own copy of the template extractors, in two languages, with nothing detecting divergence — and it diverged twice in two days. amazon-product was repaired on the MCP side on 25 August while the REST copy kept returning a null rating until the 26th.
There is now one implementation, published as crawlforge-extractors, which both surfaces install. Two more behaviours moved there because one surface had them and the other did not: readBody (charset-aware decoding and a body-size cap — the REST API called response.text(), so Shift_JIS and GBK pages came back as mojibake) and structureSignature.
The alternative was a parity test that told us after the fact which copy was wrong. Deleting the second copy is cheaper than detecting drift in it.
How to upgrade
npm install -g crawlforge-mcp-server@latest
crawlforge --version # 5.2.5
If your MCP client launches the server with npx, it picks up the release on the next restart. No tool was renamed and no output shape changed. The only price change is reddit_search, from 2 credits to 5 — it now spends the same upstream search call search_web does — so check your budget maths if you call it in a loop.
The package is crawlforge-mcp-server on npm, and the same fixes are already live on the hosted REST API.
Want to try any of this without setting up a server? The free plan's 1,000 one-time credits need no card, and the API reference covers all 28 tools.
Top comments (0)