DEV Community: quantoracledev

Detecting a website's tech stack: why grepping for framework names gets you wrong answers

quantoracledev — Thu, 30 Jul 2026 02:05:11 +0000

The obvious way to work out what a site is built with is to fetch the HTML and look for telltale strings. React, Next, Shopify, Cloudflare. It works often enough to feel like it works, which is the problem.

Here is a live example. Fetch vercel.com and grep for svelte:

curl -sSL https://vercel.com | grep -oi svelte | wc -l
# 2

Two hits. Vercel does not run SvelteKit. Both matches come from a footer link:

<a href="/docs/frameworks/full-stack/sveltekit" …>SvelteKit</a>

It is a docs nav item. Any detector built on "does the page contain the word" reports SvelteKit on a site that runs Next.js, and you would never notice, because the output looks exactly like a real detection.

That is the whole difficulty. Not fetching. Distinguishing a technology that is running from a word that happens to appear.

What actually identifies a stack

Five signal types, in rough order of how much I trust them:

1. Response headers. The most under-used and often the most precise.

curl -sSI https://vercel.com | grep -i "server\|x-powered-by"
# Server: Vercel
# X-Powered-By: Next.js, Payload

That single header names both the framework and the CMS. Payload is not visible anywhere in the rendered page — no asset path, no DOM marker, no JS global. One header, one detection you cannot get any other way.

2. Asset origins. What the page loads, and from where. cdn.shopify.com on a page means Shopify is serving it, in a way that a mention of the word "Shopify" does not:

curl -sSL https://www.allbirds.com | grep -o "cdn\.shopify" | wc -l
# 13

Compare that with 83 occurrences of the bare string Shopify on the same page — most of them copy, schema markup, and script variable names. The 13 asset references are the evidence; the 83 are noise that happens to correlate. (Both counts drift every time they ship a homepage change — I watched them move while writing this. The ratio is the durable part, not the numbers.)

3. Cookies. _shopify_y, wordpress_logged_in, ASP.NET_SessionId. Hard to fake incidentally.

4. DOM markers. [data-reactroot], #__next, [data-svelte], wp-content paths in the markup.

5. JS globals after hydration. window.__NUXT__, window.__NEXT_DATA__, window.Shopify, React on window. This is the tier that needs a real browser.

Where static fetching genuinely runs out

I want to be fair here, because "you need a headless browser" is over-claimed. Static fetching gets you further than people assume — Next.js sites usually announce themselves:

curl -sSL https://www.notion.so | grep -c "__NEXT_DATA__"   # 1
curl -sSI https://www.notion.so | grep -i x-powered-by      # x-powered-by: Next.js

Both work with no browser at all.

Where it does break down is client-rendered SPAs that ship an empty shell. If the server returns <div id="root"></div> and everything else arrives via JS, there is nothing in the static HTML to match — the framework only becomes visible once the bundle executes and mutates the DOM. Same for anything injected by a tag manager, which is most analytics and chat widgets on a lot of sites.

So the honest split: headers and asset origins cover a surprising amount. A browser is what you need for the client-side tier, and for avoiding the false-positive trap by checking what the page did rather than what it said.

The DIY version

A decent detector is a few hundred lines:

const res = await fetch(url, { redirect: 'follow' });
const html = await res.text();
const hits = [];

// headers — highest confidence
const powered = res.headers.get('x-powered-by');
if (powered) hits.push({ name: powered, category: 'framework', evidence: `header: x-powered-by: ${powered}` });

// asset origins, NOT bare strings
if (/cdn\.shopify\.com/.test(html))
  hits.push({ name: 'Shopify', category: 'ecommerce', evidence: 'asset: cdn.shopify.com' });

// DOM markers
if (/<div id="__next"/.test(html))
  hits.push({ name: 'Next.js', category: 'framework', evidence: 'dom: #__next' });

Note the shape: every hit carries the reason it fired. That is not decoration. Without it you cannot tell a real detection from a footer link, and you will not find out which you have until someone acts on it.

The ongoing cost is the part people underestimate. Signatures rot — frameworks change their markers between majors, data-reactroot disappeared in React 18, CDNs get renamed, hosts add and drop headers. A detector is a maintenance commitment, not a weekend project.

The one-call version

Tech Stack Detector renders each site in a real browser and returns one row per site, with the evidence for every match:

curl -X POST "https://api.apify.com/v2/acts/fetchbase~tech-stack-detector/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "urls": ["https://www.allbirds.com", "https://nextjs.org"] }'

{
  "url": "https://www.allbirds.com",
  "techCount": 14,
  "cms": "Shopify",
  "frameworks": "React",
  "analytics": "Google Analytics (GA4), Hotjar",
  "hosting": "Cloudflare",
  "technologies": [
    { "name": "Shopify", "category": "ecommerce", "evidence": "asset: https://cdn.shopify.com/…" },
    { "name": "React",   "category": "framework", "evidence": "dom: [data-reactroot]" },
    { "name": "Stripe",  "category": "payments",  "evidence": "asset: https://js.stripe.com/v3" }
  ]
}

Around 100 technologies across CMS, e-commerce, JS frameworks, analytics, marketing, CDN and hosting, server and language, payments, and consent tooling. Charged per site analysed; failures cost nothing.

It is also callable as an MCP tool, which is genuinely the more interesting use — an agent asked "what is this company running" can go and find out rather than guessing from training data.

Three ready-made variants if you want to skip the input wiring:

Detect any website's tech stack — the general sweep, evidence included
Check whether a site runs WordPress — the single-question version
Find out if a store is on Shopify — asset-origin check, not a word search

The underrated thing

If you only do one thing from this post: read x-powered-by before you parse any HTML.

It is one header, most detectors treat it as an afterthought, and it gave up Vercel's CMS — Payload — which is invisible by every other method. For competitive research that is often the single highest-value field on the whole response, and it costs you a HEAD request.

The corollary, and the reason I wrote this: any detection without evidence attached is a guess wearing a confident face. Vercel "uses SvelteKit" is what you get otherwise, and it looks identical to a real answer.

The App Store review ceiling nobody mentions — and how to get 8x past it

quantoracledev — Thu, 23 Jul 2026 02:55:43 +0000

If you've ever pulled App Store reviews programmatically, you've hit a wall you may not have noticed: you can only get 500 reviews per app. Not 500 per request — 500, total, forever.

Apple's public customer-reviews RSS feed is paginated at 50 entries and stops at page 10. Request page 11 and you get nothing back. Every App Store review tool inherits that ceiling, because they all read the same feed.

Here's the part that isn't widely known: the ceiling is per storefront, not per app.

Each country is its own feed

Apple runs ~175 country storefronts, and each one paginates independently. Same app, same endpoint shape, different us / gb / de in the path:

curl "https://itunes.apple.com/us/rss/customerreviews/page=1/id=310633997/sortBy=mostRecent/json"
curl "https://itunes.apple.com/de/rss/customerreviews/page=1/id=310633997/sortBy=mostRecent/json"
curl "https://itunes.apple.com/br/rss/customerreviews/page=1/id=310633997/sortBy=mostRecent/json"

No key, no auth, no scraping — this is Apple's own public feed.

I measured it against WhatsApp across 10 storefronts. Result: 4,250 unique reviews in 3.2 seconds, zero duplicates. Versus the 500 you'd get from the US alone. That's 8.5× on ten countries; sweep all ~175 and the ceiling moves to roughly 87,500.

The more interesting output

Depth was what I went looking for. The per-country breakdown turned out to be the more useful thing:

Storefront	Rating	Ratings count
🇲🇽 Mexico	4.78	5,893,510
🇧🇷 Brazil	4.78	8,313,182
🇬🇧 UK	4.71	4,116,608
🇺🇸 US	4.69	18,293,098
🇰🇷 Korea	4.68	127,839
🇮🇳 India	4.62	7,960,859
🇩🇪 Germany	4.60	3,530,320
🇫🇷 France	4.56	3,432,960
🇯🇵 Japan	4.55	254,035

Same app, same week: 4.78 in Mexico, 4.55 in Japan. If you only ever query the US store, that spread is invisible to you — and it's exactly the thing an international product team wants to know. Which markets are we losing, and what are people saying there?

Doing it yourself

The mechanics are simple; the tedium is in the details. Roughly:

const PAGE_SIZE = 50, MAX_PAGES = 10;

async function reviewsFor(appId, cc) {
  const out = [];
  for (let page = 1; page <= MAX_PAGES; page++) {
    const url = `https://itunes.apple.com/${cc}/rss/customerreviews/page=${page}/id=${appId}/sortBy=mostRecent/json`;
    let json;
    try { json = await (await fetch(url)).json(); } catch { break; } // past the ceiling Apple stops returning JSON
    const entries = json?.feed?.entry;
    if (!entries) break;
    // page 1 can lead with an app-summary entry that has no im:rating — skip it
    const reviews = [].concat(entries).filter((e) => e['im:rating']);
    if (!reviews.length) break;
    out.push(...reviews.map((e) => ({
      country: cc,
      rating: Number(e['im:rating'].label),
      title: e.title?.label,
      body: e.content?.label,
      version: e['im:version']?.label,
    })));
    if (reviews.length < PAGE_SIZE) break;
  }
  return out;
}

Things that will bite you: page 1 sometimes leads with an app-summary entry that has no rating; Apple returns non-JSON rather than an empty feed past the ceiling, so you need the try/catch to stop cleanly; and not every app exists in every storefront — China commonly returns nothing, which is app availability, not an error.

Or one call

I packaged this as App Store Reviews API — storefront sweeping, dedup, per-country rating breakdown, rating filters:

{
  "apps": ["310633997"],
  "countries": ["all"],
  "maxReviewsPerCountry": 500
}

A few use cases worth stealing whether you build it or not:

Global complaints, filtered. minRating: 1, maxRating: 2 across every storefront gives you a worldwide bug-and-churn report you can hand straight to an LLM.

Find negative reviews worldwide

Rating drift monitoring. includeReviews: false returns just the per-country ratings — cheap enough to run on a schedule and alert when a market starts sliding.

Compare an app's rating by country

Competitive teardown. Several apps, several countries, one dataset.

Compare competitor app reviews

What this can't do

500 per storefront is Apple's limit, not a tool's. Nothing gets past it for a single country. Sweeping storefronts is the only lever.
Recent-first. This is the live picture, not full review history.
No Google Play here. Play has no equivalent public API, so it needs actual scraping — a different risk profile, deliberately out of scope.

The whole thing runs on Apple's documented public endpoints, which is why it's fast, doesn't need proxies, and doesn't break when a page layout changes. The rest of the suite is at apify.com/fetchbase.

Curious whether anyone has found a documented way past the 10-page cap — if you have, I'd genuinely like to know.

Scrape any company's job postings — Greenhouse, Lever & Ashby, with one API call

quantoracledev — Wed, 22 Jul 2026 02:11:12 +0000

Almost every tech company's job board runs on one of a handful of ATS platforms — Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee. And nearly all of them expose a public, documented JSON API for their postings.

Which means "scrape job postings" isn't really a scraping problem. It's a normalization problem: six different response shapes, board-slug discovery, HTML-encoded descriptions, and compensation data that's structured differently everywhere it exists at all.

Here's the DIY version, the one-call version, and one genuinely underrated thing hiding in this data.

The raw APIs

No key, no auth. These are live right now:

# Greenhouse
curl "https://boards-api.greenhouse.io/v1/boards/stripe/jobs"
# Lever
curl "https://api.lever.co/v0/postings/palantir?mode=json"
# Ashby
curl "https://api.ashbyhq.com/posting-api/job-board/openai"

At the time of writing that's 540 open roles at Stripe, 286 at Palantir, and 749 at OpenAI — three calls, three completely different JSON shapes. (Those counts move every day. The shapes don't.)

Greenhouse nests offices and departments as arrays of objects. Lever flattens everything into categories. Ashby puts the city in location.name and hides compensation behind a separate includeCompensation=true flag. Descriptions come back as escaped HTML on some, Markdown-ish on others. Multiply by six platforms and you've got a weekend project plus ongoing maintenance every time one of them changes.

The one-call version

Job Postings API auto-detects which ATS a company uses — from a bare slug or a full careers URL — and returns one normalized row per job. Here's a real, unedited record from a live run:

curl -X POST "https://api.apify.com/v2/acts/fetchbase~job-postings-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "companies": ["stripe", "gitlab", "https://jobs.ashbyhq.com/openai"] }'

{
  "company": "stripe",
  "ats": "greenhouse",
  "id": "7954688",
  "title": "Account Executive, AI Sales (Grower)",
  "department": "1654 Account Executives (AI)",
  "location": "San Francisco, CA",
  "remote": null,
  "url": "https://stripe.com/jobs/search?gh_jid=7954688",
  "publishedAt": "2026-07-21T18:51:19-04:00",
  "salary": null,
  "description": "## Who we are\n\n### About Stripe\n\nStripe is a financial infrastructure..."
}

Note remote and salary are null there — and that's the honest part. Those fields are only as good as what the ATS publishes. Greenhouse boards frequently omit both. The actor normalizes the shape; it can't invent data the source doesn't expose.

Which brings us to the interesting bit.

The underrated part: some boards publish full comp

Ashby boards expose structured compensation, and plenty of companies leave it on. Of those 749 OpenAI roles, 602 — about 80% — carry a published pay range, formatted like $257K – $335K • Offers Equity. 477 are flagged remote.

There's a counting trap here worth naming, because I walked into it myself. The compensation object is present on every job, even when there's no range inside it:

// a role with no published range — object exists, summary is null
"compensation": { "compensationTierSummary": null, "compensationTiers": [] }

Count the objects and you get a triumphant 100%. Count compensationTierSummary and you get the real 80%. If you're building on this, key off the summary, not the parent.

Even at 80%, that's a real, public, structured compensation dataset that most people assume you have to buy from Levels.fyi or scrape out of rendered HTML. It's sitting behind a GET request.

Useful inputs while you're exploring:

Input	Does
`companies`	Slugs or careers URLs — mix ATSes freely in one run
`remoteOnly`	Keep only roles the board flags remote
`titleFilter`	Substring match on title (`"engineer"`)
`maxJobsPerCompany`	Cap big boards (default 1000)
`includeDescriptions`	Set `false` for a fast, light index

You're billed per job returned, so filters cut cost as well as noise.

Things worth building with it

A "who's hiring in AI" snapshot. One run across several labs, compare volume and department mix:

See which AI companies are hiring right now

A remote-jobs alert. remoteOnly + titleFilter: "engineer" across your target companies, on a schedule, diffed against yesterday's dataset → new postings to Slack:

Find remote engineering jobs at top startups

Comp research. Pull a whole Ashby board and you get ranges attached to titles and locations:

Scrape OpenAI's openings — 749 live roles, ~80% carrying pay ranges

Hiring as a market signal. Posting counts over time are a leading indicator — teams that are shipping are hiring, and teams in trouble quietly stop. Snapshot weekly and you've built a trends dataset nobody sells you.

Why the API route beats HTML scraping here

These are intended-use, documented endpoints. No bot walls, no proxy budget, no 3am breakage when a careers page gets redesigned. The tedious part — ATS detection, six-way shape merge, HTML→Markdown, comp parsing — is the part worth not rewriting yourself.

Pay per job returned, failed runs cost nothing, and Apify's free credits cover plenty of testing. The rest of the utility suite lives at apify.com/fetchbase.

Built this because job data shouldn't require scraping infrastructure. If you want another ATS supported or normalized salary output, say so in the comments or the actor's Issues tab.

How to check Core Web Vitals for hundreds of URLs at once (API + code)

quantoracledev — Wed, 15 Jul 2026 02:37:16 +0000

Google's PageSpeed Insights is great for auditing one page. But the moment you want to track Core Web Vitals across a whole site — or every client site you manage, or every page after a deploy — clicking through a web UI (or fighting per-key rate limits) stops scaling.

Here's how to measure LCP, CLS, FCP and TTFB in bulk, in real Chromium, with a plain API call and no rate-limit juggling.

What we're measuring

The metrics that actually affect Google ranking and user experience:

LCP (Largest Contentful Paint) — when the main content appears. Target < 2.5s.
CLS (Cumulative Layout Shift) — how much the layout jumps. Target < 0.1.
FCP (First Contentful Paint) — first pixel of content.
TTFB (Time To First Byte) — server responsiveness.

Plus the why: total page weight, request count, third-party load, DOM size — the things you actually change to fix a bad score.

The bulk approach

Website Performance Audit loads each URL in a real browser, measures Core Web Vitals via the standard PerformanceObserver APIs (lab data, comparable to Lighthouse), and returns a 0–100 score plus a prioritized list of opportunities.

curl -X POST "https://api.apify.com/v2/acts/fetchbase~website-performance-audit/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "urls": ["https://example.com", "https://example.com/pricing", "https://example.com/blog"], "device": "mobile" }'

Each URL comes back like this:

{
  "url": "https://example.com/pricing",
  "score": 72,
  "lcpMs": 2380, "cls": 0.04, "fcpMs": 1290, "ttfbMs": 410,
  "totalKb": 2648, "requests": 84, "thirdPartyRequests": 37,
  "opportunities": [
    { "severity": "warning", "title": "Render-blocking resources", "detail": "5 synchronous scripts/stylesheets in <head>…" },
    { "severity": "warning", "title": "Heavy JavaScript", "detail": "Scripts total 1180 KB…" }
  ]
}

Make it a performance budget in CI

The useful part isn't a one-time number — it's catching regressions. Fail your build if a key page's score drops or LCP crosses a threshold:

const urls = ["https://example.com", "https://example.com/pricing"];

const results = await fetch(
  "https://api.apify.com/v2/acts/fetchbase~website-performance-audit/run-sync-get-dataset-items?token=" + process.env.APIFY_TOKEN,
  {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ urls, device: "mobile" }),
  }
).then((r) => r.json());

const budget = { score: 70, lcpMs: 2500, cls: 0.1 };
const failures = results.filter(
  (r) => r.score < budget.score || r.lcpMs > budget.lcpMs || r.cls > budget.cls
);

if (failures.length) {
  console.error("Performance budget exceeded:");
  for (const f of failures) console.error(`  ${f.url} — score ${f.score}, LCP ${f.lcpMs}ms, CLS ${f.cls}`);
  process.exit(1);
}

Drop that in a GitHub Action after deploy and you get a hard gate on performance regressions — across as many pages as you want, in one call.

Auditing a whole site on a schedule

Feed it your sitemap URLs (or pair it with a crawler) and run it on an Apify Schedule weekly. Sort the resulting dataset by score ascending and you've got a prioritized worklist: worst pages first, each with the specific opportunities to fix.

SEO, not just speed?

Core Web Vitals are one ranking input. If you also want on-page SEO — titles, meta descriptions, headings, canonical tags, structured data, broken links — the companion SEO Audit actor returns all of that plus Core Web Vitals in one record, each issue paired with a concrete fix.

Cost

Pay per page audited, no subscription, no per-key rate limits, failures free. Auditing a 100-page site is pocket change and runs in a couple of minutes — and Apify's free credits cover plenty of testing.

Built this because bulk Core Web Vitals shouldn't require a spreadsheet of PageSpeed tabs. Feedback and feature requests (INP? filmstrips?) via the actor's issues tab. The full utility suite is at apify.com/fetchbase.

Turn any website into clean Markdown for your RAG pipeline (with code)

quantoracledev — Sun, 12 Jul 2026 16:18:30 +0000

If you've built anything with retrieval-augmented generation (RAG), you know the unglamorous truth: 80% of the work is getting clean text in. Your embeddings are only as good as the content you feed them, and raw web pages are a mess of nav bars, cookie banners, ads, share widgets, and <div> soup.

This post shows a fast, reliable way to convert any URL — or a whole site — into clean, LLM-ready Markdown, with code you can drop into a pipeline today.

Why not just `requests` + BeautifulSoup?

You can, and for a single static page it's fine:

import requests
from bs4 import BeautifulSoup

html = requests.get("https://example.com").text
text = BeautifulSoup(html, "html.parser").get_text()

But this falls over fast in the real world:

You get everything — menus, footers, "related articles," cookie notices — all polluting your chunks and your embeddings.
JavaScript-rendered sites (React, Vue, Next.js) return an empty shell; you need a real browser.
Structure is lost — headings, lists, tables and code blocks matter to an LLM, and .get_text() flattens them.
Crawling a whole docs site means writing link-following, dedup, robots.txt handling, and concurrency yourself.

Readability + a headless browser + a Markdown converter solves the quality problem, but now you're maintaining browser infrastructure. That's the part worth outsourcing.

The hosted approach

Website to Markdown runs the whole pipeline for you — real Chromium rendering → Mozilla Readability (main-content extraction) → Markdown with headings, links, tables and code preserved. It respects robots.txt, can crawl a site via its sitemap, and returns one clean record per page.

One page

curl -X POST "https://api.apify.com/v2/acts/fetchbase~website-to-markdown/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "urls": ["https://en.wikipedia.org/wiki/Retrieval-augmented_generation"] }'

You get back:

[{
  "url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
  "title": "Retrieval-augmented generation",
  "description": "…",
  "wordCount": 2841,
  "markdown": "**Retrieval-augmented generation (RAG)** is a technique that…"
}]

A whole documentation site

Point it at the root, turn on sitemap discovery, and filter to the section you care about:

{
  "urls": ["https://docs.your-product.com"],
  "crawl": true,
  "useSitemap": true,
  "maxPages": 300,
  "includeUrlPatterns": ["*/docs/*"],
  "excludeUrlPatterns": ["*/changelog/*"]
}

Wiring it into a RAG pipeline

Here's the whole ingest step in Node — fetch clean Markdown, chunk it, embed it:

const runResp = await fetch(
  "https://api.apify.com/v2/acts/fetchbase~website-to-markdown/run-sync-get-dataset-items?token=" + process.env.APIFY_TOKEN,
  {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      urls: ["https://docs.your-product.com"],
      crawl: true,
      useSitemap: true,
      maxPages: 50,
    }),
  }
);
const pages = await runResp.json();

// Chunk each page's markdown and embed
for (const page of pages) {
  const chunks = chunkMarkdown(page.markdown, { maxTokens: 500 });
  for (const chunk of chunks) {
    const embedding = await embed(chunk);
    await vectorStore.upsert({
      text: chunk,
      embedding,
      metadata: { url: page.url, title: page.title },
    });
  }
}

Because the output is clean Markdown, your chunker splits on real headings instead of guessing, and your citations point at real page titles and URLs.

For big crawls (hundreds of pages), use the async endpoint (POST /v2/acts/…/runs)
and poll the run, or run it on a Schedule — the run-sync-* endpoint used
above is best for a handful of pages, since it caps at ~5 minutes.

Don't forget PDFs and Word docs

Half of most companies' knowledge lives in PDFs and .docx files, not web pages. The companion PDF & DOCX to Markdown actor handles those with the same output shape, so your ingest code stays uniform:

{ "fileUrls": ["https://example.com/whitepaper.pdf"], "outputFormat": "markdown" }

Keep the index fresh

Wrap the run in an Apify Schedule (say, nightly) and re-embed changed pages. Use the sinceDate pattern on feeds or a content hash to avoid re-embedding everything.

Cost

It's pay-per-page with no subscription and no startup fee — you're only billed for pages that extract successfully, and free-tier Apify credits cover a generous amount of testing. For a few hundred docs pages that's cents, versus the ongoing cost of running and babysitting your own browser fleet.

If you try it, I'd genuinely like feedback — what breaks, what's missing. Open an issue on the actor and it gets fixed fast. There's a whole suite of these (screenshots, SEO/Core-Web-Vitals audits, tech-stack detection, feeds, job APIs) at apify.com/fetchbase if they're useful to you.

AgentKit vs LangChain vs Direct HTTP — picking the right integration for paid agent APIs

quantoracledev — Thu, 14 May 2026 01:24:30 +0000

When you're plugging an LLM agent into an external API, you have three reasonable patterns: hand-rolled HTTP, AgentKit's action provider model, or LangChain's tool calling. They all work. They produce identical outputs against the same input.

So which one should you actually use?

I built the exact same agent three different ways — answering the same Kelly Criterion question — and the answer to "which one" depends on your stack, your team, and (most underrated) your wallet model. Here's the honest comparison.

The test case

Question: "I have a 55% win rate, $150 average win, $100 average loss. What's my Kelly fraction?"

Answer: f* = 17.5% (full Kelly), or 8.75% (half-Kelly — what most quant funds actually use).

The math doesn't care which integration computes it. Kelly is a 1956 formula that fits in a tweet:

f* = (p · b − q) / b

Where p = win probability, q = 1-p, b = avg_win/avg_loss.

What changes between integrations is everything around the math: how the agent discovers the tool, how it formats inputs, how it handles errors, and — for paid services — how it pays.

Pattern 1 — Direct HTTP

The minimum-viable integration:

curl -s -X POST https://api.quantoracle.dev/v1/risk/kelly \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "discrete",
    "win_rate": 0.55,
    "avg_win": 150,
    "avg_loss": 100
  }'

Response:

{
  "full_kelly": 0.175,
  "half_kelly": 0.0875,
  "quarter_kelly": 0.0438,
  "edge": 32.5,
  "payoff_ratio": 1.5,
  "recommended": "HALF_KELLY",
  "ms": 8.2
}

Pros: zero dependencies. Zero auth setup (this API has a 1K-calls/day free tier). Works from any language. Easy to cache, easy to mock for tests.

Cons: you handle everything yourself. Schema validation, error retries, rate limit handling, payment if there are paid tiers. The LLM doesn't know about the endpoint — you're putting raw HTTP into your agent loop.

When this is right: you're building a deterministic backtest pipeline, a CI script, or any pre-prompted workflow where the agent doesn't need to discover tools at runtime. Or you're building a thin proxy that wraps a paid API for resale.

Pattern 2 — Coinbase AgentKit (TypeScript)

This is where it gets interesting if your agent has a wallet:

import { AgentKit, CdpEvmWalletProvider } from "@coinbase/agentkit";
import { quantoracleActionProvider } from "./quantoracle";

const walletProvider = await CdpEvmWalletProvider.configureWithWallet({
  apiKeyId: process.env.CDP_API_KEY_ID!,
  apiKeySecret: process.env.CDP_API_KEY_SECRET!,
  networkId: "base-mainnet",
});

const agentkit = await AgentKit.from({
  walletProvider,
  actionProviders: [quantoracleActionProvider()],
});

// LLM picks `calculate_kelly` because the Zod schema's .describe()
// text matches "Kelly fraction" / "win rate" / "payoff"
const tools = await getLangChainTools(agentkit);

The LLM doesn't see raw HTTP. It sees an action called calculate_kelly with parameters documented via Zod schemas. AgentKit handles the network call and returns the parsed result.

The killer feature: the same agent can call paid endpoints (e.g. assess_portfolio_risk at $0.04 USDC) and AgentKit's wallet handles payment automatically via x402. The LLM doesn't write any payment code. No API key. No signup. No billing system. The wallet just needs USDC.

Pros: clean tool model for the LLM. Wallet-native payment (huge if you're building agents that need to pay other agents). Type-safe via TypeScript + Zod. Built-in tracing through LangChain.

Cons: TypeScript-only as of today. Adds the @coinbase/agentkit dependency tree. Wallet provisioning is one more thing to set up (though CdpEvmWalletProvider.configureWithWallet is basically two env vars).

When this is right: Coinbase-stack agents, x402-native agents, EVM or Solana wallets, autonomous trading bots that need to pay for premium tools, anything where wallet-native auth is the natural model.

Pattern 3 — LangChain (Python)

For when you're in the Python ecosystem and want broad tool access:

from langchain_quantoracle import QuantOracleToolkit
from langchain_openai import ChatOpenAI
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate

# All 73 tools from this API (63 calculators + 10 composites)
tools = QuantOracleToolkit().get_tools()

# Or filter
tools = QuantOracleToolkit(categories=["risk", "stats"]).get_tools()

llm = ChatOpenAI(model="gpt-4o", temperature=0)
prompt = ChatPromptTemplate.from_messages([
    ("system", "Use the provided tools for any math."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

agent = create_tool_calling_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

response = executor.invoke({
    "input": "I have 55% win rate, $150 avg win, $100 avg loss — Kelly?"
})

LangChain's toolkit pattern is the established way to give an agent a curated set of tools. Pydantic schemas describe each tool to the LLM. The toolkit handles HTTP, retries, and error wrapping.

Pros: Python-native (huge for the quant + ML crowd). Composes with LangGraph for stateful workflows. Works with any LangChain-compatible LLM (OpenAI, Anthropic, local Llama, etc.). Exposes all 73 endpoints by default — useful when you don't want to pre-curate.

Cons: no native wallet integration — if you want paid endpoints, you handle x402 separately. The breadth (73 tools) can confuse smaller LLMs that aren't great at narrowing from many options.

When this is right: Python-native pipelines, multi-tool agents (combining this API with web search, file ops, vector DBs), LangGraph workflows, anywhere you want broad tool access without curating a subset.

The decision rule

Your situation	Use
Scripts / backtests / CI	Direct HTTP
Building on Coinbase / x402 / CDP wallets	AgentKit
Python, LangChain, or LangGraph workflows	LangChain Python
OpenAI custom GPT	GPT Actions (a 4th path I didn't cover here)
MCP client (Claude Desktop, Cursor)	MCP server (a 5th path)
Solana ecosystem, want sub-second x402 settlement	AgentKit with `SolanaKeypairWalletProvider`

All paths hit the same underlying API. The math is byte-identical across integrations for the same inputs. Pick by ergonomics and wallet model, not by capability.

Two patterns from production that combine these

These aren't mutually exclusive. In real systems you'll often use more than one:

Backtest in Python, deploy in TypeScript

You develop your strategy in a Jupyter notebook with langchain-quantoracle (all 73 tools available, easy to explore). When you find an edge worth productionizing, you re-implement the agent in TypeScript with AgentKit (curated 5-tool subset, wallet-native payments). Same API answers both. Your research notebook and your production agent agree because they're hitting the same engine.

Free tier for research, paid for production

The free tier (1K calls/IP/day) covers backtests across thousands of historical days. Once you've validated the strategy and want it running 24/7 as a paid signal service, you switch to the AgentKit + x402 pattern so the wallet pays per call. The economics scale linearly with usage instead of forcing a subscription decision.

What this isn't about

This isn't about which framework is "better." All three are excellent. AgentKit's wallet integration is unique value if you need x402; LangChain's tool ecosystem is unique value if you're orchestrating many tools; raw HTTP is unique value when you want maximum control.

It's also not about quant finance specifically. The same decision rule applies to any external API your agent might use — weather, web search, on-chain data, image generation. The frameworks differ in how they help your agent discover and pay for tools, not in what those tools can do.

My pick — and why

If I had to start from zero today:

Python team, no wallet → LangChain. Most flexible. Most existing tools.
TypeScript team with crypto-native agents → AgentKit. The wallet-paid x402 flow is genuinely magical when you have it working.
Mixed team or no strong preference → start with direct HTTP for the first integration, then add a framework when you have a second one. The HTTP version is your reference implementation either way.

The wrong move is "framework first, problem second." All three integrations work because the underlying API is well-designed. The framework is a thin layer on top. Pick the thin layer that matches the rest of your stack.

The QuantOracle API (the one I used for these examples) is at quantoracle.dev — free tier of 1,000 calls per IP per day, no signup. All three integration paths are documented at the repo.

The AgentKit action provider files are here. The Python LangChain toolkit is pip install langchain-quantoracle. The OpenAPI spec for the direct HTTP path is at api.quantoracle.dev/openapi.json.

What integration pattern are you using for your agent? Always curious how others land on the trade-off.

How to Give Your LangChain Agent Reliable Quant Finance Math (in 10 minutes)

quantoracledev — Mon, 20 Apr 2026 14:12:18 +0000

TL;DR

Large language models are great at reasoning about finance and noticeably unreliable at doing finance math. Ask an LLM to price an option and the price it returns can drift across runs. Ask for the Greeks and the higher-order ones (vanna, charm, speed) frequently come back wrong or inconsistent.

This is a known failure mode — and it's not specific to any particular model. The fix is standard engineering: call a dedicated calculator. This post walks through how to give any LangChain agent access to 73 deterministic quantitative finance endpoints (options pricing, Greeks, risk metrics, portfolio optimization, Monte Carlo, backtests, etc.) via one line of code. First 1,000 calls/day are free — no signup.

The problem in 30 seconds

# What you hope happens
response = llm.invoke("Price a European call: spot=100, strike=105, 6 months, 20% vol, 5% rate")
# "$4.58" ✓

# What actually happens in production
# - Price may land close, but drifts run-to-run
# - Delta, gamma, vega often reasonable; vanna, charm, speed, color frequently wrong
# - Numerics that depend on chained reasoning (IV solver, barrier options, path-dependent) degrade further

The math is deterministic. The model isn't. For anything agent-driven — backtests, risk management, paper trading, analysis pipelines — you need same-input-same-output calculations.

The fix: QuantOracle

QuantOracle is a REST API with 63 pure quant calculators plus 10 "composite" workflows (strategy backtests, portfolio rebalance plans, options strategy optimizers, hedging recommendations, full risk tearsheets). All citation-verified against Hull, Wilmott, Bailey & Lopez de Prado.

1,000 free calls/IP/day, no API key
Paid tier uses x402 micropayments in USDC on Base or Solana ($0.002–$0.10/call)
Deterministic: same inputs always produce the same outputs
MCP server, LangChain toolkit, OpenAI GPT, and plain REST all supported

Hook it into LangChain in one line

pip install langchain-quantoracle langchain-openai langchain

from langchain_quantoracle import QuantOracleToolkit
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate

# Load every QuantOracle tool — all 73 endpoints become LangChain tools
tools = QuantOracleToolkit().get_tools()

llm = ChatOpenAI(model="gpt-4o")
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a quant analyst. Use QuantOracle tools for all financial math — never compute in-context."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

agent = create_tool_calling_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

That's it. Your agent now has Black-Scholes, 22 portfolio risk metrics, Kelly sizing, 13 technical indicators, Monte Carlo, strategy backtests, and 60+ others.

Example 1: Price an option with Greeks

result = executor.invoke({
    "input": "Price a European call with spot=100, strike=105, 6 months to expiry, "
             "20% annualized vol, 5% risk-free. I want the price, delta, gamma, vega, and theta."
})

The agent picks the right tool (options_price), calls it, and returns:

Price: $4.58
Greeks:
  Delta: 0.4612
  Gamma: 0.0281
  Theta: -0.0211 (daily)
  Vega:  0.2808

These are the exact Black-Scholes values. Reproducible across runs.

Example 2: Full risk analysis from a returns series

result = executor.invoke({
    "input": "Here are daily returns: [0.01, -0.02, 0.03, 0.005, -0.01, 0.02, -0.015, 0.025, "
             "0.01, -0.005, 0.015]. Give me a complete risk breakdown."
})

The agent calls the risk_full-analysis composite (one API call that replaces 7 individual ones) and returns:

Risk Tearsheet (11 periods):
  Sharpe: 2.83
  Sortino: 4.59
  VaR (95%): -0.03
  Max Drawdown: -0.03
  Kelly leverage: 10.65x
  Hurst: 0.50 (neutral — random walk)
  CAGR: 122.98%

Same inputs always produce the same output. No drift, no hallucinations, no flaky Sharpe calculations.

Example 3: Backtest a strategy

result = executor.invoke({
    "input": "Backtest a 20/50 SMA crossover on this price series: "
             "[100, 101, 102, ...]. Initial capital $10000, 5 bps commission."
})

The agent calls backtest_strategy (a composite endpoint that replaces ~10 individual calls) and gets back: Sharpe ratio, Calmar, max drawdown, win rate, list of trades, equity curve, and a buy-and-hold benchmark comparison.

When to use composites vs individual calculators vs batch

The toolkit exposes three tiers of tools, each for a different situation:

Individual calculators (options_price, risk_portfolio, stats_hurst-exponent, ...) — fine-grained control, one concept per call. Free tier.
Composite workflows (backtest_strategy, portfolio_rebalance-plan, options_strategy-optimizer, hedging_recommend, risk_full-analysis, ...) — bundle 5–15 calculator calls into one round trip with a purpose-built output. Paid-only ($0.015–$0.10 each), but dramatically cheaper and faster than hand-chaining the pieces.
Batch endpoint (POST /v1/batch) — run up to 100 arbitrary calculator calls in a single HTTP request. Ideal for parameter sweeps, walk-forward backtests, or any workload where latency dominates cost. Price is the sum of the individual prices — no markup. First batch call per IP is free; subsequent batches are paid via x402.

Rule of thumb:

One calculation → individual calculator
Named workflow (risk analysis, backtest, hedge selection) → composite
Many small calculations at once → batch

A single backtest run that would be 200 HTTP calls one at a time becomes 2 batch calls. If your agent iterates, batch usually wins on both latency and cost.

Filter by category to keep tool lists small

A common LangChain pitfall: 73 tools in the prompt confuses smaller models. Filter by category:

# Options-only agent
tools = QuantOracleToolkit(categories=["options", "derivatives"]).get_tools()

# Risk/portfolio-only agent
tools = QuantOracleToolkit(categories=["risk", "portfolio", "stats"]).get_tools()

# Crypto-focused agent
tools = QuantOracleToolkit(categories=["crypto", "simulate"]).get_tools()

Available categories: options, derivatives, risk, indicators, simulate, portfolio, fixed-income, fi, stats, crypto, fx, macro, tvm, trade, pairs, backtest, hedging.

Past the free tier

After 1,000 calls/day (per IP), the API returns HTTP 402 with an x402 payment requirements header. If you're using an x402-capable HTTP client (e.g. AgentCash, Coinbase AgentKit), payments are automatic — USDC on Base or Solana, $0.002–$0.10 per call. Otherwise the toolkit raises an exception and you can add a payment layer yourself.

Why this matters for agentic systems

When an agent makes 50 tool calls during a backtest, every calculation has to be right. An LLM that's 85% accurate on Black-Scholes doesn't produce a backtest — it produces noise. Moving all math to a deterministic calculator means:

Reproducible results (your next run produces the same Sharpe)
Cacheable (you can memoize by input hash)
Auditable (you can replay any step)
Fast (sub-millisecond per calculation on the server)
Cheap (orders of magnitude less than equivalent LLM tokens)

This pattern — LLM for reasoning + deterministic APIs for compute — is the one thing that actually works for production agent systems. Pick it up now and you don't have to rebuild once your agent starts taking real actions.

Links

API docs: https://api.quantoracle.dev/docs
Tool discovery: https://api.quantoracle.dev/tools
x402 discovery (Base + Solana): https://api.quantoracle.dev/.well-known/x402
GitHub: https://github.com/QuantOracledev/quantoracle
Pypi: langchain-quantoracle
MCP server: npx quantoracle-mcp (npm)
OpenAI GPT: https://chatgpt.com/g/g-69d9c28bddb481918e674e2f9d9f3e97-quantoracle

Free tier is generous, no signup required, MIT licensed. If you're building an agent that touches financial math — options pricing, portfolio analytics, risk, backtests — try it before rolling your own.

DEV Community: quantoracledev

Detecting a website's tech stack: why grepping for framework names gets you wrong answers

What actually identifies a stack

Where static fetching genuinely runs out

The DIY version

The one-call version

The underrated thing

The App Store review ceiling nobody mentions — and how to get 8x past it

Each country is its own feed

The more interesting output

Doing it yourself

Or one call

What this can't do

Scrape any company's job postings — Greenhouse, Lever & Ashby, with one API call

The raw APIs

The one-call version

The underrated part: some boards publish full comp

Things worth building with it

Why the API route beats HTML scraping here

How to check Core Web Vitals for hundreds of URLs at once (API + code)

What we're measuring

The bulk approach

Make it a performance budget in CI

Auditing a whole site on a schedule

SEO, not just speed?

Cost

Turn any website into clean Markdown for your RAG pipeline (with code)

Why not just requests + BeautifulSoup?

The hosted approach

One page

A whole documentation site

Wiring it into a RAG pipeline

Don't forget PDFs and Word docs

Keep the index fresh

Cost

AgentKit vs LangChain vs Direct HTTP — picking the right integration for paid agent APIs

The test case

Pattern 1 — Direct HTTP

Pattern 2 — Coinbase AgentKit (TypeScript)

Pattern 3 — LangChain (Python)

The decision rule

Two patterns from production that combine these

Backtest in Python, deploy in TypeScript

Free tier for research, paid for production

What this isn't about

My pick — and why

How to Give Your LangChain Agent Reliable Quant Finance Math (in 10 minutes)

TL;DR

The problem in 30 seconds

The fix: QuantOracle

Hook it into LangChain in one line

Example 1: Price an option with Greeks

Example 2: Full risk analysis from a returns series

Example 3: Backtest a strategy

When to use composites vs individual calculators vs batch

Filter by category to keep tool lists small

Past the free tier

Why this matters for agentic systems

Links

Why not just `requests` + BeautifulSoup?