DEV Community

clause-netizen
clause-netizen

Posted on

Detect Any Website's Tech Stack From a Server, No Headless Browser Required

#javascript #webdev #api #tutorial

Wappalyzer's browser extension tells you a site runs Cloudflare and React. That's fine when you're looking at one site in a tab. It falls apart when you have 4,000 domains in a CSV and you want to know which ones run Shopify, or which prospects already have a chat widget installed.

Puppeteer solves it, technically. Then you're maintaining a Chromium install, a memory ceiling, and a job queue for what amounts to reading HTML headers. SiteIntel does the fetch and the fingerprinting server-side and hands you JSON.

One request 

curl --get 'https://siteintel.p.rapidapi.com/v1/analyze' \
  --data-urlencode 'url=https://stripe.com' \
  -H 'X-RapidAPI-Key: YOUR_KEY' \
  -H 'X-RapidAPI-Host: siteintel.p.rapidapi.com'
Enter fullscreen mode Exit fullscreen mode

The url param takes a full URL with the scheme. stripe.com won't work, https://stripe.com will. I lost twenty minutes to this before adding a normalizer on my side.

Response:

{
  "query": "https://stripe.com",
  "final_url": "https://stripe.com/",
  "status": 200,
  "fetched_at": "2026-07-28T14:02:11Z",
  "title": "Stripe | Financial Infrastructure to Grow Your Revenue",
  "description": "Stripe powers online and in-person payment processing...",
  "canonical": "https://stripe.com/",
  "lang": "en",
  "favicon": "https://stripe.com/favicon.ico",
  "open_graph": {
    "title": "Stripe | Financial Infrastructure",
    "image": "https://images.stripeassets.com/og.png",
    "site_name": "Stripe",
    "type": "website"
  },
  "detected_tech": ["Cloudflare", "React"],
  "social_links": ["https://twitter.com/stripe", "https://github.com/stripe"],
  "emails": [],
  "server": "nginx"
}
Enter fullscreen mode Exit fullscreen mode

Two things worth knowing about the shape. detected_tech is a flat array of strings, not objects with confidence scores, so you filter it with .includes() and move on. And final_url is the URL after redirects, while query is what you sent. Comparing those two is how you catch domains that silently park on someone else's site.

Node, no dependencies

Global fetch has been stable since Node 18, so this runs as-is:

const HEADERS = {
  'X-RapidAPI-Key': process.env.RAPIDAPI_KEY,
  'X-RapidAPI-Host': 'siteintel.p.rapidapi.com'
};

async function analyze(url) {
  const endpoint = new URL('https://siteintel.p.rapidapi.com/v1/analyze');
  endpoint.searchParams.set('url', url);

  const res = await fetch(endpoint, { headers: HEADERS });
  if (!res.ok) throw new Error(`${url} -> ${res.status}`);
  return res.json();
}

const sites = [
  'https://stripe.com',
  'https://basecamp.com',
  'https://gitlab.com'
];

for (const site of sites) {
  const data = await analyze(site);
  console.log(
    data.final_url.padEnd(32),
    (data.detected_tech ?? []).join(', ') || '(nothing detected)'
  );
}
Enter fullscreen mode Exit fullscreen mode

Note the ?? [] on detected_tech. Sites behind aggressive bot protection return a valid response with very little in it, and a bare .join() on undefined will take down your whole loop on domain 900 of 4,000. Same defensive treatment applies to social_links and emails.

Run the loop serially like that and you'll be waiting a while. In production I run a small concurrency pool of four or five and let failures collect in an array rather than throwing, because one dead domain shouldn't kill the batch.

What I actually built with it

A pre-sales filter. I had a list of ecommerce domains and no interest in pitching the ones already running a competitor's product. The rule ended up being two lines:

const stack = new Set(data.detected_tech ?? []);
const disqualified = stack.has('Shopify') || stack.has('Intercom');
Enter fullscreen mode Exit fullscreen mode

That knocked out roughly a third of the list before anyone wrote a single email. social_links fed a second pass for finding the founder's Twitter, and emails occasionally surfaces a real contact address straight off the homepage, though most sites keep those behind a form.

The same data shape works for a competitive dashboard, tracking when a set of domains migrates off one CDN onto another. You store detected_tech per domain per week and diff it. Migrations show up as a set difference, and you know before the press release does.

Two other endpoints

Same auth, same url param, both useful in the same batch pass:

# on-page SEO report
curl --get 'https://siteintel.p.rapidapi.com/v1/seo-audit' \
  --data-urlencode 'url=https://stripe.com' \
  -H 'X-RapidAPI-Key: YOUR_KEY' \
  -H 'X-RapidAPI-Host: siteintel.p.rapidapi.com'

# screenshot
curl --get 'https://siteintel.p.rapidapi.com/v1/screenshot' \
  --data-urlencode 'url=https://stripe.com' \
  -H 'X-RapidAPI-Key: YOUR_KEY' \
  -H 'X-RapidAPI-Host: siteintel.p.rapidapi.com'
Enter fullscreen mode Exit fullscreen mode

The screenshot endpoint is the one place a browser actually runs, and the point is that it runs on the API's infrastructure instead of yours.

Fair warning on detection

Fingerprinting from HTML and headers catches what a site exposes. Server-rendered frameworks hidden behind a CDN, or anything that only loads after user interaction, won't always show up. If your workflow requires certainty about a single specific site, open it in a browser. If you need a good-enough signal across thousands of them, this is the tradeoff you want.

Working examples and the batch runner: github.com/clause-netizen/siteintel-api. Managed keys on RapidAPI if you'd rather not host anything.

Top comments (0)