DEV Community

Алексей Спинов
Алексей Спинов

Posted on

How I Built a 9-Tool SEO Audit Suite Without a Single Browser Automation

Every tool runs on Node.js without Playwright or Puppeteer. Here's how:

The Tools

  1. SEO Audit — CheerioCrawler checks 50+ factors
  2. Broken Links — CheerioCrawler follows internal links
  3. PageSpeed — Google's free PSI API (no key)
  4. SSL Checker — Node.js https module reads certs directly
  5. Headers Checker — HTTP HEAD request + 10-header checklist
  6. Tech Detector — Regex on HTML source + response headers
  7. Sitemap Parser — cheerio with xmlMode
  8. Robots.txt — fetch + text parsing
  9. Keyword Density — word frequency + bigram analysis

Why No Browser?

For SEO audits, you're analyzing HTML structure — not rendering JavaScript. Cheerio (HTML parser) uses 50MB RAM vs 500MB for Playwright. Runs 10x faster.

The only tool that could benefit from a browser is PageSpeed — but Google already does the rendering via their API.

The Robots.txt Analyzer Is My Favorite

It detects which AI bots a site blocks:

{
  "blockedBots": {
    "GPTBot": true,
    "ClaudeBot": false,
    "CCBot": true,
    "Googlebot": false
  },
  "aiBotsBlocked": true
}
Enter fullscreen mode Exit fullscreen mode

With AI crawling becoming a hot topic, this data is surprisingly useful.

All 9 tools free on Apify Store — search knotless_cadence.

Top comments (0)