<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Kabeer Ahmed</title>
    <description>The latest articles on DEV Community by Kabeer Ahmed (@kabeer099).</description>
    <link>https://dev.to/kabeer099</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4206996%2Fa437acef-6af6-4750-b6d6-2d0636c8d0e0.png</url>
      <title>DEV Community: Kabeer Ahmed</title>
      <link>https://dev.to/kabeer099</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/kabeer099"/>
    <language>en</language>
    <item>
      <title>How We Built 8 Cloud Web Scrapers (RAG Markdown, Shopify, B2B Leads) &amp; 12 Zero-Bloat WordPress Plugins</title>
      <dc:creator>Kabeer Ahmed</dc:creator>
      <pubDate>Wed, 11 Nov 2026 16:40:50 +0000</pubDate>
      <link>https://dev.to/kabeer099/how-we-built-8-cloud-web-scrapers-rag-markdown-shopify-b2b-leads-12-zero-bloat-wordpress-2h03</link>
      <guid>https://dev.to/kabeer099/how-we-built-8-cloud-web-scrapers-rag-markdown-shopify-b2b-leads-12-zero-bloat-wordpress-2h03</guid>
      <description>&lt;p&gt;Over the past few months, I packaged the exact internal tools, cloud scrapers, and zero-bloat WordPress/WooCommerce plugins I use for production client builds into two places:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;8 Hosted Cloud Scrapers on Apify Store (&lt;code&gt;$2.00 / 1,000 results&lt;/code&gt;):&lt;/strong&gt; &lt;a href="https://apify.com/kabeer099" rel="noopener noreferrer"&gt;https://apify.com/kabeer099&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Full Unminified Source Code &amp;amp; White-Label Agency Bundles (&lt;code&gt;$19 - $39&lt;/code&gt;):&lt;/strong&gt; &lt;a href="https://kabeer.site/services.html#digital-store" rel="noopener noreferrer"&gt;https://kabeer.site/services.html#digital-store&lt;/a&gt;
&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  What's Inside the 8 Cloud Scrapers (&lt;code&gt;Node.js&lt;/code&gt; + &lt;code&gt;Crawlee&lt;/code&gt;)
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;AI Website to Clean Markdown &amp;amp; RAG Chunk Extractor:&lt;/strong&gt; Converts documentation and websites into clean Markdown + token-aware chunks (&lt;code&gt;~300 words&lt;/code&gt;) for OpenAI, Claude, LangChain, and Pinecone pipelines (&lt;a href="https://apify.com/kabeer099/ai-website-to-markdown-rag-scraper" rel="noopener noreferrer"&gt;Run on Apify&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B2B Website Contact, Email, WhatsApp &amp;amp; Social Scraper:&lt;/strong&gt; Extracts verified emails, phone numbers, WhatsApp links, and social profiles (&lt;a href="https://apify.com/kabeer099/b2b-contact-email-social-scraper" rel="noopener noreferrer"&gt;Run on Apify&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Shopify &amp;amp; WooCommerce Product, Price &amp;amp; Contact Scrapers:&lt;/strong&gt; Extracts full product catalogs, variants, SKUs, and merchant contacts (&lt;a href="https://apify.com/kabeer099/shopify-product-price-contact-scraper" rel="noopener noreferrer"&gt;Shopify Scraper&lt;/a&gt; / &lt;a href="https://apify.com/kabeer099/woocommerce-product-price-contact-scraper" rel="noopener noreferrer"&gt;WooCommerce Scraper&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Website Tech Stack, Pixel &amp;amp; Ad Tag Detector:&lt;/strong&gt; Detects 45+ CMS, frameworks, analytics, and ad pixels (&lt;a href="https://apify.com/kabeer099/website-tech-stack-pixel-detector" rel="noopener noreferrer"&gt;Run on Apify&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Technical SEO Audit, Broken Link &amp;amp; JSON-LD Analyzer:&lt;/strong&gt; Audits titles, headings, OpenGraph, and schema.org JSON-LD (&lt;a href="https://apify.com/kabeer099/seo-audit-broken-link-meta-analyzer" rel="noopener noreferrer"&gt;Run on Apify&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  12 Zero-Bloat WordPress &amp;amp; WooCommerce Plugins
&lt;/h3&gt;

&lt;p&gt;All 12 individual plugins are free to inspect and download at &lt;strong&gt;&lt;a href="https://kabeer.site/plugins.html" rel="noopener noreferrer"&gt;kabeer.site/plugins.html&lt;/a&gt;&lt;/strong&gt;, and agencies can grab the &lt;strong&gt;12-in-1 White-Label Agency Pack&lt;/strong&gt; or the &lt;strong&gt;8-in-1 Node.js Scraper Source Code Suite&lt;/strong&gt; directly at &lt;strong&gt;&lt;a href="https://kabeer.site/services.html#digital-store" rel="noopener noreferrer"&gt;kabeer.site/services.html#digital-store&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Also check out our &lt;strong&gt;5-in-1 Multi-Industry Live Client Booking Portals&lt;/strong&gt; at &lt;strong&gt;&lt;a href="https://kabeer.site/demos/" rel="noopener noreferrer"&gt;kabeer.site/demos/&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>javascript</category>
      <category>wordpress</category>
      <category>opensource</category>
    </item>
    <item>
      <title>I Built &amp; Open-Sourced 12 Lightweight WordPress/WooCommerce Plugins (&lt;15KB) + 6 Browser-Based AI &amp; PDF Tools</title>
      <dc:creator>Kabeer Ahmed</dc:creator>
      <pubDate>Tue, 03 Nov 2026 09:18:52 +0000</pubDate>
      <link>https://dev.to/kabeer099/i-built-open-sourced-12-lightweight-wordpresswoocommerce-plugins-15kb-6-browser-based-ai--2h5j</link>
      <guid>https://dev.to/kabeer099/i-built-open-sourced-12-lightweight-wordpresswoocommerce-plugins-15kb-6-browser-based-ai--2h5j</guid>
      <description>&lt;p&gt;Most WordPress and WooCommerce site owners end up installing 20+ heavy plugins—or paying $49 to $99/year per plugin—just to handle basic performance optimization, WhatsApp ordering, Cash-on-Delivery (COD) fraud prevention, or JSON-LD SEO schema.&lt;/p&gt;

&lt;p&gt;Worse, many "free" plugins inject bloated JavaScript frameworks, admin upsell banners, and external telemetry that slow down &lt;code&gt;wp-admin&lt;/code&gt; and hurt &lt;strong&gt;Core Web Vitals (LCP, INP, CLS)&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;To solve this for my own client projects and the wider developer community, I engineered and open-sourced &lt;strong&gt;12 ultra-lightweight (&amp;lt;15 KB each), zero-telemetry, GPLv2-licensed WordPress &amp;amp; WooCommerce plugins&lt;/strong&gt; plus &lt;strong&gt;6 client-side WebAssembly AI &amp;amp; PDF utilities&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;🔗 &lt;strong&gt;Live WordPress Plugins Vault (.ZIP Downloads + Readme + PHP Source Viewer):&lt;/strong&gt;&lt;br&gt;
👉 &lt;strong&gt;&lt;a href="https://kabeer.site/plugins.html" rel="noopener noreferrer"&gt;https://kabeer.site/plugins.html&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;🪄 &lt;strong&gt;Live Browser-Based AI &amp;amp; PDF Studio Tools:&lt;/strong&gt;&lt;br&gt;
👉 &lt;strong&gt;&lt;a href="https://kabeer.site/tools.html" rel="noopener noreferrer"&gt;https://kabeer.site/tools.html&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  🔌 The 12 Open-Source WordPress &amp;amp; WooCommerce Plugins
&lt;/h2&gt;

&lt;p&gt;Every &lt;code&gt;.zip&lt;/code&gt; archive includes the main PHP plugin file (protected with &lt;code&gt;ABSPATH&lt;/code&gt; guards, WordPress Settings API nonces, &lt;code&gt;current_user_can('manage_options')&lt;/code&gt;, and strict sanitization), a standard &lt;code&gt;readme.txt&lt;/code&gt;, &lt;code&gt;index.php&lt;/code&gt; directory protection, and a clean &lt;code&gt;uninstall.php&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. ⚡ Kabeer SpeedBoost Pro (v2.1) — &lt;em&gt;Free Alternative to WP Rocket &amp;amp; Perfmatters&lt;/em&gt;
&lt;/h3&gt;

&lt;p&gt;Removes WordPress core bloat (Emoji scripts/styles, oEmbeds, &lt;code&gt;wp_generator&lt;/code&gt;, RSD/WLW links), unloads 36 KB &lt;code&gt;dashicons.min.css&lt;/code&gt; for logged-out visitors, strips &lt;code&gt;?ver=&lt;/code&gt; query strings for better Cloudflare cache hit rates, disables XML-RPC, and injects a &lt;strong&gt;65ms Instant Hover Link Prefetcher&lt;/strong&gt;.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/kabeer-speedboost-pro.zip" rel="noopener noreferrer"&gt;kabeer-speedboost-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  2. 💬 WooChat WhatsApp Order &amp;amp; Floating Chat Pro (v2.0) — &lt;em&gt;Free Alternative to Join.chat Pro&lt;/em&gt;
&lt;/h3&gt;

&lt;p&gt;Adds a customizable site-wide floating WhatsApp button and hooks into &lt;code&gt;woocommerce_after_add_to_cart_button&lt;/code&gt; to place a direct &lt;strong&gt;"Order via WhatsApp"&lt;/strong&gt; button on WooCommerce product pages with the product title, price, and permalink pre-filled.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/woochat-whatsapp-pro.zip" rel="noopener noreferrer"&gt;woochat-whatsapp-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  3. 🛡️ WooGuard COD Fake Order Shield &amp;amp; Phone Validator (v1.5) — &lt;em&gt;Free Alternative to WooCommerce Anti-Fraud&lt;/em&gt;
&lt;/h3&gt;

&lt;p&gt;Built for Cash-on-Delivery (COD) e-commerce stores. Validates mobile phone digit length at checkout, blocks repeated fake digit strings (e.g., &lt;code&gt;00000000000&lt;/code&gt;), and enforces a configurable maximum COD cart total cap.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/wooguard-cod-shield.zip" rel="noopener noreferrer"&gt;wooguard-cod-shield.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  4. 🖼️ Auto WebP &amp;amp; KB Image Optimizer (v1.8) — &lt;em&gt;Free Unlimited Alternative to Imagify &amp;amp; Smush Pro&lt;/em&gt;
&lt;/h3&gt;

&lt;p&gt;Uses your server's native PHP GD/Imagick engine via &lt;code&gt;wp_upload_image_mime_transforms&lt;/code&gt; and &lt;code&gt;big_image_size_threshold&lt;/code&gt; to automatically convert uploaded JPEG/PNG sub-sizes to WebP and scale down oversized 4000px+ photos with &lt;strong&gt;zero monthly credit limits&lt;/strong&gt;.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/kabeer-webp-optimizer.zip" rel="noopener noreferrer"&gt;kabeer-webp-optimizer.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  5. 🔍 RankSchema AI &amp;amp; Auto JSON-LD SEO Pro (v2.2)
&lt;/h3&gt;

&lt;p&gt;Outputs clean &lt;code&gt;Schema.org&lt;/code&gt; structured data (&lt;code&gt;Article&lt;/code&gt;, &lt;code&gt;Product&lt;/code&gt;, &lt;code&gt;WebPage&lt;/code&gt;, and &lt;code&gt;Organization&lt;/code&gt;) in &lt;code&gt;wp_head&lt;/code&gt; without heavy metabox bloat.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/rankschema-seo-pro.zip" rel="noopener noreferrer"&gt;rankschema-seo-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  6. 📌 WP CodeVault — AdSense, Meta Pixel &amp;amp; Header/Footer Manager (v1.6)
&lt;/h3&gt;

&lt;p&gt;Safely inject your Google AdSense Publisher ID (&lt;code&gt;ca-pub-...&lt;/code&gt;), Meta Pixel, TikTok Pixel, GA4, and custom &lt;code&gt;&amp;lt;head&amp;gt;&lt;/code&gt; / &lt;code&gt;&amp;lt;footer&amp;gt;&lt;/code&gt; scripts without editing theme files.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/wp-codevault-pro.zip" rel="noopener noreferrer"&gt;wp-codevault-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  7. 🔒 WP IronShield Security &amp;amp; Login Guard (v2.4)
&lt;/h3&gt;

&lt;p&gt;Brute-force login limiter (locks out offending IPs for 15 minutes after 5 failed attempts using anonymized MD5 transients), disables XML-RPC DDoS vectors, blocks &lt;code&gt;/?author=1&lt;/code&gt; username enumeration scans, and sends &lt;code&gt;X-Frame-Options: SAMEORIGIN&lt;/code&gt; &amp;amp; &lt;code&gt;X-Content-Type-Options: nosniff&lt;/code&gt; headers.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/wp-ironshield-security.zip" rel="noopener noreferrer"&gt;wp-ironshield-security.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  8. 🛒 WooCheckout Turbo — Direct Checkout &amp;amp; COD Field Simplifier (v1.9)
&lt;/h3&gt;

&lt;p&gt;Redirects "Add to Cart" directly to Checkout and removes unnecessary checkout fields (Company Name, Address Line 2, optional Postcode) to reduce cart abandonment.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/woocheckout-turbo-pro.zip" rel="noopener noreferrer"&gt;woocheckout-turbo-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  9. 📑 WP Instant Table of Contents &amp;amp; Reading Time Pro (v1.4)
&lt;/h3&gt;

&lt;p&gt;Automatically parses &lt;code&gt;&amp;lt;h2...&amp;gt;&lt;/code&gt; tags on singular posts to generate an SEO-friendly Table of Contents and estimated reading time badge.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/wp-instant-toc-pro.zip" rel="noopener noreferrer"&gt;wp-instant-toc-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  10. 🔗 WP Auto-Internal Linker &amp;amp; Affiliate Manager Pro (v1.7)
&lt;/h3&gt;

&lt;p&gt;Automatically converts target keywords across blog posts into internal or affiliate links using safe regex matching outside existing HTML tags.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/wp-autolinker-seo-pro.zip" rel="noopener noreferrer"&gt;wp-autolinker-seo-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  11. 🧹 WP Database Cleaner &amp;amp; Auto-Revision Cap Pro (v1.5)
&lt;/h3&gt;

&lt;p&gt;One-click admin utility (protected with &lt;code&gt;check_admin_referer&lt;/code&gt;) to purge post revisions, auto-drafts, spam comments, and expired transients, while capping future revisions at 5 per post.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/wp-db-cleaner-pro.zip" rel="noopener noreferrer"&gt;wp-db-cleaner-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  12. 🎨 WP WhiteLabel &amp;amp; Client Dashboard Customizer Pro (v1.6)
&lt;/h3&gt;

&lt;p&gt;Adds a custom Agency Support widget to &lt;code&gt;wp-admin&lt;/code&gt;, declutters default dashboard metaboxes, and customizes the admin footer branding for client handoffs.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download:&lt;/strong&gt; &lt;a href="https://kabeer.site/plugins/wp-whitelabel-agency-pro.zip" rel="noopener noreferrer"&gt;wp-whitelabel-agency-pro.zip&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🪄 6 Client-Side AI &amp;amp; PDF Studio Tools (&lt;code&gt;kabeer.site/tools.html&lt;/code&gt;)
&lt;/h2&gt;

&lt;p&gt;Alongside the WordPress plugins, I also built 6 browser-based utilities powered by WebAssembly, HTML5 Canvas, &lt;code&gt;pdf-lib&lt;/code&gt;, &lt;code&gt;PDF.js&lt;/code&gt;, and &lt;code&gt;Tesseract.js&lt;/code&gt; (100% client-side privacy—files never leave the browser):&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;AI Background Remover &amp;amp; Passport Photo Studio&lt;/strong&gt; (&lt;code&gt;@imgly/background-removal&lt;/code&gt; WASM + Smart Edge Cutout + Transparent PNG, Passport Blue &amp;amp; Pure White backdrops)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;5-in-1 Complete PDF Studio&lt;/strong&gt; (Merge PDFs, Split/Extract Pages, PDF to JPG Images, Photos to PDF, and Diagonal Watermark)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AI OCR Image-to-Text Scanner&lt;/strong&gt; (&lt;code&gt;Tesseract.js&lt;/code&gt; Neural OCR)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CNIC / ID Card Front &amp;amp; Back Single A4 PDF Scanner&lt;/strong&gt; (with B&amp;amp;W Photocopy Filter &amp;amp; anti-misuse watermark)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Guaranteed Target-KB Image Compressor (&lt;code&gt;&amp;lt;20KB&lt;/code&gt;, &lt;code&gt;&amp;lt;50KB&lt;/code&gt;, &lt;code&gt;&amp;lt;100KB&lt;/code&gt;)&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Vector PDF Invoice, Quotation &amp;amp; Receipt Builder&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  🔍 Inspect the Code &amp;amp; Share Feedback
&lt;/h2&gt;

&lt;p&gt;You can inspect the &lt;code&gt;readme.txt&lt;/code&gt;, step-by-step installation guide, and PHP source code for every plugin directly in your browser at &lt;strong&gt;&lt;a href="https://kabeer.site/plugins.html" rel="noopener noreferrer"&gt;https://kabeer.site/plugins.html&lt;/a&gt;&lt;/strong&gt;, or test the AI/PDF tools at &lt;strong&gt;&lt;a href="https://kabeer.site/tools.html" rel="noopener noreferrer"&gt;https://kabeer.site/tools.html&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Have a feature request or building a custom Laravel, React, or WooCommerce platform? Connect with me at &lt;strong&gt;&lt;a href="https://kabeer.site/" rel="noopener noreferrer"&gt;kabeer.site&lt;/a&gt;&lt;/strong&gt; or on WhatsApp (&lt;code&gt;+92 334 2978455&lt;/code&gt;)!&lt;/p&gt;

</description>
      <category>wordpress</category>
      <category>php</category>
      <category>webdev</category>
      <category>opensource</category>
    </item>
    <item>
      <title>How I Built a Company Career &amp; ATS Hiring Signal Scraper + AI Markdown RAG Extractor in Node.js</title>
      <dc:creator>Kabeer Ahmed</dc:creator>
      <pubDate>Mon, 26 Oct 2026 16:43:57 +0000</pubDate>
      <link>https://dev.to/kabeer099/how-i-built-a-company-career-ats-hiring-signal-scraper-ai-markdown-rag-extractor-in-nodejs-25hm</link>
      <guid>https://dev.to/kabeer099/how-i-built-a-company-career-ats-hiring-signal-scraper-ai-markdown-rag-extractor-in-nodejs-25hm</guid>
      <description>&lt;p&gt;When building LLM-powered RAG pipelines or B2B outbound intelligence systems, two data bottlenecks come up repeatedly:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Raw HTML is terrible for LLM context windows&lt;/strong&gt; — navigation menus, cookie banners, inline scripts, and SVG icons burn 70–85% of your tokens before the model even sees the actual content.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Static company directories don't tell you who has active budget&lt;/strong&gt; — knowing whether a company is actively hiring on &lt;strong&gt;Greenhouse&lt;/strong&gt;, &lt;strong&gt;Lever&lt;/strong&gt;, &lt;strong&gt;Ashby&lt;/strong&gt;, or &lt;strong&gt;Workable&lt;/strong&gt; (and what engineering/product roles are open right now) is one of the strongest B2B buying signals available.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;As part of my suite of &lt;strong&gt;&lt;a href="https://apify.com/kabeer099" rel="noopener noreferrer"&gt;8 production Node.js scrapers on the Apify Store&lt;/a&gt;&lt;/strong&gt;, I built two focused actors to solve both problems using fast HTTP/Cheerio pipelines in Node.js:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/ai-website-to-markdown-rag-scraper" rel="noopener noreferrer"&gt;AI Website to Clean Markdown &amp;amp; RAG Chunk Extractor&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/company-career-hiring-signal-scraper" rel="noopener noreferrer"&gt;Company Career Page, ATS &amp;amp; Hiring Signal B2B Lead Scraper&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Here is a practical walkthrough of the architecture and implementation behind both tools.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 1: Stripping DOM Noise &amp;amp; Chunking Semantic Markdown for RAG
&lt;/h2&gt;

&lt;p&gt;Most naive web-to-Markdown scrapers run &lt;code&gt;turndown&lt;/code&gt; or &lt;code&gt;innerText&lt;/code&gt; across &lt;code&gt;document.body&lt;/code&gt;. That leaves mega-menus, footer sitemaps, cookie consent modals, and hydration blobs inside your vector database embeddings.&lt;/p&gt;

&lt;p&gt;In &lt;strong&gt;&lt;a href="https://apify.com/kabeer099/ai-website-to-markdown-rag-scraper" rel="noopener noreferrer"&gt;AI Website to Clean Markdown &amp;amp; RAG Chunk Extractor&lt;/a&gt;&lt;/strong&gt;, the pipeline runs in three deterministic stages:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Pruning Non-Content DOM Subtrees First
&lt;/h3&gt;

&lt;p&gt;Before converting anything to Markdown, we strip noisy tags and common boilerplate selectors while preserving semantic headings (&lt;code&gt;h1&lt;/code&gt;–&lt;code&gt;h4&lt;/code&gt;), paragraphs, lists, code blocks, and tables:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;cleanDomForMarkdown&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;$&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;noiseSelectors&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;script&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;style&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;noscript&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;iframe&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;svg&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;canvas&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;nav&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;footer&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;header&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;aside&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;[role="navigation"]&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;[role="banner"]&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;[role="contentinfo"]&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;.cookie-banner&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;#cookie-consent&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;.newsletter-popup&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;.ads&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
    &lt;span class="p"&gt;];&lt;/span&gt;
    &lt;span class="nf"&gt;$&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;noiseSelectors&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;, &lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="nf"&gt;remove&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;root&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;$&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;main, article, [role="main"]&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;first&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;root&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="nx"&gt;root&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;$&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;body&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  2. Heading-Aware Token Chunking (~300 Words / ~400 Tokens)
&lt;/h3&gt;

&lt;p&gt;Splitting Markdown by fixed character counts often slices tables or code blocks in half and loses the parent section heading. Instead, we chunk on paragraph boundaries while prepending the active &lt;code&gt;# / ##&lt;/code&gt; heading hierarchy to each chunk so every vector embedding retains its semantic context:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;chunkMarkdownWithHeadings&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;markdown&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;maxWords&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;blocks&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;markdown&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\n{2,}&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;chunks&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;currentHeading&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;''&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;buffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;wordCount&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;block&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;blocks&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;trimmed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;block&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;trimmed&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;continue&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/^#&lt;/span&gt;&lt;span class="se"&gt;{1,3}\s&lt;/span&gt;&lt;span class="sr"&gt;+/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;trimmed&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nx"&gt;currentHeading&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;trimmed&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;

        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;words&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;trimmed&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\s&lt;/span&gt;&lt;span class="sr"&gt;+/&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;wordCount&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nx"&gt;words&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;maxWords&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nx"&gt;chunks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
                &lt;span class="na"&gt;headingContext&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;currentHeading&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="na"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
                &lt;span class="nx"&gt;wordCount&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="na"&gt;estimatedTokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ceil&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;wordCount&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mf"&gt;1.33&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="p"&gt;});&lt;/span&gt;
            &lt;span class="nx"&gt;buffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;currentHeading&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;currentHeading&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
            &lt;span class="nx"&gt;wordCount&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;currentHeading&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="nx"&gt;currentHeading&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\s&lt;/span&gt;&lt;span class="sr"&gt;+/&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;

        &lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;trimmed&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="nx"&gt;wordCount&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nx"&gt;words&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;chunks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
            &lt;span class="na"&gt;headingContext&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;currentHeading&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="nx"&gt;wordCount&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;estimatedTokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ceil&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;wordCount&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mf"&gt;1.33&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;chunks&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each output item includes the page metadata (title, canonical URL, OpenGraph/JSON-LD summary), full cleaned Markdown, word/token estimates, and the ready-to-embed &lt;code&gt;ragChunks&lt;/code&gt; array for OpenAI, Claude, or LangChain.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 2: Detecting 10+ ATS Platforms &amp;amp; Hiring Signals from Company Websites
&lt;/h2&gt;

&lt;p&gt;When enriching B2B leads, a company with 8 open engineering roles on &lt;strong&gt;Ashby&lt;/strong&gt; or &lt;strong&gt;Greenhouse&lt;/strong&gt; is a very different prospect from a dormant brochure site.&lt;/p&gt;

&lt;p&gt;In &lt;strong&gt;&lt;a href="https://apify.com/kabeer099/company-career-hiring-signal-scraper" rel="noopener noreferrer"&gt;Company Career Page, ATS &amp;amp; Hiring Signal B2B Lead Scraper&lt;/a&gt;&lt;/strong&gt;, we start from a company's root domain, discover its &lt;code&gt;/careers&lt;/code&gt; or &lt;code&gt;/jobs&lt;/code&gt; links, and fingerprint the underlying Applicant Tracking System (ATS):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ATS_SIGNATURES&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Greenhouse&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/boards&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;greenhouse&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;io|job-boards&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;greenhouse&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;io|greenhouse&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;io&lt;/span&gt;&lt;span class="se"&gt;\/&lt;/span&gt;&lt;span class="sr"&gt;embed/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Lever&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/jobs&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;lever&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;co|api&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;lever&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;co/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Ashby&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/jobs&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;ashbyhq&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com|ashbyhq&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Workable&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/apply&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;workable&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com|workable&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;BambooHR&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;bamboohr&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com&lt;/span&gt;&lt;span class="se"&gt;\/(&lt;/span&gt;&lt;span class="sr"&gt;careers|jobs&lt;/span&gt;&lt;span class="se"&gt;)&lt;/span&gt;&lt;span class="sr"&gt;/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;BreezyHR&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;breezy&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;hr/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Personio&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;jobs&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;personio&lt;/span&gt;&lt;span class="se"&gt;\.(&lt;/span&gt;&lt;span class="sr"&gt;de|com&lt;/span&gt;&lt;span class="se"&gt;)&lt;/span&gt;&lt;span class="sr"&gt;/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Teamtailor&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/teamtailor&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;SmartRecruiters&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/jobs&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;smartrecruiters&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com/i&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Workday&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sr"&gt;/myworkdayjobs&lt;/span&gt;&lt;span class="se"&gt;\.&lt;/span&gt;&lt;span class="sr"&gt;com/i&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;];&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;detectAtsPlatforms&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;html&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;outboundLinks&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;combined&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;html&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;&lt;span class="nx"&gt;outboundLinks&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt; &lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;ATS_SIGNATURES&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;sig&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;sig&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;combined&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;sig&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;sig&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Prioritizing Talent &amp;amp; Recruiting Contacts
&lt;/h3&gt;

&lt;p&gt;Alongside open role titles and external ATS board URLs, the actor extracts emails and prioritizes recruiting-specific inboxes (&lt;code&gt;careers@&lt;/code&gt;, &lt;code&gt;hr@&lt;/code&gt;, &lt;code&gt;talent@&lt;/code&gt;, &lt;code&gt;jobs@&lt;/code&gt;, &lt;code&gt;people@&lt;/code&gt;) ahead of generic support addresses:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;prioritizeHiringEmails&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;emails&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;priorityRegex&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sr"&gt;/^&lt;/span&gt;&lt;span class="se"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;careers|jobs|talent|hr|recruiting|people&lt;/span&gt;&lt;span class="se"&gt;)&lt;/span&gt;&lt;span class="sr"&gt;@/i&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[...&lt;/span&gt;&lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;emails&lt;/span&gt;&lt;span class="p"&gt;)].&lt;/span&gt;&lt;span class="nf"&gt;sort&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;aPri&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;priorityRegex&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;bPri&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;priorityRegex&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;aPri&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nx"&gt;bPri&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Try the Full Suite of 8 Actors &amp;amp; Browser Extensions
&lt;/h2&gt;

&lt;p&gt;Both actors are live on the Apify Store alongside 6 additional B2B, e-commerce, and technical SEO scrapers:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/ai-website-to-markdown-rag-scraper" rel="noopener noreferrer"&gt;AI Website to Clean Markdown &amp;amp; RAG Chunk Extractor&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/company-career-hiring-signal-scraper" rel="noopener noreferrer"&gt;Company Career Page, ATS &amp;amp; Hiring Signal B2B Lead Scraper&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/b2b-website-contact-scraper" rel="noopener noreferrer"&gt;B2B Website Email, Phone, WhatsApp &amp;amp; Social Link Scraper&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/shopify-store-product-contact-scraper" rel="noopener noreferrer"&gt;Shopify Store Products, Pricing &amp;amp; Contact Scraper&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/website-tech-stack-detector" rel="noopener noreferrer"&gt;Website Tech Stack, Pixel, CMS &amp;amp; E-Commerce Detector&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/seo-audit-broken-link-meta-analyzer" rel="noopener noreferrer"&gt;Automated Website Technical SEO Audit &amp;amp; Meta Tag Analyzer&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/local-business-lead-enricher" rel="noopener noreferrer"&gt;Local Business Lead, Schema, Rating &amp;amp; Booking Enricher&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href="https://apify.com/kabeer099/woocommerce-product-catalog-scraper" rel="noopener noreferrer"&gt;WooCommerce Store Product, SKU &amp;amp; Price Catalog Scraper&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;All 8 Apify Actors:&lt;/strong&gt; &lt;a href="https://apify.com/kabeer099" rel="noopener noreferrer"&gt;https://apify.com/kabeer099&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Portfolio &amp;amp; MV3 Browser Extensions:&lt;/strong&gt; &lt;a href="https://kabeer.site/" rel="noopener noreferrer"&gt;https://kabeer.site/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub:&lt;/strong&gt; &lt;a href="https://github.com/Kabeer099" rel="noopener noreferrer"&gt;https://github.com/Kabeer099&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you have feedback on chunking heuristics or ATS edge cases, drop a comment below!&lt;/p&gt;

</description>
      <category>ai</category>
      <category>javascript</category>
      <category>node</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Building 8 Production Web Scrapers &amp; Manifest V3 Browser Extensions in Node.js</title>
      <dc:creator>Kabeer Ahmed</dc:creator>
      <pubDate>Sun, 25 Oct 2026 18:09:32 +0000</pubDate>
      <link>https://dev.to/kabeer099/building-8-production-web-scrapers-manifest-v3-browser-extensions-in-nodejs-4m08</link>
      <guid>https://dev.to/kabeer099/building-8-production-web-scrapers-manifest-v3-browser-extensions-in-nodejs-4m08</guid>
      <description>&lt;p&gt;Over the past few weeks, I built and shipped 8 production cloud scrapers on the &lt;a href="https://apify.com/kabeer099" rel="noopener noreferrer"&gt;Apify Store&lt;/a&gt; alongside two cross-browser Manifest V3 extensions on the Mozilla Add-ons Store.&lt;/p&gt;

&lt;p&gt;Most scraping tutorials default to spinning up full headless Chromium instances for every URL. While Playwright and Puppeteer are essential when dealing with heavy client-side hydration, they are often overkill for structured B2B extraction, e-commerce catalog indexing, and technical SEO auditing. By combining &lt;code&gt;CheerioCrawler&lt;/code&gt; in &lt;a href="https://crawlee.dev/" rel="noopener noreferrer"&gt;Crawlee&lt;/a&gt; with hidden platform JSON endpoints and regex-backed DOM parsers, you can cut memory overhead and runtime costs by over 85%.&lt;/p&gt;

&lt;p&gt;Here is a practical look at the architecture behind these tools and where you can try them out.&lt;/p&gt;




&lt;h2&gt;
  
  
  1. Fast E-Commerce Catalog Extraction Without Headless Browsers
&lt;/h2&gt;

&lt;p&gt;When scraping Shopify or WooCommerce stores, rendering product grids in a headless browser is slow and fragile. Both platforms expose structured JSON APIs out of the box on almost every storefront:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Shopify:&lt;/strong&gt; &lt;code&gt;GET /products.json?limit=250&amp;amp;page=1&lt;/code&gt; returns full product variants, SKUs, pricing, tags, and image URLs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;WooCommerce:&lt;/strong&gt; &lt;code&gt;GET /wp-json/wc/store/v1/products?per_page=100&lt;/code&gt; (Store API v1) exposes catalog items, stock status, and pricing metadata, with HTML fallback to Schema.org &lt;code&gt;Product&lt;/code&gt; JSON-LD when the REST route is restricted.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;fetchShopifyProducts&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;baseUrl&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;maxProducts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;250&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;/products.json?limit=250&amp;amp;page=1&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;baseUrl&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;res&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;User-Agent&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Mozilla/5.0 (compatible; CatalogBot/1.0)&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;res&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;res&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="k"&gt;return &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;products&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="p"&gt;[]).&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;maxProducts&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;p&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;({&lt;/span&gt;
    &lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;p.title,&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;
    &lt;span class="na"&gt;handle&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;handle&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;vendor&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;vendor&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;productType&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;product_type&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;price&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;variants&lt;/span&gt;&lt;span class="p"&gt;?.[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]?.&lt;/span&gt;&lt;span class="nx"&gt;price&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;available&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;variants&lt;/span&gt;&lt;span class="p"&gt;?.&lt;/span&gt;&lt;span class="nf"&gt;some&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;v&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;v&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;available&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;??&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;
  &lt;span class="p"&gt;}));&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I packaged these patterns into two dedicated cloud actors:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://apify.com/kabeer099/shopify-store-product-contact-scraper" rel="noopener noreferrer"&gt;Shopify Store Products, Pricing &amp;amp; Contact Scraper&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://apify.com/kabeer099/woocommerce-store-product-scraper" rel="noopener noreferrer"&gt;WooCommerce &amp;amp; WordPress Store Product &amp;amp; Contact Scraper&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  2. Converting Arbitrary Websites into Token-Sized Markdown Chunks for AI &amp;amp; RAG
&lt;/h2&gt;

&lt;p&gt;Feeding raw HTML into an LLM wastes context window tokens on navigation bars, cookie banners, inline SVGs, and tracking scripts. When building the &lt;a href="https://apify.com/kabeer099/website-to-markdown-ai-rag-scraper" rel="noopener noreferrer"&gt;Website to Clean Markdown &amp;amp; AI RAG Content Extractor&lt;/a&gt;, I structured the pipeline in three stages:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Boilerplate Stripping:&lt;/strong&gt; Remove &lt;code&gt;script&lt;/code&gt;, &lt;code&gt;style&lt;/code&gt;, &lt;code&gt;noscript&lt;/code&gt;, &lt;code&gt;svg&lt;/code&gt;, &lt;code&gt;nav&lt;/code&gt;, &lt;code&gt;footer&lt;/code&gt;, &lt;code&gt;header&lt;/code&gt;, and &lt;code&gt;aside&lt;/code&gt; nodes via Cheerio before inspecting text nodes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Semantic Markdown Conversion:&lt;/strong&gt; Convert headings (&lt;code&gt;h1&lt;/code&gt; to &lt;code&gt;h6&lt;/code&gt;), lists, tables, code blocks, and anchor links into clean Markdown while preserving document hierarchy.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Token-Aware Chunking:&lt;/strong&gt; Split long documents on heading boundaries into configurable chunk sizes (e.g. 500 to 1,000 words) with metadata (&lt;code&gt;url&lt;/code&gt;, &lt;code&gt;title&lt;/code&gt;, &lt;code&gt;chunkIndex&lt;/code&gt;, &lt;code&gt;estimatedTokens&lt;/code&gt;) ready for direct ingestion into vector databases like Pinecone, Qdrant, or pgvector.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  3. Detecting 40+ Tech Stack Signatures, Ad Pixels &amp;amp; B2B Contact Signals
&lt;/h2&gt;

&lt;p&gt;For sales intelligence and agency prospecting, knowing whether a company runs Shopify Plus, Next.js, HubSpot, Klaviyo, Meta Pixel, or Google Tag Manager is just as important as finding their contact email.&lt;/p&gt;

&lt;p&gt;In the &lt;a href="https://apify.com/kabeer099/website-tech-stack-detector" rel="noopener noreferrer"&gt;Website Tech Stack, Pixel &amp;amp; CMS Detector&lt;/a&gt;, each page response is evaluated across four lightweight inspection layers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Script &lt;code&gt;src&lt;/code&gt; &amp;amp; Link &lt;code&gt;href&lt;/code&gt; patterns:&lt;/strong&gt; Detects CDN signatures (&lt;code&gt;cdn.shopify.com&lt;/code&gt;, &lt;code&gt;/_next/static/&lt;/code&gt;, &lt;code&gt;js.hs-scripts.com&lt;/code&gt;, &lt;code&gt;static.klaviyo.com&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Inline Script Globals:&lt;/strong&gt; Matches initialization snippets such as &lt;code&gt;fbq('init'&lt;/code&gt;, &lt;code&gt;gtag('config'&lt;/code&gt;, &lt;code&gt;lintrk&lt;/code&gt;, and &lt;code&gt;ttq.load&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta Generator &amp;amp; DOM Attributes:&lt;/strong&gt; Checks &lt;code&gt;meta[name="generator"]&lt;/code&gt;, &lt;code&gt;data-wf-site&lt;/code&gt; (Webflow), and &lt;code&gt;ng-version&lt;/code&gt; (Angular).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;HTTP Response Headers:&lt;/strong&gt; Inspects &lt;code&gt;x-powered-by&lt;/code&gt;, &lt;code&gt;server&lt;/code&gt;, and &lt;code&gt;x-shopify-stage&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Here are the rest of the B2B and SEO actors in the suite:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://apify.com/kabeer099/b2b-website-contact-scraper" rel="noopener noreferrer"&gt;B2B Website Email, Phone, WhatsApp &amp;amp; Social Links Scraper&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://apify.com/kabeer099/seo-meta-schema-auditor" rel="noopener noreferrer"&gt;Technical SEO, Meta Tags, OpenGraph &amp;amp; Schema JSON-LD Auditor&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://apify.com/kabeer099/website-broken-link-sitemap-checker" rel="noopener noreferrer"&gt;Website Sitemap, Broken Links &amp;amp; Status Code Health Checker&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://apify.com/kabeer099/company-careers-job-postings-scraper" rel="noopener noreferrer"&gt;Company Careers &amp;amp; Open Job Listings Scraper (Greenhouse, Lever &amp;amp; Ashby)&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  4. Shipping Cross-Browser Manifest V3 Extensions (Chrome &amp;amp; Firefox)
&lt;/h2&gt;

&lt;p&gt;Alongside the cloud scrapers, I wanted instant in-browser inspection tools when reviewing client websites. I built and published two Manifest V3 browser extensions on the official Mozilla Add-ons Store:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://addons.mozilla.org/en-US/firefox/addon/leadsnap-pro-b2b-extractor/" rel="noopener noreferrer"&gt;LeadSnap Pro - B2B Contact &amp;amp; Email Extractor&lt;/a&gt;:&lt;/strong&gt; Extracts verified emails, phone numbers, WhatsApp links, and social profiles directly from the active tab DOM and &lt;code&gt;mailto:&lt;/code&gt; / &lt;code&gt;tel:&lt;/code&gt; links, with one-click CSV and JSON export.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://addons.mozilla.org/en-US/firefox/addon/metapeek-pro-seo-auditor/" rel="noopener noreferrer"&gt;MetaPeek Pro - Technical SEO &amp;amp; Schema Auditor&lt;/a&gt;:&lt;/strong&gt; Audits canonical URLs, title/description pixel lengths, heading hierarchy (&lt;code&gt;H1&lt;/code&gt; to &lt;code&gt;H6&lt;/code&gt;), OpenGraph/Twitter cards, image alt coverage, and Schema.org JSON-LD blocks in real time.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Tip for Cross-Browser MV3 Manifests
&lt;/h3&gt;

&lt;p&gt;When packaging a single Manifest V3 codebase for both Chromium and Firefox, remember that Firefox requires &lt;code&gt;browser_specific_settings.gecko.id&lt;/code&gt; and currently uses background scripts or event pages alongside standard &lt;code&gt;scripting&lt;/code&gt; and &lt;code&gt;activeTab&lt;/code&gt; permissions. Keeping your DOM analyzer inside a self-contained content script injected via &lt;code&gt;chrome.scripting.executeScript&lt;/code&gt; (with &lt;code&gt;browser.*&lt;/code&gt; fallback) makes the extension work seamlessly across both engines with zero external runtime dependencies.&lt;/p&gt;




&lt;h2&gt;
  
  
  Wrap-Up &amp;amp; Links
&lt;/h2&gt;

&lt;p&gt;All 8 cloud actors and both browser extensions are live right now:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Apify Store Profile (All 8 Actors):&lt;/strong&gt; &lt;a href="https://apify.com/kabeer099" rel="noopener noreferrer"&gt;https://apify.com/kabeer099&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;LeadSnap Pro on Firefox Add-ons:&lt;/strong&gt; &lt;a href="https://addons.mozilla.org/en-US/firefox/addon/leadsnap-pro-b2b-extractor/" rel="noopener noreferrer"&gt;https://addons.mozilla.org/en-US/firefox/addon/leadsnap-pro-b2b-extractor/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;MetaPeek Pro on Firefox Add-ons:&lt;/strong&gt; &lt;a href="https://addons.mozilla.org/en-US/firefox/addon/metapeek-pro-seo-auditor/" rel="noopener noreferrer"&gt;https://addons.mozilla.org/en-US/firefox/addon/metapeek-pro-seo-auditor/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub:&lt;/strong&gt; &lt;a href="https://github.com/Kabeer099" rel="noopener noreferrer"&gt;https://github.com/Kabeer099&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Portfolio:&lt;/strong&gt; &lt;a href="https://kabeer.site/" rel="noopener noreferrer"&gt;https://kabeer.site/&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you are building data pipelines, e-commerce integrations, or custom web applications and want to chat, feel free to reach out through my portfolio or GitHub!&lt;/p&gt;

</description>
      <category>javascript</category>
      <category>node</category>
      <category>webdev</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
