Over the past few weeks, I built and shipped 8 production cloud scrapers on the Apify Store alongside two cross-browser Manifest V3 extensions on the Mozilla Add-ons Store.
Most scraping tutorials default to spinning up full headless Chromium instances for every URL. While Playwright and Puppeteer are essential when dealing with heavy client-side hydration, they are often overkill for structured B2B extraction, e-commerce catalog indexing, and technical SEO auditing. By combining CheerioCrawler in Crawlee with hidden platform JSON endpoints and regex-backed DOM parsers, you can cut memory overhead and runtime costs by over 85%.
Here is a practical look at the architecture behind these tools and where you can try them out.
1. Fast E-Commerce Catalog Extraction Without Headless Browsers
When scraping Shopify or WooCommerce stores, rendering product grids in a headless browser is slow and fragile. Both platforms expose structured JSON APIs out of the box on almost every storefront:
-
Shopify:
GET /products.json?limit=250&page=1returns full product variants, SKUs, pricing, tags, and image URLs. -
WooCommerce:
GET /wp-json/wc/store/v1/products?per_page=100(Store API v1) exposes catalog items, stock status, and pricing metadata, with HTML fallback to Schema.orgProductJSON-LD when the REST route is restricted.
async function fetchShopifyProducts(baseUrl, maxProducts = 250) {
const url = new URL('/products.json?limit=250&page=1', baseUrl).toString();
const res = await fetch(url, {
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; CatalogBot/1.0)' }
});
if (!res.ok) return [];
const data = await res.json();
return (data.products || []).slice(0, maxProducts).map(p => ({
id: p.id,
title: "p.title,"
handle: p.handle,
vendor: p.vendor,
productType: p.product_type,
price: p.variants?.[0]?.price || null,
available: p.variants?.some(v => v.available) ?? false
}));
}
I packaged these patterns into two dedicated cloud actors:
- Shopify Store Products, Pricing & Contact Scraper
- WooCommerce & WordPress Store Product & Contact Scraper
2. Converting Arbitrary Websites into Token-Sized Markdown Chunks for AI & RAG
Feeding raw HTML into an LLM wastes context window tokens on navigation bars, cookie banners, inline SVGs, and tracking scripts. When building the Website to Clean Markdown & AI RAG Content Extractor, I structured the pipeline in three stages:
-
Boilerplate Stripping: Remove
script,style,noscript,svg,nav,footer,header, andasidenodes via Cheerio before inspecting text nodes. -
Semantic Markdown Conversion: Convert headings (
h1toh6), lists, tables, code blocks, and anchor links into clean Markdown while preserving document hierarchy. -
Token-Aware Chunking: Split long documents on heading boundaries into configurable chunk sizes (e.g. 500 to 1,000 words) with metadata (
url,title,chunkIndex,estimatedTokens) ready for direct ingestion into vector databases like Pinecone, Qdrant, or pgvector.
3. Detecting 40+ Tech Stack Signatures, Ad Pixels & B2B Contact Signals
For sales intelligence and agency prospecting, knowing whether a company runs Shopify Plus, Next.js, HubSpot, Klaviyo, Meta Pixel, or Google Tag Manager is just as important as finding their contact email.
In the Website Tech Stack, Pixel & CMS Detector, each page response is evaluated across four lightweight inspection layers:
-
Script
src& Linkhrefpatterns: Detects CDN signatures (cdn.shopify.com,/_next/static/,js.hs-scripts.com,static.klaviyo.com). -
Inline Script Globals: Matches initialization snippets such as
fbq('init',gtag('config',lintrk, andttq.load. -
Meta Generator & DOM Attributes: Checks
meta[name="generator"],data-wf-site(Webflow), andng-version(Angular). -
HTTP Response Headers: Inspects
x-powered-by,server, andx-shopify-stage.
Here are the rest of the B2B and SEO actors in the suite:
- B2B Website Email, Phone, WhatsApp & Social Links Scraper
- Technical SEO, Meta Tags, OpenGraph & Schema JSON-LD Auditor
- Website Sitemap, Broken Links & Status Code Health Checker
- Company Careers & Open Job Listings Scraper (Greenhouse, Lever & Ashby)
4. Shipping Cross-Browser Manifest V3 Extensions (Chrome & Firefox)
Alongside the cloud scrapers, I wanted instant in-browser inspection tools when reviewing client websites. I built and published two Manifest V3 browser extensions on the official Mozilla Add-ons Store:
-
LeadSnap Pro - B2B Contact & Email Extractor: Extracts verified emails, phone numbers, WhatsApp links, and social profiles directly from the active tab DOM and
mailto:/tel:links, with one-click CSV and JSON export. -
MetaPeek Pro - Technical SEO & Schema Auditor: Audits canonical URLs, title/description pixel lengths, heading hierarchy (
H1toH6), OpenGraph/Twitter cards, image alt coverage, and Schema.org JSON-LD blocks in real time.
Tip for Cross-Browser MV3 Manifests
When packaging a single Manifest V3 codebase for both Chromium and Firefox, remember that Firefox requires browser_specific_settings.gecko.id and currently uses background scripts or event pages alongside standard scripting and activeTab permissions. Keeping your DOM analyzer inside a self-contained content script injected via chrome.scripting.executeScript (with browser.* fallback) makes the extension work seamlessly across both engines with zero external runtime dependencies.
Wrap-Up & Links
All 8 cloud actors and both browser extensions are live right now:
- Apify Store Profile (All 8 Actors): https://apify.com/kabeer099
- LeadSnap Pro on Firefox Add-ons: https://addons.mozilla.org/en-US/firefox/addon/leadsnap-pro-b2b-extractor/
- MetaPeek Pro on Firefox Add-ons: https://addons.mozilla.org/en-US/firefox/addon/metapeek-pro-seo-auditor/
- GitHub: https://github.com/Kabeer099
- Portfolio: https://kabeer.site/
If you are building data pipelines, e-commerce integrations, or custom web applications and want to chat, feel free to reach out through my portfolio or GitHub!
Top comments (3)
Great catch! Flattening variants so each SKU emits its own row (or computing min/max variant price alongside per-variant stock status) prevents sold-out base SKUs from masking active variant pricing. Updating the Shopify actor to include per-variant availability mapping in the next release.
The Shopify mapping takes the first variant's price but any variant's availability. A sold-out cheap SKU can inherit another SKU's stock; emit one row per variant.
Great catch! Flattening variants so each SKU emits its own row (or computing min/max variant price alongside per-variant stock status) prevents sold-out base SKUs from masking active variant pricing. Updating the Shopify actor to include per-variant availability mapping in the next release.