How to Convert Any Webpage to Clean Markdown for RAG & Save 90% LLM Tokens in Node.js

Artuturoh — Fri, 06 Nov 2026 05:57:30 +0000

If you are building AI agents, RAG (Retrieval-Augmented Generation) pipelines, or feeding web content into vector databases (Pinecone, Chroma, Qdrant), you already know the big problem with web scraping: Raw HTML is a token killer.

A typical news article or documentation page weighs 500 KB to 1 MB. Most of that weight is bloated navigation menus, ads, tracking scripts, and cookie banners. Ingesting raw HTML directly into an LLM context window burns tens of thousands of tokens and confuses the model.

Here is a clean, lightweight way to parse any URL directly into clean, noise-free GitHub Flavored Markdown (GFM) in Node.js while cutting token consumption by ~90%.

The Problem with Sending Raw Web Content to LLMs

Consider fetching a long Wikipedia or news article:

// Raw HTML fetch
const html = await (await fetch("[https://en.wikipedia.org/wiki/Artificial_intelligence](https://en.wikipedia.org/wiki/Artificial_intelligence)")).text();
console.log(html.length); // ~2,200,000 characters (~570,000 tokens!)

Sending 570k tokens into a model like GPT-4o or Claude 3.5 Sonnet costs real money and easily blows past standard context windows.

The Solution: Extraction via Mozilla Readability & Turndown
To solve this for AI pipelines, I built MarkRAG API: a microservice in Node.js that fetches the page with realistic browser header emulation, extracts the core content using Mozilla's Readability engine (the same engine behind Firefox Reader View), and converts it into clean Markdown with tables and code blocks preserved.

It also calculates exact token metrics so you know how much context you saved:

import express from "express";

const app = express();
app.use(express.json());

app.post("/api/rag/ingest", async (req, res) => {
  const { url } = req.body;

  try {
    const response = await fetch("[https://markrag-web-to-clean-markdown-and-rag-parser.p.rapidapi.com/v1/extract](https://markrag-web-to-clean-markdown-and-rag-parser.p.rapidapi.com/v1/extract)", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "X-RapidAPI-Key": process.env.RAPIDAPI_KEY,
        "X-RapidAPI-Host": "markrag-web-to-clean-markdown-and-rag-parser.p.rapidapi.com"
      },
      body: JSON.stringify({
        url: url,
        options: {
          strip_links: true, // Flattens links to plain text to save extra tokens
          cache: true        // Sub-2ms response for recurring URLs
        }
      })
    });

    const { data } = await response.json();

    console.log(`Title: ${data.title}`);
    console.log(`Tokens saved: ${data.metadata.tokensSaved}`);
    console.log(`Context reduction: ${data.metadata.savingsPercentage}%`);

    // Ingest clean data.markdown directly into your vector store or LLM prompt
    return res.status(200).json({
      title: data.title,
      markdown: data.markdown,
      tokens: data.metadata.estimatedTokens
    });

  } catch (error) {
    console.error("Extraction error:", error);
    return res.status(500).json({ error: "Failed to extract content" });
  }
});

Benchmark: Raw HTML vs Clean Markdown
Testing the API against the Wikipedia Artificial Intelligence article yielded these real numbers:

Raw HTML Size: ~2.2 MB (~572,000 tokens).

Clean Markdown Size: ~201 KB (~50,300 tokens).

Tokens Saved: 521,693 tokens.

Savings Percentage: 91.2%.

It strips all sidebar clutter, footers, and redundant scripts while keeping markdown headings (#, ##), bullet points, and code blocks intact.

Try it out for your AI agents
I published the microservice on RapidAPI with a free tier of 100 requests/month for developers and side projects:

👉 MarkRAG API on RapidAPI

rapidapi.com

If you are building LangChain, LlamaIndex, or custom RAG workflows, test it out with your target URLs and let me know how it handles them!

How to Block Disposable Emails & Fake Signups in Node.js (in under 50ms)

Artuturoh — Fri, 06 Nov 2026 03:41:29 +0000

If you've ever built a SaaS, a side project, or a newsletter, you know the pain: bot registrations, disposable emails (10-Minute Mail, GuerrillaMail, Mailinator), and mistyped domains (user@gmial.com) filling up your database and burning your transactional email quotas.

Instead of writing a custom blocklist or maintaining thousands of domains yourself, here is a clean, lightweight way to validate incoming emails directly in your Express auth pipeline.

The Problem with Basic Regex Validation

Most tutorials recommend just validating email syntax with a regex:

// This only checks syntax, not if the inbox or domain is legitimate
const isValid = /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email);

While this checks RFC formatting, it won't stop someone signing up with throwaway@mailinator.com or an unconfigured domain with no real records.

The Clean Solution: Validating in Express
To solve this for my own projects, I built EmailGuard API, a microservice in Node.js that checks in-memory disposable domain sets (O(1) lookups) and resolves real records with an 800ms strict timeout.

It returns an actionable recommendation field ("allow", "review", or "block"), making integration straightforward:

import express from "express";

const app = express();
app.use(express.json());

app.post("/api/register", async (req, res) => {
  const { email, password } = req.body;

  try {
    const response = await fetch("[https://emailguard-disposable-email-detector-and-hygiene.p.rapidapi.com/v1/verify](https://emailguard-disposable-email-detector-and-hygiene.p.rapidapi.com/v1/verify)", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "X-RapidAPI-Key": process.env.RAPIDAPI_KEY,
        "X-RapidAPI-Host": "emailguard-disposable-email-detector-and-hygiene.p.rapidapi.com"
      },
      body: JSON.stringify({ email })
    });

    const result = await response.json();
    const data = result.data;

    // Direct 1-line check
    if (data.recommendation === "block") {
      return res.status(400).json({ 
        error: "Disposable or invalid emails are not allowed." 
      });
    }

    // Optional: Auto-correct common typos (e.g. gmial.com -> gmail.com)
    if (data.checks.typo_guard.has_typo) {
      console.log(`Suggested email: ${data.checks.typo_guard.suggested_email}`);
    }

    // Continue with user creation...
    return res.status(201).json({ message: "User registered successfully" });

  } catch (error) {
    console.error("Verification failed:", error);
    return res.status(500).json({ error: "Registration error" });
  }
});

What the API Returns
Calling the verification endpoint returns a comprehensive hygiene breakdown:
{
  "status": "success",
  "data": {
    "email": "spam@mailinator.com",
    "score": 0,
    "verdict": "invalid",
    "recommendation": "block",
    "checks": {
      "syntax": { "valid": true, "rfc_compliant": true },
      "dns": { "has_mx_records": false },
      "disposable": { "is_disposable": true, "provider": "mailinator" },
      "account_type": { "is_free_provider": false, "is_role_based": false },
      "typo_guard": { "has_typo": false }
    },
    "meta": {
      "execution_time_ms": 1
    }
  }
}

Try it out for your projects
I published the microservice on RapidAPI with a free tier of 100 requests/month for developers and side projects:

👉 [EmailGuard API on RapidAPI]

rapidapi.com

Feedback and edge cases are welcome! If you test it on weird domains, let me know how it handles them.

DEV Community: Artuturoh

How to Convert Any Webpage to Clean Markdown for RAG & Save 90% LLM Tokens in Node.js

The Problem with Sending Raw Web Content to LLMs

How to Block Disposable Emails & Fake Signups in Node.js (in under 50ms)

The Problem with Basic Regex Validation