<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Ajay Jaknale</title>
    <description>The latest articles on DEV Community by Ajay Jaknale (@ajay_jaknale).</description>
    <link>https://dev.to/ajay_jaknale</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4254504%2F8d3cb70b-5c64-4ceb-8d5a-bc87c6be6e7d.PNG</url>
      <title>DEV Community: Ajay Jaknale</title>
      <link>https://dev.to/ajay_jaknale</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/ajay_jaknale"/>
    <language>en</language>
    <item>
      <title>Stop Feeding Raw HTML to Your LLM: Clean Web Scrapes for RAG in One API Call</title>
      <dc:creator>Ajay Jaknale</dc:creator>
      <pubDate>Thu, 19 Nov 2026 22:16:23 +0000</pubDate>
      <link>https://dev.to/ajay_jaknale/stop-feeding-raw-html-to-your-llm-clean-web-scrapes-for-rag-in-one-api-call-i3b</link>
      <guid>https://dev.to/ajay_jaknale/stop-feeding-raw-html-to-your-llm-clean-web-scrapes-for-rag-in-one-api-call-i3b</guid>
      <description>&lt;p&gt;If you build Retrieval-Augmented Generation (RAG) pipelines, AI agents, or web-scraping workflows in Python or n8n, you know the pain of messy web data.&lt;/p&gt;

&lt;p&gt;Passing raw scraped HTML directly into an LLM prompt or vector database wastes context window tokens on &lt;code&gt;&amp;lt;script&amp;gt;&lt;/code&gt;, &lt;code&gt;&amp;lt;style&amp;gt;&lt;/code&gt;, &lt;code&gt;&amp;lt;iframe&amp;gt;&lt;/code&gt;, and navigation noise—and hurts embedding retrieval accuracy. Writing and maintaining custom BeautifulSoup, Turndown, and regex cleanup scripts across every microservice quickly turns into repetitive boilerplate.&lt;/p&gt;

&lt;p&gt;Here is how to turn raw HTML into clean, token-efficient Markdown and normalize messy text in one REST call using the &lt;strong&gt;Wordflow Text &amp;amp; Markdown API&lt;/strong&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Clean Markdown Matters for RAG
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Lower token usage:&lt;/strong&gt; Stripping inline CSS, tracking scripts, and layout tags often reduces payload size by 50–80% before it hits your LLM.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Better chunking &amp;amp; embeddings:&lt;/strong&gt; Headings (&lt;code&gt;#&lt;/code&gt;, &lt;code&gt;##&lt;/code&gt;), lists, and code blocks give semantic text splitters clean structural boundaries.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deterministic output:&lt;/strong&gt; No LLM hallucination risk during preprocessing—just fast, predictable text transformation.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Step 1: Convert Raw HTML to Clean Markdown
&lt;/h2&gt;

&lt;p&gt;First, grab a free API key by subscribing to the &lt;strong&gt;BASIC&lt;/strong&gt; tier (500 free requests/month) on the &lt;a href="https://rapidapi.com/AjayJaknale/api/wordflow-text-markdown-api1" rel="noopener noreferrer"&gt;Wordflow Text &amp;amp; Markdown API on RapidAPI&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Send a &lt;code&gt;POST&lt;/code&gt; request to &lt;code&gt;/html-to-markdown&lt;/code&gt; with &lt;code&gt;stripScriptsAndStyles&lt;/code&gt; enabled:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="n"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"[https://wordflow-text-markdown-api.p.rapidapi.com/html-to-markdown](https://wordflow-text-markdown-api.p.rapidapi.com/html-to-markdown)"&lt;/span&gt;

&lt;span class="n"&gt;headers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"Content-Type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"application/json"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"X-RapidAPI-Key"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"YOUR_RAPIDAPI_KEY"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"X-RapidAPI-Host"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"wordflow-text-markdown-api.p.rapidapi.com"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;raw_html&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"""
&amp;lt;article&amp;gt;
  &amp;lt;h1&amp;gt;Q3 Architecture Review&amp;lt;/h1&amp;gt;
  &amp;lt;script&amp;gt;window.analytics.track('pageview');&amp;lt;/script&amp;gt;
  &amp;lt;style&amp;gt;.nav-banner { display: none; }&amp;lt;/style&amp;gt;
  &amp;lt;p&amp;gt;API latency dropped by &amp;lt;strong&amp;gt;42%&amp;lt;/strong&amp;gt; after edge caching.&amp;lt;/p&amp;gt;
  &amp;lt;ul&amp;gt;
    &amp;lt;li&amp;gt;Zero cold-start overhead&amp;lt;/li&amp;gt;
    &amp;lt;li&amp;gt;Deterministic JSON output&amp;lt;/li&amp;gt;
  &amp;lt;/ul&amp;gt;
&amp;lt;/article&amp;gt;
"""&lt;/span&gt;

&lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"html"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;raw_html&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"headingStyle"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"atx"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"stripScriptsAndStyles"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"markdown"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Output
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gu"&gt;# Q3 Architecture Review
&lt;/span&gt;
API latency dropped by &lt;span class="gs"&gt;**42%**&lt;/span&gt; after edge caching.

&lt;span class="k"&gt;- &lt;/span&gt;Zero cold-start overhead
&lt;span class="k"&gt;- &lt;/span&gt;Deterministic JSON output
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All tracking scripts and inline CSS are gone, leaving clean Markdown ready for chunking or prompt injection.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 2: Sanitize Whitespace and Duplicate Lines
&lt;/h2&gt;

&lt;p&gt;When extracting text from PDFs, emails, or legacy web pages, you often end up with erratic line breaks, extra spaces, and repeated lines. Chain the &lt;code&gt;/clean&lt;/code&gt; endpoint right after extraction:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nb"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'[https://wordflow-text-markdown-api.p.rapidapi.com/clean](https://wordflow-text-markdown-api.p.rapidapi.com/clean)'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'POST'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s1"&gt;'Content-Type'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'application/json'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'X-RapidAPI-Key'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'YOUR_RAPIDAPI_KEY'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'X-RapidAPI-Host'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'wordflow-text-markdown-api.p.rapidapi.com'&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;stringify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Chunk one:   API latency metrics.   \n\n\n\nChunk two:   Vector index updated.  "&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Using It in n8n or Make (No-Code Workflows)
&lt;/h2&gt;

&lt;p&gt;If you orchestrate your AI pipelines visually in &lt;strong&gt;n8n&lt;/strong&gt; or &lt;strong&gt;Make&lt;/strong&gt;:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Add an &lt;strong&gt;HTTP Request&lt;/strong&gt; node after your web scraper or RSS node.&lt;/li&gt;
&lt;li&gt;Set &lt;strong&gt;Method&lt;/strong&gt; to &lt;code&gt;POST&lt;/code&gt; and &lt;strong&gt;URL&lt;/strong&gt; to &lt;code&gt;https://wordflow-text-markdown-api.p.rapidapi.com/html-to-markdown&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Add headers for &lt;code&gt;X-RapidAPI-Key&lt;/code&gt;, &lt;code&gt;X-RapidAPI-Host&lt;/code&gt;, and &lt;code&gt;Content-Type: application/json&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Send the JSON payload below in the request body and pass the returned Markdown directly to your OpenAI, Anthropic, or vector store node:
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"html"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"{{ $json.body }}"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Try It Free
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;API &amp;amp; Endpoints (&lt;code&gt;/html-to-markdown&lt;/code&gt;, &lt;code&gt;/clean&lt;/code&gt;, &lt;code&gt;/analyze&lt;/code&gt;, &lt;code&gt;/transform&lt;/code&gt;, &lt;code&gt;/diff&lt;/code&gt;):&lt;/strong&gt; &lt;a href="https://rapidapi.com/AjayJaknale/api/wordflow-text-markdown-api1" rel="noopener noreferrer"&gt;Wordflow Text &amp;amp; Markdown API on RapidAPI&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Free Browser Text Tools:&lt;/strong&gt; &lt;a href="https://www.getwordflow.com" rel="noopener noreferrer"&gt;Wordflow Web App&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What preprocessing steps do you run before feeding scraped documents into your RAG pipeline? Drop your setup in the comments!&lt;/p&gt;

</description>
      <category>ai</category>
      <category>python</category>
      <category>rag</category>
      <category>api</category>
    </item>
  </channel>
</rss>
