How to stop LLMs from hallucinating on complex HTML tables (Python)

#python #ai #webscraping #opensource

If you are building Retrieval-Augmented Generation (RAG) pipelines or feeding web data into Large Language Models (LLMs), you already know the golden rule: Garbage in, garbage out.

Text extraction is mostly a solved problem. But the moment you encounter HTML tables, things usually get messy. And if those tables contain rowspan or colspan attributes, standard extraction pipelines break completely.

The result? Misaligned Markdown, missing context, and an LLM that hallucinates relationships between headers and cells.

After hitting this wall one too many times, I decided to build a Python library to solve this specific problem once and for all: html-table-rescuer.

Here is why standard parsers fail, and how this new tool fixes it.

The Problem: Spanned Cells destroy Context

Most standard table-to-markdown scripts or naive web scrapers read HTML row by row (<tr>), cell by cell (<td>).

But look at what happens when a table uses a <td rowspan="2">:

The HTML:

<table border="1">
  <tr>
    <td rowspan="2">Spanned Header</td>
    <td>Row 1 Data</td>
  </tr>
  <tr>
    <td>Row 2 Data</td>
  </tr>
</table>

What standard parsers output:

| Header | Value | 
| ----- | ----- | 
| Spanned Header | Row 1 Data | 
| Row 2 Data |  |

Notice the shift? In the second row, the data is pushed into the wrong column. If you feed this into an LLM, the model will associate "Row 2 Data" with the first column header. The context of the "Spanned Header" is completely lost for the second row.

The Solution: A Grid Logic Solver

To fix this, we can't just parse the tags; we need to understand the underlying mathematical grid of the table.

I built html-table-rescuer to do exactly that. It uses BeautifulSoup to parse the DOM, but then applies a custom Grid Logic Solver. It calculates the exact matrix coordinates for every cell, resolving all row and column spans perfectly.

But more importantly for AI workflows: It preserves the semantic context.

Instead of leaving spanned cells blank, it actively fills them so the LLM retains the context for every single row.

What html-table-rescuer outputs:

| Header | Value | 
| ----- | ----- | 
| Spanned Header | Row 1 Data | 
| dito (Spanned Header) | Row 2 Data |

(Notice how the tool fills the gap with a customizable prefix like "dito" so the LLM knows it belongs to the span above!)

Deep Tag Parsing & Nested Tables

Another massive headache in web scraping is deep nesting.
Often, a <td> doesn't just contain text. It contains a <div>, which contains a <span>, which contains an <a href="..."> and some <b> bold text.

If you just extract the text, you lose valuable metadata (like links). html-table-rescuer recursively parses inline tags. It keeps your links, bold, and italic formatting alive in the final Markdown output.

It even safely extracts nested tables without destroying the grid of the parent table.

Quick Start

I wanted to make it as easy to use as possible. You can install it via pip:

pip install html-table-rescuer

And parse any complex HTML in just a few lines of Python:

from table2md.core import TableParser

html = """
<!-- Insert your gnarly, complex HTML table here -->
"""

# Initialize parser
parser = TableParser(html)

# Parse all tables
tables = parser.parse()

if tables:
    # Export to clean Markdown for your LLM context window
    print(tables[0].to_markdown())

    # Or export to JSON / CSV if you need structured data
    # print(tables[0].to_json())