Diff Two Snapshots of a Messy CSV in Python: No-Break Spaces, Duplicate Rows and Renamed Keys

Carter Wang — Fri, 30 Oct 2026 05:18:02 +0000

If you build anything on top of a public dataset, sooner or later a new version lands and you have to answer a simple question: what changed? With a hand-maintained CSV, that question is harder than it looks. Keys get renamed, cells carry footnote symbols, and one invisible character can turn an unchanged row into an "added" and a "removed" at the same time.

This tutorial builds a small Python diff for two git snapshots of the VPS Pricing Index, a hosting-price dataset I maintain. I drafted the article with AI assistance; every command was run against the real repository, and the outputs below are copied from those runs and checked against the files by hand. One disclosure: VPS Hosting Radar earns affiliate commissions on some of its pages. This post has no affiliate links.

The two snapshots

The dataset lives in a public GitHub repository under CC BY 4.0. pricing.csv has three commits, and git keeps every version:

git clone https://github.com/vpshostingradar/vps-pricing-data.git
cd vps-pricing-data
git log --oneline -- pricing.csv
# 39a4cad Update pricing.csv
# 53b7d49 Initial dataset: 16 plans across 13 providers, collected 2026-08-11
# 8e6fc61 Create pricing.csv

git show 53b7d49:pricing.csv > aug.csv   # verified 2026-08-11
git show 39a4cad:pricing.csv > sep.csv   # verified 2026-09-15

git show <commit>:<path> writes a file exactly as it was at that commit, so you don't need to check anything out. (The oldest commit, 8e6fc61, is a header-only placeholder with a different snake_case schema and no data rows, so I left it out.)

Open aug.csv and you'll see the things that break naive tooling:

Row 1 is a title, not a header. It's a single quoted sentence describing the snapshot. The real header is on row 2.
Money cells aren't numbers. You'll find "2,520.00" with a thousands separator, €4.40‡ with a currency sign and a footnote dagger, € 5.50 with a space, and — (an em dash) meaning "not published."
Some names contain U+00A0. Three lines use a no-break space instead of a regular one, for example between Cloud VPS 4 and (EUR). It looks identical on screen.
Two rows appear twice. Liquid Web and Contabo each show up a second time at the bottom of the August file, byte for byte identical. The September file doesn't have this problem.

None of this is unusual. It's what happens when people copy values out of pricing pages and spreadsheets by hand. You want a diff that survives it.

Step 1: parse the cells you actually have

Start with three helpers: one that pulls a number out of whatever is in a money cell, one that normalizes text so invisible differences stop mattering, and a loader that finds the real header.

import csv
import re
import sys
import unicodedata
from collections import Counter

MONEY = re.compile(r"-?\d+(?:\.\d+)?")


def money(cell):
    """'2,520.00' -> 2520.0, '€ 5.50' -> 5.5, '35.00‡' -> 35.0, '—' -> None."""
    cell = (cell or "").replace(",", "").strip()
    m = MONEY.search(cell)
    return float(m.group()) if m else None


def norm(text):
    """NFKC turns U+00A0 (no-break space) into a plain space; then collapse runs."""
    return " ".join(unicodedata.normalize("NFKC", text).split())


def load(path):
    with open(path, newline="", encoding="utf-8") as f:
        lines = list(csv.reader(f))
    # Row 0 is a human-readable title, not the header. Find the real header.
    start = next(i for i, row in enumerate(lines) if row and row[0] == "Provider")
    header = lines[start]
    return [dict(zip(header, r)) for r in lines[start + 1:] if any(r)]

A few choices here are deliberate.

money() returns None for an em dash rather than 0. "The vendor didn't publish a monthly price" and "the monthly price is zero" are different facts, and the diff should report a move from None to 26.99 as new information, not as a $26.99 price rise.

The regex ignores the currency symbol, which is safe only because currency doesn't change within a row between snapshots. If you're diffing a dataset where a row could switch from USD to EUR, keep the symbol as a separate field and compare it too.

norm() uses NFKC normalization. Among other things, NFKC maps U+00A0 to an ordinary space. str.split() with no arguments already treats U+00A0 as whitespace in Python 3, so the split/join pass alone would catch this particular character. NFKC also handles full-width digits and similar compatibility characters, which turn up when values get pasted from some PDFs and from CJK input methods.

Step 2: key the rows, and say out loud which ones were replaced

A diff needs a key. Here the natural key is (Provider, Plan). Two things can go wrong with it: duplicates within one file, and keys that change between files because the maintainer swapped one plan for another.

def keyed(rows, label):
    keys = [(norm(r["Provider"]), norm(r["Plan"])) for r in rows]
    for key, n in Counter(keys).items():
        if n > 1:
            print(f"[{label}] duplicate key {key} appears {n}x")
    out = {}
    for key, r in zip(keys, rows):
        out.setdefault(key, r)  # keep first occurrence
    return out


# Row replacements are editorial decisions, so they come from the dataset's
# own notes/changelog, not from fuzzy matching. new key -> old key.
REPLACES = {
    ("Hostinger", "Web Premium"): ("Hostinger", "Cloud Startup"),
    ("Hostinger", "KVM 1"): ("Hostinger", "KVM 2"),
    ("InMotion", "VPS 4 vCPU"): ("InMotion", "VPS 4 (Most Popular)"),
    ("ScalaHosting", "Managed Cloud #1"): ("ScalaHosting", "Managed Build #1"),
}

keyed() warns about duplicates instead of silently overwriting them. Keeping the first occurrence is fine here because the duplicates are identical; if they weren't, you'd want the script to stop.

The REPLACES map is the part people are tempted to automate. Don't. Hostinger has two rows in each snapshot, so "match by provider" is ambiguous. Fuzzy string matching would happily pair KVM 2 with KVM 1, which happens to be right, but for the wrong reason: that pairing is true because the September notes column says "Replaces KVM 2 (8 GB), which overshot the 4 GB baseline," not because the strings are similar. Every entry in the map above comes from a note in the September file or from the repository's CHANGELOG.md. When the next snapshot arrives, you read the changelog and update the map. It takes two minutes and you'll trust the output afterwards.

Step 3: compare field by field

FIELDS = {
    "First term $/mo": "intro",
    "Prepay months": "prepay",
    "Monthly $/mo": "monthly",
    "Renewal $/mo": "renewal",
    "36-month total $": "total_36",
}


def compare(old, new):
    matched = set()
    for key, n in new.items():
        okey = key if key in old else REPLACES.get(key)
        if okey not in old:
            print(f"+ added    {key[0]} {key[1]}")
            continue
        matched.add(okey)
        tag = "same plan" if okey == key else f"replaces {okey[1]}"
        changes = []
        for col, short in FIELDS.items():
            a, b = money(old[okey].get(col)), money(n.get(col))
            if a != b:
                changes.append(f"{short} {a} -> {b}")
        if changes:
            print(f"~ changed  {key[0]} {key[1]} ({tag}): " + "; ".join(changes))
    for okey in old.keys() - matched:
        print(f"- removed  {okey[0]} {okey[1]}")


if __name__ == "__main__":
    old = keyed(load(sys.argv[1]), "old")
    new = keyed(load(sys.argv[2]), "new")
    print(f"old: {len(old)} unique plans, new: {len(new)} unique plans\n")
    compare(old, new)

Save the three blocks as snapdiff.py and run python3 snapdiff.py aug.csv sep.csv. Here's the output, with one changed line removed for length (marked …):

[old] duplicate key ('Liquid Web', 'Managed VPS 4 GB') appears 2x
[old] duplicate key ('Contabo', 'Cloud VPS 4 (EUR)') appears 2x
old: 16 unique plans, new: 19 unique plans

~ changed  SiteGround StartUp (same plan): monthly 24.99 -> 17.99
~ changed  Hostinger Web Premium (replaces Cloud Startup): intro 7.99 -> 2.99; prepay 48.0 -> 12.0; monthly 27.99 -> 11.99; renewal 25.99 -> 11.99; total_36 287.64 -> 323.64
~ changed  Hostinger KVM 1 (replaces KVM 2): intro 8.79 -> 6.49; monthly 24.49 -> 19.49; renewal 14.99 -> 19.49; total_36 390.84 -> 389.64
~ changed  InMotion VPS 4 vCPU (replaces VPS 4 (Most Popular)): intro 10.99 -> 13.99; monthly None -> 26.99; total_36 587.64 -> 659.64
~ changed  ScalaHosting Managed Cloud #1 (replaces Managed Build #1): prepay 36.0 -> 1.0; total_36 1078.2 -> 1953.2
+ added    ScalaHosting Unmanaged Config #1
~ changed  WP Engine Startup (same plan): intro 30.0 -> 28.0; total_36 1200.0 -> 1176.0
~ changed  Contabo Cloud VPS 4 (EUR) (same plan): intro 4.4 -> 5.28; monthly None -> 6.6; renewal 5.5 -> 6.6; total_36 None -> 205.92
…
+ added    SiteGround Cloud Jump Start (4 CPU / 8 GB / 40 GB)
+ added    Cloudways Autonomous Growth

The omitted line is the Rocket Starter row, where prepay goes from 1 to 12.

Sixteen unique plans become nineteen, with three real additions and nothing removed. That matches the changelog, which lists exactly those three as new rows.

What the naive version would have told you

To see what normalization buys, replace the body of norm() with return text.strip() and run it again. The added/removed lines become:

+ added    ScalaHosting Unmanaged Config #1
+ added    Contabo Cloud VPS 4 (EUR)
+ added    Cloudways DigitalOcean 4 GB Basic (premium CPU)
+ added    SiteGround Cloud Jump Start (4 CPU / 8 GB / 40 GB)
+ added    Cloudways Autonomous Growth
- removed  Contabo Cloud VPS 4 (EUR)
- removed  Cloudways DigitalOcean 4 GB Basic (premium CPU)

That's seven lines where the true answer is three. Two unchanged plans show up as churn because their August names contain a no-break space. The printed names look identical in the terminal, which is exactly what makes this bug hard to spot. Drop the REPLACES map as well and four more plans would appear as removed-plus-added pairs, leaving you with a report that's mostly noise.

A diff tells you where to look, not why

Here's the part I think matters most. The script reports that values changed, but it can't tell a vendor price change from a correction in the dataset, and in this pair of snapshots most of the movement is corrections.

SiteGround StartUp monthly 24.99 -> 17.99 isn't a price cut. The changelog says the September cycle started recording monthly and renewal as a single figure, the price a plan reverts to after its intro term. Under that definition the figure is $17.99. (SiteGround's own pricing page still showed $2.99 for a 12-month first term, renewing at $17.99, when I checked it on 2026-10-28.)
InMotion intro 10.99 -> 13.99 is a correction. The September note says $10.99 belonged to a shared-hosting tier, not the VPS.
The two Hostinger rows were swapped for plans closer to the 4 GB baseline.
Contabo was corrected to a VAT-inclusive 24-month effective rate.

The rest (WP Engine's intro rate, ScalaHosting's prepay term, the Rocket prepay term) aren't explained line by line in the changelog. For those, the diff gives you a short list of rows to check against each vendor's page, and the vendor page is the tiebreaker.

That's a good reason to keep editorial context, meaning notes and a changelog, next to any hand-collected dataset. A diff turns two hundred cells into ten lines worth reading. A human still has to say what each line means. I wrote about the collection process and the mistakes that shaped it on Medium, if you want the background.

Using it on your own data

The script is short enough to adapt rather than configure:

Change the key in keyed() to whatever uniquely identifies a row in your data. If there's an ID column, use it. The current site export at /data/vps-pricing-index.csv has a stable id column (hostinger-kvm-1, siteground-startup and so on), which removes the whitespace problem entirely. You'd still map deliberately replaced plans by hand.
Keep REPLACES small and sourced. Add a comment next to each entry saying where the mapping came from.
Fail loudly on non-identical duplicates. Swap the print in keyed() for an exception once you've checked that existing duplicates are harmless.
Run it in CI whenever the upstream file changes, and post the output as a pull-request comment. Reviewing ten lines is a lot easier than reviewing a CSV diff.

The site's tools page lists the CSV and JSON exports if you want the current snapshot instead of the git history. Prices in both are dated (the latest verification was 2026-09-15), so check the vendor's page before you act on any of them.

Carter Wang is a serial founder who has bought hosting and run production stacks for more than a decade. He maintains the VPS Pricing Index at VPS Hosting Radar.

DEV Community: Carter Wang