A repeatable Python workflow for cleaning CSV files before CSV-to-JSON conversion

Messy CSV imports commonly fail because headers vary, blank cells are inconsistent, duplicate records remain, and values such as dates and email addresses have not been normalized. A safe workflow is: preserve the source, standardize headers, trim text fields, normalize missing values, deduplicate against explicit columns, validate critical fields, then export both cleaned CSV and JSON.

import csv, json
from pathlib import Path
rows=[]; seen=set()
with open('input.csv', newline='', encoding='utf-8-sig') as f:
    for r in csv.DictReader(f):
        r={str(k).strip().lower().replace(' ','_'): (v or '').strip() for k,v in r.items()}
        key=(r.get('email','').lower(), r.get('name','').lower())
        if key not in seen:
            seen.add(key); rows.append(r)
with open('clean.json','w',encoding='utf8') as f: json.dump(rows,f,indent=2)

Choose a deduplication key that matches the business meaning of a record; never silently merge records just because two names match. For a small dataset where a human needs to inspect malformed headers, duplicates, dates, encoding, and validation exceptions, an optional 24-hour flat-rate cleanup service is available at CSV Cleanup Kit.