A spreadsheet contains a customer named Emma. The message preview says Mia. The import did not raise an error. What went wrong? With duplicate CSV headers, converting a row into a dictionary can silently discard the earlier value. The same class of silent failure can occur when a parser accepts a row that violates the application’s schema.
For an Excel or CSV-based WhatsApp sender, a successful import is not enough: the rendered message must match the intended recipient data. This article builds and tests a small pre-send validator instead of testing a messaging service.
This is an offline test of the boundary between a spreadsheet and a personalized message preview. It compares three small Python implementations on identical synthetic inputs. It does not test a WhatsApp sender, its delivery mechanism, or the official WhatsApp Cloud API.
Technical evidence from GitHub and Stack Overflow
| Technical discussion | What was reported | Test implication |
|---|---|---|
| Papa Parse #1007 | Quoted headers containing commas were misinterpreted in version 5.4.1. | Preserve field identity and CSV quoting. |
| Papa Parse #1052 | Similar comma-containing headers interfered with duplicate-header handling. | Parse before validating field names. |
| Papa Parse #1071 | A report involving quoted values and a trailing carriage return. | Treat record boundaries and quoting explicitly. |
| CPython #61739 | Duplicate column names can overwrite earlier values in DictReader. | Reject duplicate names before building a dictionary. |
| pandas #54665 | Numeric type inference stripped leading zeroes; using a string dtype resolved the reporter's problem. | Preserve identifiers as strings. |
| Stack Overflow #75941839 | A UTF-8 byte-order mark appeared in the first DictReader key; the asker confirmed that UTF-8-sig decoding worked. | Test BOM handling before named-field lookup. |
| Stack Overflow #27780542 | A developer reported that attempted line breaks did not render as intended in a WhatsApp sharing workflow. | Distinguish CSV newline preservation from later URL encoding and client rendering. |
These seven selected discussions are not seven independent defects. Among the five GitHub issue reports, the two quoted-header reports were addressed under the same header-renaming fix in the Papa Parse 5.5.0 changelog. The changelog also documents a BOM-related header fix in 5.5.4. The quoted-value report is a reported issue, not proof of a defect in every version. Python tests below are inspired by the failure modes; they do not reproduce the exact Papa Parse bugs or benchmark that JavaScript library. Only the CPython duplicate-header behavior and pandas type-inference behavior are directly exercised with those libraries. The other fixtures are independently constructed, including basic controls. The Stack Overflow BOM behavior is directly exercised with synthetic CSV data. The WhatsApp sharing discussion motivates a separate boundary: the local test preserves message text but does not reproduce the reported sharing-link behavior or validate WhatsApp clients. These sources are a purposive technical review, not a frequency study.
Define the output before testing the parser
name,booking_id,appointment_time
Emma,0007,14:30
Expected message preview:
Hi Emma, booking 0007 is at 14:30.
Reply CONFIRM or RESCHEDULE.
These are fictional values. The placeholders use Python's string.Template syntax; they are not a commercial product's template syntax. A correct result must preserve the entire text, not merely contain the name.
Build a pre-send validation pipeline
The validator is intentionally separate from the sender. Fail before returning any previews if the input schema is invalid; review business rules and recipient eligibility separately.
Compare three CSV-to-message implementations
The first implementation is deliberately simplistic: it splits text on lines and commas. The second uses csv.DictReader with its default handling of field lengths and duplicate names and ordinary UTF-8 decoding. The third uses a CSV reader with explicit UTF-8 BOM handling, unique and required header checks, strict row-length checks and nonblank required fields. It returns previews only after the entire file has been processed successfully.
Here, MATCH means the complete preview equals the expected output; REJECT means the invalid fixture triggers the exact intended validation error. ERROR is an unintended exception, WRONG is a wrong preview for valid data, and ACCEPTED means an invalid fixture was not rejected. Unexpected exceptions are deliberately not counted as successful validation. The comparison measures correctness on this set, not parsing speed or production reliability.
Save the script below as compare_csv.py. It uses the standard library for the main experiment and pandas 2.2.3 for the separate type-inference control (python -m pip install pandas==2.2.3). Run python compare_csv.py.
import csv
import io
import sys
from string import Template
TEMPLATE = Template('Hi $name, booking $booking_id is at $appointment_time.\n\nReply CONFIRM or RESCHEDULE.')
REQUIRED = {'name', 'booking_id', 'appointment_time'}
def naive(raw):
lines = raw.decode('utf-8').splitlines()
headers = lines[0].split(',')
return [TEMPLATE.substitute(dict(zip(headers, line.split(',')))) for line in lines[1:]]
def dict_reader(raw):
rows = csv.DictReader(io.StringIO(raw.decode('utf-8'), newline=''))
return [TEMPLATE.substitute(row) for row in rows]
def validated(raw):
rows = csv.reader(io.StringIO(raw.decode('utf-8-sig'), newline=''), strict=True)
headers = next(rows, [])
if len(headers) != len(set(headers)):
raise ValueError('duplicate header')
if not REQUIRED.issubset(headers):
raise ValueError('missing required column')
result = []
for fields in rows:
if len(fields) != len(headers):
raise ValueError('wrong field count')
row = dict(zip(headers, fields))
if any(not row[key].strip() for key in REQUIRED):
raise ValueError('blank required value')
result.append(TEMPLATE.substitute(row))
return result
HEADER = 'name,booking_id,appointment_time\n'
BASE = HEADER + 'Emma,0007,14:30\n'
EXPECTED = ['Hi Emma, booking 0007 is at 14:30.\n\nReply CONFIRM or RESCHEDULE.']
CASES = [
('Normal', BASE, EXPECTED),
('Reordered', 'booking_id,appointment_time,name\n0007,14:30,Emma\n', EXPECTED),
('Quoted comma', HEADER + '"Lee, Mia",0007,14:30\n', [EXPECTED[0].replace('Emma', 'Lee, Mia')]),
('Quoted newline', HEADER + '"Emma\nLee",0007,14:30\n', [EXPECTED[0].replace('Emma', 'Emma\nLee')]),
('Escaped quotes', HEADER + '"Emma ""E"" Lee",0007,14:30\n', [EXPECTED[0].replace('Emma', 'Emma "E" Lee')]),
('UTF-8 BOM', '\ufeff' + BASE, EXPECTED),
('Unicode', HEADER + 'José,0007,14:30\n', [EXPECTED[0].replace('Emma', 'José')]),
('Leading-zero ID', HEADER + 'Emma,000001,14:30\n', [EXPECTED[0].replace('0007', '000001')]),
('Duplicate header', 'name,name,booking_id,appointment_time\nEmma,Mia,0007,14:30\n', 'duplicate header'),
('Missing column', 'nam,booking_id,appointment_time\nEmma,0007,14:30\n', 'missing required column'),
('Blank time', HEADER + 'Emma,0007,\n', 'blank required value'),
('Whitespace time', HEADER + 'Emma,0007, \n', 'blank required value'),
('Short row', HEADER + 'Emma,0007\n', 'wrong field count'),
('Extra field', HEADER + 'Emma,0007,14:30,unexpected\n', 'wrong field count'),
]
def score(fn, raw, expected):
try:
actual = fn(raw.encode('utf-8'))
except (ValueError, KeyError, csv.Error) as exc:
if isinstance(expected, str) and isinstance(exc, ValueError) and str(exc) == expected:
return 'REJECT'
return 'ERROR'
if isinstance(expected, list) and actual == expected:
return 'MATCH'
return 'WRONG' if isinstance(expected, list) else 'ACCEPTED'
if __name__ == '__main__':
methods = [naive, dict_reader, validated]
print('Python', sys.version.split()[0])
print('Case | split | DictReader | validated')
counts = [0,0,0]
for label, raw, expected in CASES:
results = [score(fn, raw, expected) for fn in methods]
print(' | '.join([label] + results))
for i, result in enumerate(results):
counts[i] += result in ('MATCH', 'REJECT')
assert results[-1] == ('MATCH' if isinstance(expected, list) else 'REJECT')
print('Exact outcomes:', counts, '/', len(CASES))
# A current booking value is not present in the CSV. All previews remain stale.
latest = ['Hi Emma, booking 0007 is at 15:00.\n\nReply CONFIRM or RESCHEDULE.']
assert all(fn(BASE.encode('utf-8')) != latest for fn in methods)
print('Stale source control: all three render 14:30; latest booking is 15:00.')
# Separate reproduction of the pandas type-inference behavior.
import pandas as pd
inferred = pd.read_csv(io.StringIO(BASE)).loc[0, 'booking_id']
preserved = pd.read_csv(io.StringIO(BASE), dtype={'booking_id': str}).loc[0, 'booking_id']
assert str(inferred) == '7' and preserved == '0007'
print('pandas', pd.__version__, ': default =', str(inferred), '; explicit string =', preserved)
Actual results
Executed with Python 3.13.5 and pandas 2.2.3. All inputs are synthetic. No external messaging services or customer records are used.
Python 3.13.5
Case | split | DictReader | validated
Normal | MATCH | MATCH | MATCH
Reordered | MATCH | MATCH | MATCH
Quoted comma | WRONG | MATCH | MATCH
Quoted newline | ERROR | MATCH | MATCH
Escaped quotes | WRONG | MATCH | MATCH
UTF-8 BOM | ERROR | ERROR | MATCH
Unicode | MATCH | MATCH | MATCH
Leading-zero ID | MATCH | MATCH | MATCH
Duplicate header | ACCEPTED | ACCEPTED | REJECT
Missing column | ERROR | ERROR | REJECT
Blank time | ACCEPTED | ACCEPTED | REJECT
Whitespace time | ACCEPTED | ACCEPTED | REJECT
Short row | ERROR | ACCEPTED | REJECT
Extra field | ACCEPTED | ACCEPTED | REJECT
Exact outcomes: [4, 7, 14] / 14
Stale source control: all three render 14:30; latest booking is 15:00.
pandas 2.2.3 : default = 7 ; explicit string = 0007
| Implementation | Exact expected outcomes | Valid fixtures matched | Invalid fixtures explicitly rejected |
|---|---|---|---|
| Naive split | 4/14 | 4/8 | 0/6 |
| DictReader without additional validation | 7/14 | 7/8 | 0/6 |
| CSV parser with schema checks | 14/14 | 8/8 | 6/6 |
The third implementation matched all fourteen predefined expectations. That does not establish universal correctness: the fixtures are small, hand-selected, and the expected outcomes are limited to the demonstrated schema. A raw parser also has a different contract from a domain validator.
Why parsing is not schema validation
Decode before matching header names
The Stack Overflow BOM discussion shows why a visually familiar header can be a different string after decoding. In our BOM fixture, ordinary UTF-8 decoding leaves U+FEFF attached to the first header. The validated implementation uses UTF-8-sig so the expected name can be matched. This does not imply support for arbitrary file encodings.
Preserve line breaks at the right boundary
The WhatsApp line-break discussion on Stack Overflow concerns a later stage: passing text into a sharing link. Our quoted-newline fixture only checks that the parser and local preview preserve the original characters. It does not establish that any WhatsApp integration renders them correctly. If your integration uses URLs, test URL encoding and client rendering separately.
Duplicate headers can change a message without raising an error
name,name,booking_id,appointment_time
Emma,Mia,0007,14:30
The default DictReader mapping keeps the later name value, so the preview addresses Mia rather than Emma. This directly exercises the behavior in CPython #61739. If your application relies on named fields, reject ambiguous headers before creating dictionaries. Renaming duplicates automatically can be appropriate for data exploration, but not when field identity is a business requirement.
Identifiers are not quantities
In the separate pandas test, default inference converted the booking ID 0007 to 7. Explicitly declaring the column as a string preserved 0007:
pd.read_csv(io.StringIO(BASE), dtype={'booking_id': str})
This reproduces the behavior discussed in pandas #54665 with a new synthetic input. Treat identifiers as strings when their literal representation matters. If a spreadsheet has already lost digits before exporting, this setting cannot reconstruct them. Phone numbers also require their own normalization and validity checks; preserving text is only one step.
A valid CSV can still be stale
The separate negative control changes the expected appointment time to 15:00 while leaving the CSV at 14:30. All three implementations continue rendering the old time. The validator has no current calendar to compare against. This control is intentionally excluded from the fourteen-case correctness score: no method detects stale source data.
Where this fits before a WhatsApp Sender
Export CSV -> parse records -> validate schema -> render previews
-> verify current business data and recipient eligibility
-> separately approve any sending operation
The code does not validate phone numbers, appointment date formats, time zones, consent, duplicate recipients, message delivery or customer replies. Newlines inside a field are tested for preservation, not whether they are appropriate in a person's name. For production, define those policies separately and replace or extend the fixtures with inputs relevant to your own system.
What to add to your own regression suite
Start with a valid row and a reordered-header control. Test quoted delimiters, embedded newlines, escaped quotes and BOM decoding. Then add negative cases for missing, duplicate and blank fields, short records and extra values. Compare the entire rendered text for valid fixtures and the precise error for invalid ones. Finally, include a stale-data control to show what this validator cannot know.
A successful parse, a correct preview, and current business data are separate guarantees. A WhatsApp sender workflow should not infer the latter two from the first.



Top comments (0)