A scrape can return several reviews while its CSV contains one data row. That does not necessarily mean the scraper found only one review. The exporter may have serialized the whole response, including its results array, into a single spreadsheet cell.
Fix the record boundary before changing the file extension. If the consumer needs to filter reviews, each exported row should represent a review. A run-level object containing reviews is a different record type.
Here is a small Python adapter that makes that choice explicit and checks its own serialized output. The data is fictional. This is an application-side conversion example, not evidence of a BrowserAct Bot or customer portal test.
Choose a collection, not every array you can find
Suppose the agreed input shape is an object with a data array. Each item has a string review_id and a comment that is either text or null. Other fields remain in the original JSON and are deliberately outside this table's contract.
Do not recursively expand every array. If a parent contains both variants and reviews, combining them can multiply rows and change what the dataset counts. Choose one collection, document its row type, and preserve identifiers for relationships you need later.
This adapter rejects an unexpected shape rather than guessing which array contains the intended records. It also rejects duplicate identifiers rather than silently deciding which version to keep.
Preserve missing values explicitly
CSV does not supply an application-level null convention. An empty comment and an unavailable comment can look identical after a naive conversion.
For this example, a separate comment_missing column records the distinction. An empty string produces an empty comment with comment_missing=false; null produces an empty comment with comment_missing=true. Other conventions can work, provided the receiving application understands them.
The Python CSV documentation explains DictWriter, DictReader, and opening file objects with newline="". Use the library to handle delimiters, quotes, and embedded line breaks instead of splitting text on commas.
import csv
import io
def review_csv(payload):
if not isinstance(payload, dict):
raise ValueError("expected an object")
records = payload.get("data")
if not isinstance(records, list):
raise ValueError("expected a data array")
rows = []
seen = set()
for record in records:
if not isinstance(record, dict):
raise ValueError("expected review objects")
key = record.get("review_id")
if not isinstance(key, str) or not key.strip():
raise ValueError("expected a nonblank string ID")
if key in seen:
raise ValueError("duplicate review ID")
if "comment" not in record:
raise ValueError("comment field absent")
comment = record["comment"]
if comment is not None and not isinstance(comment, str):
raise ValueError("comment must be text or null")
seen.add(key)
rows.append({
"review_id": key,
"comment": "" if comment is None else comment,
"comment_missing": "true" if comment is None else "false",
})
stream = io.StringIO(newline="")
writer = csv.DictWriter(
stream,
fieldnames=["review_id", "comment", "comment_missing"],
)
writer.writeheader()
writer.writerows(rows)
output = stream.getvalue()
parsed = list(csv.DictReader(io.StringIO(output, newline="")))
if parsed != rows:
raise ValueError("CSV round-trip changed the projected records")
return output
The function validates all records before writing output. A malformed last record therefore cannot cause this function to return a partially accepted dataset. An empty data array returns a header-only CSV, representing zero review records.
Exercise the awkward values
Use an input with punctuation and a line break before trusting a conversion built around a clean demo:
payload = {
"request_metadata": {"requested_count": 3},
"data": [
{"review_id": "001", "comment": 'Quiet, "bright" room\nGood desk'},
{"review_id": "002", "comment": None},
{"review_id": "003", "comment": ""},
],
}
text = review_csv(payload)
rows = list(csv.DictReader(io.StringIO(text, newline="")))
assert len(rows) == 3
assert rows[0]["review_id"] == "001"
assert rows[0]["comment"] == payload["data"][0]["comment"]
assert rows[1]["comment_missing"] == "true"
assert rows[2]["comment_missing"] == "false"
The embedded newline belongs to one field. Counting physical lines would give the wrong record count. Parse the CSV and count records instead.
Also try a duplicate ID, an absent comment, a numeric ID, a nested object used as a comment, and a response whose data value is an object. Each should fail this contract. Those failures do not prove the source data is wrong; they show that it needs a different, explicitly chosen mapping.
Verify the handoff separately
The round-trip check establishes that Python can recover the projected string fields from the generated CSV. It does not establish that the source page was scraped correctly or that a spreadsheet application will preserve its interpretation.
When importing, set identifier columns to text and check a leading-zero identifier afterward. Retain the original JSON so omitted fields and relationships remain available. If the consumer needs nested data or typed application values, delivering JSON alongside the CSV may be the clearer contract.
For browser-collected data, keep three checks separate: the task ran, the file is valid, and the dataset answers the requested question. This adapter addresses one part of the last two checks. It does not replace source sampling, coverage checks, or a real workflow acceptance test.
I work on BrowserAct. This tutorial develops the record-boundary example in our CSV vs JSON guide. Its examples are synthetic and intentionally limited to the stated review schema.
Top comments (0)