Two product exports had the same ASIN in each row. A join treated that as one offer and inflated the result. An ASIN may identify the product you want to follow, but an observation also belongs to a marketplace. The Amazon Scraper README describes different marketplaces and exported product fields. A test for the downstream join can catch a mixed market even when both inputs separately look valid.
Written with AI. The records and assertions below were tested locally. The cover illustration is AI-generated.
Make a fixture whose incorrect result looks plausible
Put one ASIN in two markets. Use different amounts, and make one market missing from the second snapshot. A join by ASIN alone can find a match for that missing market and silently copy the wrong amount. The desired join uses both marketplace and ASIN. The fixture also checks that ambiguous repeated keys cause an explicit failure.
The example keeps observed price strings without converting them or computing a cross-currency difference. It’s testing identity and the handling of absent observations. Save as test_market_join.py and run with Python.
def rows_by_offer(rows):
result = {}
for row in rows:
offer = row["marketplace"], row["asin"]
if offer in result:
raise ValueError(f"Duplicate offer key: {offer}")
result[offer] = row
return result
def paired(left, right):
old, new = rows_by_offer(left), rows_by_offer(right)
return [(key, old.get(key), new.get(key)) for key in sorted(old.keys() | new.keys())]
us = {"marketplace": "US", "asin": "TEST", "price": "15.00", "currency": "USD"}
uk = {"marketplace": "UK", "asin": "TEST", "price": "17.00", "currency": "GBP"}
us_now = {**us, "price": "16.00"}
assert paired([us, uk], [us_now]) == [(('UK', 'TEST'), uk, None),
(('US', 'TEST'), us, us_now)]
try:
paired([us, us], [us_now])
except ValueError as exc:
assert "Duplicate offer key" in str(exc)
else:
raise AssertionError("Duplicate key was accepted")
print("UK stays missing. US pairs with US. Duplicate key rejected.")
The local fixture produced the expected two comparisons. The UK row stayed missing on the right rather than matching the US row, and the duplicate was rejected. For a real pipeline that keeps several sellers or time snapshots per market, extend the key with the relevant fields. State that level of detail before deciding whether a duplicate is an error or a legitimate extra observation.
When tuning a report, test the market with missing data first. That’s where a loose join can appear to solve an absence and quietly create a wrong result.
Use a comparison summary someone can reconcile
For this fixture I’d report one paired offer and one observation absent from the newer file. A summary with only the price delta would silently omit the missing record. Keep coverage beside the delta, and record how the source determined marketplace membership. Sorting by an identifier after an ambiguous join may make the values look stable while still pairing the wrong rows. A failing test should show the two context keys as well as the old and new amounts.
Further reading
- Detecting rank ties in benchmarks to make the rules behind a reported ranking explicit.
- Testing a benchmark CSV with reordered columns to make an import depend on its headers instead of positions.
Top comments (0)