A row count can stay the same even when a merge drops one participant and duplicates another. If your analysis expects one row per participant, compare the identifier counts before and after the join.
This example uses fictional identifiers. The two files have the same number of rows, but the second file lost P03 and has an extra P02.
from collections import Counter
before = Counter(["P01", "P02", "P03"])
after = Counter(["P01", "P02", "P02"])
lost = before - after
extra = after - before
assert sum(before.values()) == sum(after.values()) == 3
assert dict(lost) == {"P03": 1}
assert dict(extra) == {"P02": 1}
print(f"Lost rows: {dict(lost)}")
print(f"Extra rows: {dict(extra)}")
The output is Lost rows: {'P03': 1} and Extra rows: {'P02': 1}.
Before interpreting any summary, inspect the join keys on both sides. If your data intentionally has one row per participant and visit, count the pair (participant_id, visit_id) instead. Keep the row-count and key-count checks in the same validation step.
Top comments (0)