I was pulling data from SAP through an OData API — every Internal Order from 2020 to today. The response came back with exactly 1000 rows. Not 987, not 1214. A clean, round 1000.
Was I happy? No. I got suspicious.
In data work, a number that "too perfect" is usually a lie. Real data comes back messy — fractions, odd totals, nothing that lines up so neatly. That 1000 wasn't telling me "here's your data." It was telling me "this is a ceiling."
I tried the standard fix: paginate using OData's own $skip and $top parameters. SAP's gateway ignored both completely and kept returning the exact same first 1000 rows, as if the parameters didn't exist. There was no __next link in the response either — nothing pointing to a next page.
So I tried a different check. I ran $count (a request that returns just the row count, no data) over the same date range — and got 1000 again. Even the count itself was capped by the same limit. I split the range in half and ran $count on each half separately: 938 on the first half, 651 on the second. Real total: 1589. Not 1000.
589 rows would have quietly disappeared, and nobody would have known — if I hadn't been suspicious of a number that looked too clean.
The lesson: any time an API can return a lot of data, assume it's silently capped until you prove otherwise. A round number isn't a sign of success — it's the first flag that something needs verification. And trusting the standard protocol alone isn't enough: here, SAP's own gateway simply didn't honor OData's standard pagination mechanism. The fix is building independent verification — like splitting time windows and cross-checking partial counts — before you trust any number an API hands you.
Have you run into this? An API silently truncating your data without telling you?
Top comments (0)