DEV Community

Cover image for I Compared Amazon Data APIs by Field. The Checkboxes Were Lying.
Pangolinfo
Pangolinfo

Posted on

I Compared Amazon Data APIs by Field. The Checkboxes Were Lying.

#api #data

Every "best Amazon data API" article has the same table.

Vendor | Product Data | Reviews | Search | Price Monitoring
-------|--------------|---------|--------|-----------------
A      | ✓            | ✓       | ✓      | ✓
B      | ✓            | ✓       | ✓      | ✓
C      | ✓            | ✗       | ✓      | ✓
Enter fullscreen mode Exit fullscreen mode

A and B look identical. You pick based on price, or whoever replied to your email fastest.

Here is the thing I learned after actually running the comparison: one of those vendors returned 12 fields. The other returned 58. Same checkmark.

This post is the method I used to find that out. It takes two days, it needs no cooperation from any vendor, and the script is about 90 lines. I am publishing it because the alternative — reading vendor docs and hoping — does not work.

Related: if you have not already, I stopped trusting "real-time" claims after one bad quarter covers the reliability side of this. This post goes one level deeper: not is it fast, but is it complete.

One scope note, because it removes an entire category of pointless comparison: everything below is about vendors competing for the same job. The official SP-API is not one of them — its authorisation model binds to a seller account, not to the marketplace, so competitor data and market-level rank were never in scope. That is a design boundary, not a missing feature. If your requirements sit near that line, read Amazon API vs Web Scraping: Stop Choosing, Build Both Routes first; it will tell you whether you need this post at all.

Why "supported" is an information black hole

A feature grid uses a binary. Real delivery is continuous, and it has at least three layers:

Layer 1: does the schema have the field?        → field coverage
Layer 2: when it has it, does it have a value?  → non-null fill rate
Layer 3: how many records pass both?            → usable record rate
Enter fullscreen mode Exit fullscreen mode

Only layer 3 has anything to do with your business. Layers 1 and 2 are diagnostics.

Let me show you why layer 2 matters more than most people assume.

The null field problem

Here is a real response shape, with values removed:

{
  "asin": "B0C7V9K2XQ",
  "title": "...",
  "coupon": null,
  "deliveryEstimate": null,
  "price": { "current": 49.99, "original": null }
}
Enter fullscreen mode Exit fullscreen mode

Schema validation passes. JSON parsing succeeds. Row count is correct. HTTP status is 200. Your monitoring is green.

And your coupon dimension is permanently empty.

I know a team that signed an annual contract after seeing 60+ fields and 92% coverage. Three months in, the coupon column was empty in most reports and delivery estimates were empty in most reports. Both fields existed in the schema. Both were populated less than 15% of the time. On an annual deal, switching mid-term was expensive.

Ten minutes with the script below would have caught it.

A counterintuitive result

A vendor with 95% coverage and 60% fill rate is usually worth less to you than one with 70% coverage and 98% fill rate.

The first hands you a pile of nulls. You write defensive if (x != null) branches in every downstream consumer, and you still cannot tell "this product has no coupon" from "we failed to fetch the coupon."

The second hands you records that are solid end to end. Fewer dimensions, every one of them trustworthy.

Certain absence beats false completeness. Low coverage means "I do not have this dimension," which you can plan around. Low fill rate means "I think I have this dimension and I do not," which produces bad decisions.

The three response shapes

Feed the same ASINs to different vendors and responses cluster into three shapes. This taxonomy is more useful than any ranking.

Shape A — thin, 12–18 fields 

{
  "asin": "B0C7V9K2XQ",
  "title": "...",
  "brand": "...",
  "price": 49.99,
  "currency": "USD",
  "rating": 4.3,
  "reviewCount": 1284,
  "availability": "In Stock",
  "mainImage": "https://...",
  "bsr": 1842,
  "category": "Electronics",
  "url": "https://..."
}
Enter fullscreen mode Exit fullscreen mode

Good for: price monitoring, basic sourcing. Useless for anything involving variants, promotions, ad placement or seller dimensions.

Shape B — mid, 30–40 fields, variants but no ad flags 

{
  "asin": "B0C7V9K2XQ",
  "parentAsin": "B0C7V90001",
  "price": {"current": 49.99, "original": 69.99, "currency": "USD"},
  "coupon": {"amount": 5.00, "type": "percentage"},
  "availability": {"status": "In Stock", "deliveryEstimate": "..."},
  "rating": {"value": 4.3, "count": 1284,
             "histogram": {"5": 62, "4": 21, "3": 9, "2": 4, "1": 4}},
  "bsr": [{"category": "Electronics", "rank": 1842},
          {"category": "Portable Audio", "rank": 97}],
  "variants": [
    {"asin": "B0C7V9K2XQ", "attributes": {"color": "Black", "size": "256GB"},
     "price": 49.99, "availability": "In Stock"},
    {"asin": "B0C7V9K2XR", "attributes": {"color": "White", "size": "512GB"},
     "price": 79.99, "availability": "In Stock"}
  ],
  "seller": {"name": "...", "rating": 4.6, "isFulfilledByAmazon": true},
  "images": [{"url": "...", "variant": "MAIN"}],
  "badges": ["Amazon's Choice"]
}
Enter fullscreen mode Exit fullscreen mode

Good for: sourcing, variant analysis, rating distribution. Useless for rank attribution, because ads and organic results are indistinguishable.

Shape C — complete 

{
  // everything in shape B, plus:
  "variants": [
    {"asin": "B0C7V9K2XQ", "attributes": {"color": "Black", "size": "256GB"},
     "price": {"current": 49.99, "original": 69.99},
     "availability": "In Stock", "rating": {"value": 4.3, "count": 812},
     "isBuyBoxWinner": true, "offerCount": 7}
  ],
  "isSponsored": false,
  "organicPosition": 1,
  "searchContext": {"keyword": "...", "page": 1, "positionOnPage": 3,
                    "sponsoredCountAhead": 2,
                    "adTypesAhead": ["SB", "SP"]},
  "priceHistory": {"min90d": 44.99, "max90d": 69.99},
  "subscribeAndSave": {"discount": 5},
  "buyBoxWinner": {"seller": "...", "price": 49.99, "shipping": "FREE"}
}
Enter fullscreen mode Exit fullscreen mode

Good for: rank attribution, ad-competitive analysis, variant-level insight, historical price ranges.

On a feature grid these three are the same checkmark. In your reporting, you are not buying data — you are buying which questions you are allowed to ask.

The method

Step 1: write the field contract

Not vendor docs. Your own list, with priorities.

  • P0 — the record is dead without it. Goes to the dead-letter queue.
  • P1 — analysis gets shallower, still works.
  • P2 — nice to have. 
CONTRACT = {
    "P0": ["asin", "title", "price.current", "availability.status",
           "rating.value", "rating.count", "bsr", "variants"],
    "P1": ["parentAsin", "brand", "price.original", "price.currency",
           "coupon", "seller.name", "offerCount", "images",
           "badges", "isSponsored", "categoryPath"],
}
Enter fullscreen mode Exit fullscreen mode

Eight to fifteen P0 fields is the sweet spot. The test is one question: if this field were missing, would you dead-letter the record? Yes means P0.

Per object, the critical fields differ:

Object Critical fields Why
Product price.current, bsr, variants pricing and sourcing fundamentals
Search page, items[].position, items[].isSponsored, items[].adType, items[].organicRank without the last three, your rank tracker is sorting ads and organic together
Reviews reviewId, asin (child), parentAsin, variant, rating, date, verifiedPurchase variant decides whether insight can be pinned to a configuration

Step 2: the flatten function (this is the whole trick) 

def flatten(obj, prefix=""):
    """Flatten to a.b.c paths; sample list items; empty values are dropped."""
    out = {}
    if isinstance(obj, dict):
        for k, v in obj.items():
            out.update(flatten(v, f"{prefix}.{k}" if prefix else k))
    elif isinstance(obj, list):
        for v in obj[:50]:
            out.update(flatten(v, f"{prefix}[]"))
    elif obj is not None and obj != "":
        out[prefix] = obj          # <-- the entire trick
    return out
Enter fullscreen mode Exit fullscreen mode

That last elif is the whole method. Empty values never enter the result. So when you later ask "which contract fields came back," you are asking "which contract fields came back with a value."

One function, two metrics. Coverage comes from the count. Fill rate is handled implicitly — the coupon: null case from earlier simply never lands in out.

Step 3: evaluate and aggregate 

import statistics

def evaluate(record, cov_threshold=0.8):
    flat = flatten(record)
    contract = CONTRACT["P0"] + CONTRACT["P1"]
    present = [f for f in contract if f in flat]
    coverage = len(present) / len(contract)
    p0_ok = all(f in flat for f in CONTRACT["P0"])   # P0 must be 100% populated
    return {
        "coverage": round(coverage, 3),
        "usable": bool(p0_ok and coverage >= cov_threshold),
    }

def run(records, spend_usd):
    ev = [evaluate(r) for r in records]
    usable = [e for e in ev if e["usable"]]
    return {
        "samples": len(ev),
        "usable_rate": round(len(usable) / len(ev), 4) if ev else 0,
        "avg_coverage": round(statistics.mean(e["coverage"] for e in ev), 3),
        "cost_per_1k_usable": round(spend_usd / len(usable) * 1000, 2)
                              if usable else None,
    }
Enter fullscreen mode Exit fullscreen mode

p0_ok uses all(), not a ratio. P0 is a hard constraint — half present is not usable.

Step 4: classify failures 

def classify(record):
    if record.get("_http_status") != 200:
        return "hard"        # visible, retry it
    if not record.get("asin"):
        return "soft"        # 200 but empty body
    flat = flatten(record)
    if not all(f in flat for f in CONTRACT["P0"]):
        return "partial"     # SILENT — the expensive one
    return "ok"
Enter fullscreen mode Exit fullscreen mode
Type Symptom Visibility Cost
Hard non-200, timeout Fully visible Low
Soft 200, empty body Half visible Medium
Partial 200, valid structure, P0 null Silent Highest
Stale values present but old Silent High

Row three is why this post exists. It raises no error, trips no alert, and generates no billing dispute. It just quietly degrades your reporting.

Step 5: what the output looks like 

=== Vendor A ===
  samples            : 300
  avg coverage       : 0.94
  usable rate        : 61.33%
  cost per 1k usable : $12.40
  failures           : {'ok': 184, 'partial': 97, 'soft': 14, 'hard': 5}

=== Vendor B ===
  samples            : 300
  avg coverage       : 0.71
  usable rate        : 92.67%
  cost per 1k usable : $4.85
  failures           : {'ok': 278, 'partial': 12, 'soft': 8, 'hard': 2}
Enter fullscreen mode Exit fullscreen mode

A has much better coverage and two-thirds the usable rate, at 2.5x the cost per usable record.

If you only look at coverage — or at a feature grid — you pick A. Look at usable rate and the conclusion inverts.

Note vendor A's 97 partial records. Measured the traditional way, A's success rate is (184 + 97 + 14) / 300 = 98.3%. Looks excellent. With a field contract applied, real usable rate is 61%.

That gap is exactly where conventional monitoring fails. It confuses "returned" with "usable."

Sample design: why your test showed no differences

When someone tells me "I tested four vendors and they were all the same," the sample set is almost always the problem.

The most common mistake: testing fifteen bestseller ASINs. Those products have the richest data on the platform, so every vendor returns everything and all four score 100%.

You measured Amazon's data completeness for bestsellers, not the vendors' capability.

A discriminating sample has five kinds of record:

  1. Across categories — four or more top-level categories. Page templates differ, field availability follows.
  2. Across marketplaces — US plus at least two non-US. This is the direct test for the "US-only field" surprise.
  3. Heavy variants — 20+ variants, to probe variant depth.
  4. Weak-data products — new listings with no reviews, long-term out of stock, no Buy Box. This is the litmus test for fill rate.
  5. Deep pages — for search, go through page five.

Two hundred to five hundred records per vendor is enough. Below that, noise dominates.

Four execution disciplines — break any one and the numbers stop being comparable:

same ASINs
same time window    <- compress into hours; prices and stock move
same concurrency
same retry policy   <- easiest one to get wrong
Enter fullscreen mode Exit fullscreen mode

That last one: give vendor A three retries and vendor B one, and A's usable rate is inflated while its real cost is triple. You are not measuring the vendor, you are measuring your retry policy.

Five places where vendors actually diverge

After running this repeatedly, differentiation concentrates in five spots. If you are short on time, look here first.

1. Ad placement flags and organic rank

The most discriminating item. Vendors carrying isSponsored and organicRank are a minority; most return only page position.

Without them, the rank metric is systematically distorted. A team tracked a core keyword at a steady position three for four months, concluded organic had plateaued, and prepared to raise ad spend. Recomputed with isSponsored applied, the truth was organic position one with two competitors' Sponsored Brands units above it. Organic had been improving the whole time.

One boolean field put a quarter of budget decisions on an inverted premise.

2. Variant-level depth

Many vendors ship a variants array containing nothing but ASINs — no per-variant price, stock or rating. That tells you five variants exist without telling you which is selling, which is out of stock, or which is losing rating.

A review analysis once produced a high-confidence finding that users complain about battery life. It could not be actioned. Adding variant localised it to one capacity tier — concentrated in the low-spec model, near zero in the high-spec one. The response changed from "redesign the battery across the range" to "fix spec copy on the low tier." Cost fell by an order of magnitude.

Variant attribution is the field that takes an insight from plausible to executable.

3. Deep search pages

Little difference across the first two pages. Sharp divergence from page three: some vendors return duplicates, some return empty.

Consequence: competitive density is decided in the deep pages. Seeing only two pages makes you systematically underestimate competition. And when you can only see two pages, "this market is not competitive" describes your observation range, not the market.

4. Promotions and coupons

coupon, promotions and Subscribe & Save discounts are usually thin. But they matter enormously for pricing — list price alone systematically overstates what competitors transact at.

5. Seller and Buy Box detail

offerCount, buyBoxWinner and seller ratings are core metrics in heavily resold categories, and absent from many responses entirely.

Billing: why entry price tells you nothing

Four billing models, not comparable to each other:

Model Headline Reality
Per request Cheapest, most seductive Failed requests bill too; deep pages and retries multiply
Per successful result Middle 200 with missing fields still bills — you pay full price for a broken record
Per record Impossible to rank by eye Only comparable as cost per thousand usable records
Monthly quota Priciest, least friction Overage rates are steep

A vendor billing per request can quote a third of one billing per record and cost more in practice, because deep pages fail, retries are frequent, and failures bill anyway.

The only comparable number is cost per thousand usable records. Do not do cost accounting on the vendor's billing basis — do it on your own usable record count. The multiple between the two is precisely the cost of silent failures and retries.

Make regression routine

A one-off acceptance test proves it works now. Vendor coverage drifts: page redesigns, policy changes, capacity shifts.

Run a fixed sample weekly — fifty to a hundred records is plenty and cheap — and trend four numbers:

median latency / p95 latency / usable record rate / average coverage
Enter fullscreen mode Exit fullscreen mode

Chart them, and chart the failure classification as a stacked area. A rising partial share is the most valuable early warning signal you can have.

The checklist

  1. Field contract written down, P0/P1/P2 agreed with the business side — not invented by an engineer.
  2. Sample covers all five record types.
  3. Four or more vendors, identical terms.
  4. All four numbers in hand — coverage, fill rate, usable rate, cost per 1k usable.
  5. Failures classified, partial share confirmed. This is the only class needing dedicated code.
  6. Quotas modelled per operation type, with backoff tested against real limits.
  7. Regression scheduled weekly, four numbers trended.

Item seven is the easiest to skip and the most valuable.

Two questions that end the sales conversation

Once the contract is written, there are two questions worth asking every vendor before signing anything. They are not gotcha questions. They are diagnostic, and the shape of the answer tells you more than the content does.

"Can you send me the field-level schema for the product object, not the object list?" Most vendors will send a page saying "product data, search data, review data." Push once. If they can send an actual field dictionary, you just saved two days of testing. If they cannot — or if what comes back is clearly a marketing page with field names sprinkled on it — you have learned something about how they think about delivery.

"If a P0 field comes back null for a week, what happens?" Listen for whether they have a category for that. Vendors who monitor fill rate answer immediately, because they already have the dashboard. Vendors who only monitor HTTP status pause. That pause is the answer.

Neither question is hostile. Both are what a serious buyer asks, and a vendor who handles them well is showing you they have been through this before.

The retry multiplier nobody puts in the pricing calculator

Here is an arithmetic that reorders rankings more often than any feature comparison does.

Say vendor A charges \$0.001 per request and vendor B charges \$0.0025 per record. On paper A looks 2.5x cheaper. Now add behaviour. A has an 82% usable rate on your sample, so you retry failures twice, and failed attempts still bill because the billing unit is the request:

effective_cost = unit_price × attempts_per_usable_record / usable_rate

A: 0.0010 × (1 + 2 × 0.18) / 0.82 = 0.00166
B: 0.0025 × 1.00            / 0.97 = 0.00258
Enter fullscreen mode Exit fullscreen mode

A still wins, but by 1.55x rather than 2.5x. Now change one assumption. Suppose A's deep-page coverage is weak, so the pages you actually care about need three retries and the usable rate on page 3+ drops to 60%:

A (pages 3+): 0.0010 × (1 + 3 × 0.40) / 0.60 = 0.00367
Enter fullscreen mode Exit fullscreen mode

Now A costs more than B, on precisely the task you bought it for. Nothing on the pricing page changed. Only the denominator did.

This is why I stopped normalising list prices and started computing cost per thousand usable records on my own sample. It takes an afternoon, and it is the only number that survives contact with production.

Catching staleness before it reaches your dashboard

Silent failure is the expensive class and staleness is its quietest variant. A field returning a plausible but frozen value passes every schema check you can write. Here is the smallest useful detector — it watches for values that stop moving:

from collections import defaultdict
from datetime import datetime, timedelta

history = defaultdict(list)          # field_path -> [(ts, value)]
STALE_AFTER = timedelta(days=14)

def observe(record, now=None):
    now = now or datetime.utcnow()
    for path, value in flatten(record).items():
        history[path].append((now, value))

def stale_fields(now=None, window=STALE_AFTER):
    now = now or datetime.utcnow()
    out = []
    for path, obs in history.items():
        recent = [(t, v) for t, v in obs if now - t <= window]
        if len(recent) < 5:
            continue                                  # not enough signal yet
        if len({str(v) for _, v in recent}) == 1:
            out.append((path, recent[-1][1]))
    return out
Enter fullscreen mode Exit fullscreen mode

Run it weekly over a fixed sample. A price field that has not moved in two weeks across fifty ASINs is not a stable market — it is a frozen parser. The distinction matters, and no status code will tell you which one you are looking at.

Two ways to accidentally measure the wrong thing

Two production realities will distort every number in this article if you ignore them.

Concurrency changes your success rate. If you run the evaluation at 50 parallel requests and the vendor throttles at 20, the failures you observe are yours, not theirs. You will record a low usable rate that has nothing to do with field coverage, and you will carry that error into the vendor comparison. Run the test at the concurrency you actually plan to use in production, and record that number alongside the results. A measurement you cannot repeat is an anecdote.

Caches hide staleness. If responses are cached, a field can look fresh because you are being served last week's copy. A weekly regression run against a cached endpoint will report perfect stability right up until the cache expires and the parser turns out to have been broken for a month. Either bypass the cache for the measurement run, or record the cache TTL as part of the result and compare like with like — one vendor's five-minute TTL and another's twenty-four-hour TTL are not the same product, whatever the fill rates say.

Both are easy to miss for the same reason: both make the numbers look better than reality. Be suspicious of measurements that come out clean.

Wrapping up

The core idea is one sentence: write down what you need first, then see who can deliver it.

Most people do the reverse — they read the vendor docs and reverse-engineer requirements from them. The docs lead, you buy a pile of fields nobody uses, and the one field you needed is missing.

Ninety lines of Python, two days, four vendors. That is a much better deal than two weeks of reading marketing pages and emailing sales teams for a field dictionary.

If you want to run this against real responses: Amazon Scraper API covers product and search objects, Amazon Review API covers reviews, and Amazon Data MCP is the agent-facing path. Grab a key from the console and run the test; docs are at the Amazon Data MCP overview.

One last thing, and I mean it: run this on us too. Completeness numbers a vendor reports about itself — ours included — deserve to be verified.

If you have run something like this, I would be curious which failure class dominated for you. In my experience partial is the one nobody budgets for.

Top comments (0)