DEV Community

ricco020
ricco020

Posted on

Two traps in the Bing Webmaster API that silently invert your site comparison

#seo #webdev #api #debugging

I spent a night comparing two sites in the Bing Webmaster API and reached three different
conclusions, each one demolishing the last. All three came from the same two traps. Neither raises
an error. Neither shows up in the JSON. Both silently invert the comparison.

Figures below are anonymised, because the ratios are the point, not whose sites they are.

Trap 1: GetCrawlStats history length is per site, and it is not the same 

site A   76 data points   2026-06-05 .. 2026-08-19
site B    7 data points   2026-08-13 .. 2026-08-19
Enter fullscreen mode Exit fullscreen mode

I wrote the obvious thing:

for p in data[:30]:
    total += p["CrawledPages"]
Enter fullscreen mode Exit fullscreen mode

For site A that sums 30 days. For site B it sums 7, because there are only 7. No error, no
warning, just a number that is roughly a quarter of what I thought I was comparing.

My conclusion was "site B is crawled nine times less". Normalised per day, site B is crawled
more: 168 pages/day against 121.

The cause is that Bing does not backfill. A property verified last week has a week of history, not
a week of low activity. A short history looks exactly like a quiet site if you only read the
sum.

# what it should have been
per_day = sum(p["CrawledPages"] for p in data) / len(data)
window  = (min(dates), max(dates))   # and print it, always
Enter fullscreen mode Exit fullscreen mode

Trap 2: GetPageStats covers a different date range per site, and never tells you

This one is worse, because there is no date parameter to get wrong. You call it, you get rows, each
row has a Date. I never looked at those dates.

site A   GetPageStats covers  2026-06-05 .. 2026-06-19    (15 days, in JUNE)
site B   GetPageStats covers  2026-08-14 .. 2026-08-14    (ONE day, in AUGUST)
Enter fullscreen mode Exit fullscreen mode

I had been comparing June against August, fifteen days against one, and calling it a finding
about the sites. The whole analysis, and the follow-up work it justified, rested on that.

The check that caught it, and why it was written first

Before running any of this I had written down one rule:

If GetPageStats total impressions differ from GetRankAndTrafficStats over the same window, the
endpoint is sampling and the page counts are worthless. Stop there.

The measurement:

site A   PageStats 12 impressions   RankAndTraffic 10   ->  120 % coverage
site B   PageStats 714 impressions  RankAndTraffic 325  ->  220 % coverage
Enter fullscreen mode Exit fullscreen mode

Two endpoints, same property, same window, and they do not agree with each other. So the check
fired and I stopped, instead of building one more conclusion on top.

Writing that rule down before measuring is the only reason I caught it. I had also written a
"reserve" in the earlier analysis, hedging that the average was fragile on a small sample. That
hedge was aimed at the right object for the wrong reason: the problem was never the average, it was
the window. A smart-sounding caveat can cover a defect without naming it, and reassure you into
keeping the wrong number.

What I would tell anyone using this API

  • Print the date range and the row count of every response before you compute anything. Not in debug mode. Always.
  • Never sum across sites without dividing by the number of points.
  • Cross-check two endpoints on the same window. If they disagree, neither is a total.
  • Write the invalidation condition before you measure, not after you like the result.

None of this is specific to Bing. GSC anonymises rare queries, so summing by query dimension covered
only 19 % of impressions on one of my properties, and comparing two sites that way reversed their
ranking. Same shape of error: an API answered exactly what I asked, and I had asked something
slightly different from what I meant.

The sites in the table are mine; one of them is VPNSmith,
where the traffic data came from. I am not going to publish their raw numbers, which is why the
ratios above are anonymised.

If you have a comparison in a dashboard right now that spans two properties, check that both sides
cover the same dates. Mine did not, for three hours, and it was confident the whole time.

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev •

One guardrail worth adding: publish a route only when the business has evidence for that service and area. It protects the architecture from turning into a matrix of thin location pages later.