We ran the same 235 domains through our scanner three times. In the first run, 69 were analyzable. In the paced run, 218 were. On a later pass with other changes, coverage dropped to 59. An enrichment job that reports only the rows it can classify can make these runs look much more similar than they actually were. Benchmark methodology and data
I work on StoreTechScan, where our team ran these measurements. AI tools helped draft and edit this article.
Measure access before comparing detections
The fixed list contained 202 domains from Shopify case studies, nine from WooCommerce showcases or case studies, and 24 non-store controls. We kept the same list through all three runs. Historical case studies were the platform reference. A store could have changed platforms by the time we requested its homepage.
| Run | Analyzable | HTTP 429 | WAF blocks |
|---|---|---|---|
| Initial run with 16 workers | 69/235 | 155 | 11 |
| Four workers with 250 ms between starts | 218/235 | 1 | 14 |
| Later repeat with extraction and memory changes | 59/235 | 164 | 11 |
The runs varied in timing and other settings, so the comparison measures variation in coverage under changing conditions. In the paced run, 209 of the 218 analyzable domains agreed with their historical platform reference. That 95.9% agreement applies to this reachable, Shopify-heavy cohort. The other 17 domains in the input should still appear in the report with their access status.
DNS can add evidence when the page is quiet
In a separate November 19 pilot, we checked public MX, TXT, NS, www CNAME and DMARC records for 71 domains from historical Shopify references, nine from WooCommerce references and 20 B2B SaaS sites. The script reported at least one recognized signal for 97 of 100 domains, Google Workspace MX on 52 and Shopify www CNAME on 41. The original run saved aggregate output but did not retain its individual domain results, so those figures cannot be independently recomputed per domain. A TXT verification record can persist after a service has been removed. Keep that evidence alongside the HTTP or markup signal so a downstream reviewer can decide how much weight it deserves.
The script reported a p50 of 345 ms and a p90 of about 1.5 seconds, using a zero-based index of floor(n × p) in the sorted timings. For 100 observations, its p50 was the 51st ordered timing rather than the mean of the two middle timings. The pilot used a longer deadline than the production lookup. Put DNS alongside the page request and give it a deadline of its own so an unresponsive nameserver does not hold up the result.
Keep failure rows in the result
Keep the access status alongside the technology columns in your dataset. A blank technology field on an accessible page and a blank field on a blocked page deserve different follow-up. Report the input count, analyzable count and reasons for failed requests together so readers can see how much of the original list the results cover. Avoid presenting the 95.9% historical-reference agreement as accuracy for all websites.
Top comments (0)