DEV Community

Cover image for My Shopify store index got Apify's Rising Star badge. What 950K stores taught me
Insane Developer
Insane Developer

Posted on

My Shopify store index got Apify's Rising Star badge. What 950K stores taught me

I'm a developer from Ukraine building a small data business on Apify. The Store page of my Actor Shopify Store Leads now shows a Rising Star badge. Apify assigns it automatically; there's nothing to apply for. Here is what went into the tool, in case it helps someone building their own.

It runs on my own index

The Actor doesn't resell someone else's database. I keep an index of Shopify stores that I build myself: popular-site lists (Tranco, Chrome UX Report for 42 countries, Common Crawl) go through a DNS check for Shopify's IP range, and every hit is confirmed with the store's own meta.json. The October build holds 951,105 stores. A daily job adds newly found stores, and every run reads the details of the stores it returns live from their storefronts.

The full method is on insanedev.dev/methodology.

What 950K stores look like

I read 765,826 of those storefronts and counted which apps and themes they run.

Which apps 951K Shopify stores run: PayPal messaging 232,224, Judge.me 212,895, Klaviyo 162,731, Microsoft Clarity 85,554, Mailchimp 62,076

Judge.me is on more stores than any other review app, and Mailchimp is still on 62,076 of them.

Dawn still runs 1 in 5 Shopify stores: Dawn 159,954, Horizon 47,806, Debut 29,080

Dawn runs on 21% of the storefronts I read. Debut, a theme from before Online Store 2.0, is still on 29,080.

Where Shopify stores are: US 325,308 (34%), UK 72,219, Australia 56,138, Canada 53,486, India 49,019

A third of the stores sell from the US. The biggest niches by keyword are clothing, fashion and jewelry.

Speed came from measuring against the competition

Before every change I run the same query through the top Shopify lead Actors on the Store. The latest run: coffee stores in the US, contacts on, 200 stores. My Actor now returns them in 56 seconds, with an e-mail for most rows plus phones, socials and addresses where the store publishes them. The storefront-reading alternative I compare against took 829 seconds for the same query a day earlier.

Most of the time went into network waits, not parsing. Reading the few contact pages that matter in parallel and pacing requests to Shopify per IP gave most of the gain.

Pricing by the row

Users pay $5.50 per 1,000 leads, and the first 25 are free. Errors, duplicates and empty results are never billed. That rule shaped the code more than anything else: a store that fails to load costs me compute and earns nothing, so it's in my interest to make every read count.

Try it without an account

insanedev.dev/shopify has counts for every app, theme, niche and country in the index, plus a free sample CSV with real contacts (up to 15 stores per list, all from outside the top 50K by traffic). The aggregate counts are also an open dataset on Hugging Face and GitHub under CC BY 4.0.

If you need data from somewhere else, I build custom scrapers and lead lists too: insanedev.dev.

Top comments (0)