The latest articles on DEV Community by ScraperWatch (@scraperwatch_6e8ad2ecf5d8). https://dev.to/scraperwatch_6e8ad2ecf5d8 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4192116%2F5bae0c22-57a2-4e4c-9181-9ad415c5d339.png https://dev.to/scraperwatch_6e8ad2ecf5d8 en ScraperWatch Sun, 18 Oct 2026 12:55:52 +0000 https://dev.to/scraperwatch_6e8ad2ecf5d8/i-analyzed-12000-ready-made-web-scrapers-1-in-10-popular-ones-fails-more-than-10-of-the-time-30n https://dev.to/scraperwatch_6e8ad2ecf5d8/i-analyzed-12000-ready-made-web-scrapers-1-in-10-popular-ones-fails-more-than-10-of-the-time-30n <p>If you've ever needed data from Zillow, Google Maps or Amazon, you've probably landed on the Apify Store and found <strong>dozens of scrapers for the same site</strong>. Which one actually works? The store shows ratings, but most tools have a handful of reviews or none.</p> <p>It turns out Apify's official store API publishes something much better than reviews: <strong>real usage and the outcome of every public run over the last 30 days</strong>. So I pulled it for all ~12,000 listed tools.</p> <h2> What the data says </h2> <ul> <li> <strong>240 million runs</strong> in the last 30 days; <strong>1.2%</strong> failed or timed out overall.</li> <li>But averages hide a lot. Among <em>established</em> tools (300+ runs and 20+ users a month), <strong>129 of 1,375 (9.4%) fail more than 10% of their runs.</strong> </li> <li> <strong>Equally reliable tools differ in price by orders of magnitude.</strong> Among Amazon scrapers that succeed at least 95% of the time, the median costs <strong>$2.00 per 1,000 results</strong> and the most expensive costs <strong>$100</strong>. For Google Maps it's a $1.20 median against $60 at the top.</li> <li>The market is extremely concentrated: <strong>68% of tools had fewer than 10 users</strong> last month, while the <strong>top 1% got 71% of all users</strong>.</li> </ul> <h2> What I built </h2> <p><strong><a href="https://scraperwatch.com" rel="noopener noreferrer">ScraperWatch</a></strong> turns that into a neutral comparison: one page per website (167 so far, from <a href="https://scraperwatch.com/t/zillow/" rel="noopener noreferrer">Zillow</a> to <a href="https://scraperwatch.com/t/etsy/" rel="noopener noreferrer">Etsy</a> to <a href="https://scraperwatch.com/t/linkedin-jobs/" rel="noopener noreferrer">LinkedIn jobs</a>), ranked by real 30-day usage, with success rate, runs, price per 1,000 results and last run. It refreshes daily.</p> <p>A few extras that might be useful:</p> <ul> <li> <strong><a href="https://scraperwatch.com/picks/" rel="noopener noreferrer">Best picks</a></strong>: for every site, the most used, the most reliable and the cheapest reliable tool.</li> <li> <strong><a href="https://scraperwatch.com/rising/" rel="noopener noreferrer">Rising</a></strong>: tools gaining users fastest this week.</li> <li> <strong><a href="https://scraperwatch.com/data/" rel="noopener noreferrer">Open data</a></strong>: the full dataset as daily JSON and CSV (CC BY 4.0).</li> <li> <strong>MCP server for AI agents</strong> at <code>https://scraperwatch.com/mcp</code> (listed in the official MCP Registry as <code>com.scraperwatch/scraperwatch</code>). Ask your agent "which scraper works for Zillow?" and it gets live numbers.</li> <li> <strong><a href="https://scraperwatch.com/badges/" rel="noopener noreferrer">Reliability badges</a></strong> for actor developers: <code></code> </li> </ul> <h2> Ground rules </h2> <ul> <li>Rankings come from data only. Some links are affiliate links (disclosed on every page), and they never change the order.</li> <li>I <strong>exclude tools built to harvest personal contact data</strong> (email/phone finders, lead lists of people, follower lists, engagement bots): about 1,400 of them.</li> <li>Success rate is shown only with 20+ runs, and a failed run can also mean bad input, so treat small differences with care. Full <a href="https://scraperwatch.com/methodology/" rel="noopener noreferrer">methodology</a>.</li> </ul> <p>If you build or use scrapers, I'd genuinely like to know what would make this more useful.</p> <p><em>Written with AI assistance. Every number comes from code over public data, and you can check it all on the site.</em></p> webscraping data opensource ai