Intro
Most AI Scraping looks impressive in a demo. Most of them conveniently stop right after the scraper successfully extracts a single page. Like Firecrawl, ScrapeGraphAI.
Though ScrapeOps AI Claims were different, they claimed to provide the data plus the reusable code to scrape multiple pages. Which means not relying on AI for getting data again and again, but relying on code. So I wanted to see how useful ScrapeOps AI Assistant is actually once you push past the demo. So I picked five different websites, across five different categories, and tried generating scrapers for each. Then, because one clean run doesn't prove much, I pushed it further into the cases where I expected it to fail.
Test Setup
For each site, I was checking the same five things with ScrapeOps:
Can it identify the schema on its own?
Can it generate working code?
Can it correctly detect whether JS rendering is needed?
Can it actually retrieve the data?
Does it produce usable output?
Website 1: Ecommerce — Amazon Product Page
I scraped an Amazon listing directly: Web Scraping with Python book listing.
Result: worked.
Schema detection: this is where it immediately separated itself from Firecrawl and ScrapeGraphAI. Both of those ask you for a schema upfront, and even then don't always pull the full picture. With ScrapeOps, I didn't have to define anything. It looked at the page, recognized it as product data, and generated this on its own:
aggregateRating:
type: object
fields:
bestPossibleRating: integer
ratingValue: number
reviewCount: integer
worstPossibleRating: integer
availability: string
brand: string
category: string
currency: string
description: string
features:
type: array
items: string
images:
type: array
items:
type: object
fields:
alt_text: string
url: string
name: string
preDiscountPrice: number
price: number
productId: string
reviews:
type: array
items:
type: object
fields:
author: string
content: string
date: string
rating: integer
title: string
seller:
type: object
fields:
name: string
rating: number | null
url: string
serialNumbers:
type: array
items:
type: object
fields:
type: string
value: string
specifications:
type: array
items:
type: object
fields:
key: string
value: string
url: string
videos:
type: array
items:
type: object
fields:
url: string
Code generation: I tested the generated code locally and it worked. My one open question is maintenance. Scraper code like this will inevitably break when a site updates its markup, and the real unlock would be if it could self-heal when selectors shift, rather than needing a fresh generation every time.
JS rendering detection: correctly set to "No" here, since the product page didn't need it. Worth noting for later, because on a category page it flipped that to "Yes" automatically without me telling it anything.
Data retrieval and output quality: better than what I got out of Firecrawl or ScrapeGraphAI on comparable pages. It also scored the output for me directly in the dashboard.
I still went and tested it locally myself, and hit this on the first run:
PS C:\ScrapeOps\> python amazon.com_scraper_v1.py
Traceback (most recent call last):
File "C:\ScrapeOps\amazon.com_scraper_v1.py", line 357, in <module>
html_file = sys.argv[1]
~~~~~~~~^^^
IndexError: list index out of range
Turns out the script expects an HTML file as input. I right-clicked the page, saved it as HTML, and reran it. Then, out of curiosity, I saved three more Amazon product pages, a pair of crocs, another book, and a chopping board, as input-crocs.html, input-book.html, and input-choppingboard.html. The exact same generated script worked on all three without any changes, as long as the page matched the same product schema.
Then I tried feeding it a category page instead, just to see what would happen. The code ran without crashing, but returned an entirely empty result, which makes sense in hindsight, since the script was built specifically for product-page schema, not category listings:
{
"url": "https://www.amazon.com/kitchen-dining/b?ie=UTF8&node=284507/",
"name": "",
"productId": "",
"brand": "",
"category": "",
"currency": "USD",
"price": null,
"preDiscountPrice": null,
"availability": "out_of_stock",
"aggregateRating": {
"bestPossibleRating": 5,
"ratingValue": null,
"reviewCount": null,
"worstPossibleRating": 1
},
"description": "",
"features": [],
"images": [],
"videos": [],
"specifications": [],
"serialNumbers": [],
"reviews": [],
"seller": {
"name": "Amazon.com",
"rating": null,
"url": "https://www.amazon.com"
}
}
That told me exactly what I needed to try next: an actual category page, generated on its own terms.
Website 2: Ecommerce Category Page — Target + Ebay
The tool lets you feed in up to five URLs from the same domain so the generator can build a stronger, more general parsing pattern. I gave it two, since one was already plenty for a first test:
This one took around 20 minutes. It worked, but the data came back incomplete for Target.
The pagination logic looked reasonable on paper, but it didn't capture all the products in the category, and that showed up clearly in the scoring too; With Completeness: 5
I tried a couple more Target category pages and got similar results, so I switched targets.
My guess is that giving it two very distinct categories at once may have confused the pattern it was building, so I went back to a single URL, this time on eBay:
Ebay Category page worked well, scoring 9.
In my own scoring language, that's basically "the code is cream," and I could trust it without even testing locally.
I still did, out of habit, but it held up.
Website 3: Real Estate — Redfin
Next I tried a Redfin listing page: 255 E 74th St, Unit 10A, New York.
This is where things got messy in an interesting way.
The dashboard itself failed to parse the output, throwing Failed to parse provided JSON output: invalid character 'L' looking for beginning of value. Generation also took noticeably longer than usual, about 45 minutes compared to the typical 15. The final score came back as 0.0/10, with no scored output shown.
Here's the twist though.
The code itself was still generated.
When I ran it locally with python .\redfin.com_scraper_v1.py .\input-redfin-product.html, it worked perfectly.
I compared the local terminal output against the live listing manually, field by field, and it matched:
{
"address": {
"city": "New York",
"country": "US",
"neighborhood": "New York",
"state": "NY",
"streetAddress": "255 E 74th St Unit 10A",
"zipCode": "10021"
},
"appliances": [
"Dishwasher",
"In-unit washer and dryer",
"Washer/dryer stacked",
"Miele appliances"
],
"bathrooms": 3.5,
"bedrooms": 4.0,
"climateRisks": {
"floodRisk": "1/10 Flood Factor",
"fireRisk": "1/10 Fire Factor"
},
"cooling": {
"type": "Central air conditioning"
},
"currency": "USD",
"description": "Experience the perfect blend of contemporary design, luxury, and functionality in Residence 10A at Casa 74, one of the Upper East Side's premier full-service condominiums...",
"features": [
"Unfurnished",
"Hardwood floors",
"Has view",
"6 total rooms",
"3 full bathrooms",
"1 half bathroom",
"Dishwasher",
"In-unit washer and dryer",
"Washer/dryer stacked",
"Common area laundry room",
"Hardwood",
"Central air conditioning",
"Condo in a 30-story building",
"Entry level at floor 10",
"Building name: Casa 74",
"Elevator(s)",
"Monthly association fee of $3,965",
"Building allows pets",
"24-hour doorman and concierge",
"Private garden",
"Bicycle storage",
"Refrigerated storage",
"Children's Pavilion",
"Equinox direct access"
],
"floorPlan": {
"description": "Floor Plans",
"url": null
},
"heating": {
"type": "Radiant heated floors"
},
"images": [
{
"altText": "255 E 74th St Unit 10A, New York, NY 10021",
"url": "https://ssl.cdn-redfin.com/photo/211/bigphoto/638/RLS20101638_2.jpg"
}
]
}
So the actual extraction logic was sound, the tool just failed to score and display it correctly on the dashboard.
Given that ScrapeOps AI Scraper Builder is still in beta, my read is that this was a scoring/display bug rather than a code generation problem. Worth knowing, because it means the score on screen isn't always the full story, and testing locally is still a good habit even when the dashboard looks discouraging.
Website 4: Review Site — Trustpilot
Next up, a Trustpilot category page: Animals & Pets category.
This one worked surprisingly well.
Generation took only about 10 minutes, and the score came back at a clean 10/10, across accuracy, critical fields, data types, structure, and completeness.
Here's a sample of what came back for one listing in the category:
{
"address": {
"city": "Pismo Beach",
"country": "United States",
"fullAddress": "PO Box 417, Pismo Beach, United States",
"state": null,
"street": "PO Box 417",
"zipCode": "93448"
},
"badges": ["Most relevant"],
"businessId": "57ec1a9d0000ff0005955109",
"categories": ["Pet Supply Store"],
"description": null,
"distance": null,
"email": "Don@BoomerangTags.com",
"highlights": [],
"hours": null,
"images": [
{
"alt": "Boomerang Pet ID Tags",
"url": "https://consumersiteimages.trustpilot.net/business-units/57ec1a9d0000ff0005955109-198x149-1x.jpg"
}
],
"isClaimed": null,
"isOpen": null,
"isSponsored": false,
"name": "Boomerang Pet ID Tags",
"phone": null,
"priceRange": null,
"rating": {
"maxScore": 5,
"reviewCount": 12719,
"score": 4.9
},
"url": "https://www.trustpilot.com/review/www.boomerangtags.com",
"website": "https://www.boomerangtags.com"
}
Clean, complete, and genuinely one of the best results across the whole test.
Website 5: Job Listings — Indeed
Last one in the main round was an Indeed search page: software engineer jobs in California.
It worked well, generating in about 15 minutes and scoring 9.4/10.
It correctly pulled the search parameters and returned a clean, easily readable structure of job listings for the California search. It also didn't need JS rendering, which by this point had become the pattern rather than the exception across every site I tried. That's really the strongest thing about this tool: I never had to think about schema or JS rendering myself, it just handled both automatically every time.
Things It Did Well
JS detection. This impressed me the most across every single site I tried. None of the pages in my test set actually needed JS rendering, and the tool correctly identified that every time without me specifying anything. For beginners, this alone removes a huge chunk of trial and error that used to eat entire afternoons.
Schema detection. Consistently solid across ecommerce, real estate, and listing-style pages.
Production-oriented code. It gives actual scraper code, in the language and framework stack of your choice, that you can open, read, and edit like any other script.
Anti-bot awareness. The whole workflow feels designed around real scraping constraints rather than toy examples. It integrates with ScrapeOps' own proxy aggregator to handle anti-bot systems on the target site. If a site is running something like Cloudflare or DataDome, it applies the right bypass proxies automatically.
Where It Didn't Work
This is the part that actually matters most, because a lot of people treat AI code generators like magic. They're not, but they get impressively close. The honest gaps show up in situations that need a human's judgment or credentials, not just clever parsing.
Test 6: Website Behind a Login — LinkedIn
I tried a LinkedIn people-search page: https://www.linkedin.com/search/results/people/?keywords=software%20developer. It failed outright, with a Proxy Error.
Not surprising in the slightest. The tool has no credentials, no authenticated session, and no way to know what exists behind a login wall.
This is squarely a "build it yourself" problem, implementing your own session and credential management, and it's entirely possible to do, just not something a URL-based generator can infer on its own.
Test 7: Form-Driven Workflow — IRCTC
I also tried IRCTC, the Indian railway booking site. It presents a search form first. I filled it out for a Ludhiana to Jaipur route, and it took me to a results page listing trains.
But when I opened that results URL fresh, in a new private window, it just dumped me back at an empty search form instead of showing results.
Same root issue as any complex marketplace or travel site with dropdown selections, date pickers, and multi-step search interactions: the data literally doesn't exist at a stable URL until you complete the interaction.
The tool can't infer a workflow that hasn't happened yet. Again, not really a failure of the tool, just a boundary of what URL-based generation can reasonably do. So there is no point to waste my credits on scrapeops for this URL.
Most Important Observation
The more I tested this, the clearer it became that it isn't trying to replace scraping engineers. It's trying to remove the first 30 to 50 percent of the grind.
Traditionally, that chunk of work looks like inspecting HTML by hand, hunting for the right selectors, and writing extraction code from scratch. This tool compresses all of that dramatically. What's left afterward is still real engineering work: validation, monitoring, scaling, and maintenance as sites inevitably change.
Ending
My conclusion is that vibe scraping is real. Just maybe not in the way most people picture it.
The value is that getting from "I need data from this website" = Vibe Discovery
to
"I have a working scraper" = Vibe Scraping
is becoming dramatically faster than it used to be.
So the interesting question isn't really "can AI generate scrapers" anymore. That part's answered.
The more interesting question now is what happens once every developer can generate hundreds of them.
Top comments (0)