DEV Community: Samuel Raphael

Firecrawl vs Apify vs Quorel

Samuel Raphael — Sat, 25 Jul 2026 12:04:12 +0000

Firecrawl and Apify both do the hard part well. They render JavaScript, dodge bot detection, and hand you back clean markdown or structured JSON. If you need a page turned into text, either one will do that job.

But most of the time the point isn't a single page. It's a page you need to check again tomorrow, and the day after, and the day after that: a price, a listing, a status, a set of rows that might quietly change. And that's where the workflow, not the extraction, starts costing you.

Firecrawl bills per credit, and the credit math is not what the pricing page implies. Scrape is 1 credit. Add JSON extraction and Enhanced Mode — both of which you probably want for structured data — and it's 9 credits per page. A $16/mo Hobby plan advertised at 3,000 credits works out closer to 300–500 pages once you're actually extracting fields, not just fetching markdown. Credits also don't roll over, so a quiet month doesn't bank anything for a busy one.

Apify flips the complaint around. Pricing is compute-unit based (RAM × time) plus separate line items for proxy and storage. It's flexible and it scales, but you can't look at a number on the pricing page and know what a job will cost until you've run it. For teams that want a predictable bill, that unpredictability is the whole problem.

Neither tool keeps history by default. You get the current state of the page. If you want yesterday's version to compare against today's, that's on you to build — a database, a diffing script, a place to store snapshots, a way to roll back if a refresh comes back wrong.

Quorel starts from a different question: not "how do I fetch this page," but "how do I keep a page as structured, versioned data over time." You describe the fields you want in plain English. Every refresh (nightly by default, or on demand via a ping URL, webhook, or your own cron) is saved as its own immutable version. Nothing is overwritten. You can diff any two versions to see exactly what changed, and roll back in one click if a refresh comes back bad.

Pricing is flat, not metered. Free tier is one dataset, 20 URLs, full version history included. Pro is $19/mo for 5 datasets and 100 URLs, with on-demand refresh and webhooks. No credit multipliers, no compute-unit math, no surprise bill.

This isn't a claim that Quorel replaces Firecrawl or Apify at everything they do. If you need to fetch an arbitrary one-off page as clean markdown for an LLM right now, Firecrawl is built for exactly that. If you need thousands of site-specific scrapers off a marketplace, Apify's Actor ecosystem has years of head start. Quorel is built for the narrower, recurring job: turning a page you check regularly into a structured, versioned API you can trust and query later.

Every dataset ships with an MCP server on every plan, free included, so an agent can query it directly — or pull the raw rows, fix nulls, and publish a new version itself.

If you've been burned by a scraper that silently broke, or wished you'd kept last week's data to compare against, that's the specific gap Quorel is built to close.

I'm building Quorel — happy to answer questions on versioning, pricing, or how the MCP integration works.

I Built a Job Listing + E-commerce API Dataset in 4 Hours, No Code

Samuel Raphael — Fri, 24 Jul 2026 21:27:44 +0000

The problem with "just build your own API"

If you've ever wanted a personal dataset (job listings, product prices, whatever) you know the real cost isn't the idea, it's the pipeline. Scraping. Cleaning. Storage. Refreshing on a schedule. Serving it over an API. Handling schema drift when the source site redesigns. Every one of those is its own mini-project, and each is its own source of friction.

I wanted a dataset combining Amazon products for developers and job listings scraped from multiple sites. Normally that's a multi-day side project. I built it in about 4 hours, without writing a single line of scraping or backend code.

Here's how.

The magic part: describing data instead of scraping it

I used a tool called Quorel. The workflow was almost embarrassingly simple:

I pasted in my source URLs (Amazon product pages, a few job board listing pages).
I described the schema and extraction intent in plain English: no selectors, no XPath, no code. Something like "product name, price, rating, whether it's in stock" for the Amazon side, and "job title, company, location, salary range, posted date" for the listings side.
I hit run and left it alone for 30–60 minutes.

When I came back, the data was fully structured and ready to query.

From there I asked Claude to build me a dashboard on top of the API so I could actually see and explore the data. That part took the rest of the 4 hours. The result is live here: diffboard.vercel.app.

The part that actually sold me: it's versioned like git

Structuring the data was the easy win. What kept me around is that Quorel treats datasets like a git repo for live web data:

Nightly refresh (or your own cron schedule): the data doesn't go stale.
Every refresh is a saved version, nothing overwrites the last run.
Diffing between versions: I can see exactly what changed, like prices that moved, listings that disappeared, new rows that showed up.
Time travel: I can pull the dataset as it looked days or months ago.
Rollback: a bad refresh doesn't mean starting over.
Forking: you can clone a public dataset the way you'd fork a repo, and extend it for your own use case.

That combination, structure plus history, is the part a plain scraper never gives you. Most tools hand you today's snapshot. This hands you the whole timeline.

Talking to your data instead of querying it

The other piece that stood out: Quorel ships an MCP server with every dataset, on every plan, including free. That means I can point an AI agent at my dataset and just talk to it, like "find listings with price drops this week" or "fix the null prices and publish a new version," and it does the pull, clean, and push cycle for me, still landing in the same versioned system. It's genuinely like chatting with your data pipeline instead of writing scripts for it.

Try it yourself

The datasets I built are public and free to use. You can query them directly, or clone one and extend it to fit your own project:

Quorel: quorel.vercel.app
Public datasets: quorel.vercel.app/datasets
My dashboard: diffboard.vercel.app

I've barely scratched what's possible here. Next I want to enrich the job listings with salary normalization and wire the Amazon dataset into a price-drop alert bot. If you build something on top of this, I'd love to see it.

I Found Firecrawl Too Expensive for My AI Agent's Knowledge Base, So I Built My Own

Samuel Raphael — Tue, 30 Jun 2026 22:47:51 +0000

Firecrawl is a great tool. I want to say that upfront, because what I'm about to describe isn't really Firecrawl's fault. It's just not what it was built for.

I was using it as the data layer behind an AI agent, and the costs kept climbing for a simple reason: every time my agent needed fresh context, I paid to re-fetch the same sources again. Same pages, same data, paid for again and again, just to get a raw dump I then had to clean up myself before my agent could even use it.

It felt wasteful. Not because the tool was bad, but because I was using a scraper to do a knowledge base's job.

The Difference That Was Costing Me Money

A scraper gives you data right now: raw, unstructured, one-time. A knowledge base gives you data that's always current, structured, and queryable.

I didn't need "fetch this page again." I needed something that already knew what I wanted, kept it fresh on its own, and let my agent ask for exactly the slice it needed, not the whole page dumped into context every single call.

Three things were actually breaking my workflow:

No structure. Raw scraped output meant my agent was spending half its context window parsing format instead of using data: nav menus mixed into content, tables that didn't parse cleanly, no consistency between pages.

No memory. I had no way to know what changed between fetches. Did a price update? Did a listing get removed? The only way to find out was diffing two raw dumps by hand.

No queryability. I couldn't ask for just the rows I needed. Every call meant everything, whether my agent needed it or not, straight into the context window, every time.

So I Built Quorel

Quorel turns any public website into a versioned, queryable API. You describe what you want in plain English, it extracts and structures the data against that schema, and it keeps itself current with a nightly refresh, no re-scraping on my end, no maintenance.

Every refresh that changes something meaningful creates a new immutable version, so I can see exactly what changed, roll back if something looks wrong, or freeze a version as a stable source while the live one keeps updating.

The part that actually changed my agent's workflow: Quorel ships with a native MCP server on every plan. My agent doesn't receive a page dump anymore, it calls a tool and gets back exactly what it asked for:

query_dataset(
  dataset_id: 1,
  keywords: "pricing tiers",
  filter_range: "updated_at:gte:2026-06-01",
  sort: "updated_at:desc",
  limit: 10
)

Structured, filtered, sorted, fresh: no glue code, no cleanup step, no paying again to re-fetch something I already had last week.

Why the Versioning Matters More Than I Expected

This is the part I didn't know I needed until I had it. An agent that reasons about change (what's different since last week, did this price move) genuinely can't do that with a plain scraper, because a fresh fetch only ever gives you now. It has no memory of then.

With versioned data, that comparison is just... there. Diff two versions, see what moved, done.

Where This Stands Right Now

I'll be straightforward: Quorel is early. I'm still building out the public dataset marketplace, so right now this is mainly for people who want to point it at their own sources and build their own structured, agent-queryable data layer, not yet a place to browse a library of ready-made datasets. That's coming soon, but I'd rather tell you where it actually stands than oversell it.

If you're building an AI agent that needs fresh, structured context from the web, and you're tired of paying to re-fetch the same pages over and over, it's worth a look. Free plan, no credit card, MCP server included from day one.

→ quorel.vercel.app
→ MCP docs
→ GitHub

Happy to answer questions about the architecture, the MCP server, or how the versioning works under the hood. This is genuinely the tool I use for my own agent's data layer now.