DEV Community: Samuel Raphael

If your site shows up in ChatGPT, does that mean Claude and Perplexity will show it too?

Samuel Raphael — Tue, 18 Aug 2026 19:46:35 +0000

No. And most people optimizing for AI visibility right now don't realize this.

I went down a rabbit hole this week on how search behavior has shifted since AI chatbots went mainstream, and a few numbers stopped me.

Gartner predicted traditional search volume would drop 25% by 2026 because of AI chatbots. A mid-2026 study covering 35+ billion monthly searches found the real number is 29%. AI beat the prediction.

Zero-click searches, where you get an answer and never visit a site, now make up close to 60% of all searches. When an AI Overview shows up, that number jumps to roughly 83%.

Google still holds about 90% of traditional search. But when you count AI chat as part of "how people find information," Google's share of that combined pie has reportedly fallen from around 89% to under 60% in three years.

The bigger surprise

The "does one AI's trust transfer to all AI" question. It mostly doesn't:

Different retrieval. Perplexity searches live almost every time. ChatGPT and Claude often answer from training data and only search when needed. Same question, completely different sourcing.
Different indexes. These platforms don't all pull from the same search backend.
Training data and live citation are separate channels. A model "knowing" about you from training isn't the same as it citing you live.

GEO is becoming a real discipline

Some call it GEO (Generative Engine Optimization), some call it AEO (Answer Engine Optimization). Same idea: instead of optimizing to rank in a list of links, you're optimizing to become the answer itself.

A few things that actually seem to move the needle:

Put your direct answer in the first 100-150 words. One analysis found 44% of AI citations pull from the first third of a page, mostly the intro.
Back claims with real numbers and named sources instead of hedging language.
Brand mentions across the web now matter more than backlinks for getting cited by AI. One study found mentions correlate with AI citation about 3x more strongly than backlinks do.
Keep content fresh. AI systems seem to favor recently updated pages.

This isn't a replacement for SEO, it sits on top of it. You still need a fast, crawlable, well-structured site. GEO is the next layer.

Where this is heading

It's also why I've been watching what tools like Landing Creator are building. Instead of manually writing hundreds of location or offer-specific pages, it generates them directly structured for both Google and AI citation, schema markup, FAQ blocks, internal linking, the works, and maintains an llms.txt index so AI crawlers can discover what a site actually offers. That's a fairly direct answer to the shift these numbers are describing.

Curious what others are seeing in their own traffic.

Gnat, self-hosted analytics as a single Go binary, with a built-in MCP server

Samuel Raphael — Fri, 14 Aug 2026 19:20:29 +0000

A few months back I was crawling the web for a side project and came across an analytics platform's data that had exposed users' raw IPs. It wasn't malicious, just careless. But it stuck with me: most people using these tools have no idea their visitors' data is sitting around like that.

That's the moment that pushed me to build Gnat, a self-hosted analytics tool that doesn't cut corners on privacy, but also doesn't force you into a painful setup to get it.

No third party, no phoning home

Gnat runs entirely on infrastructure you control. There's no third party in the loop, no API calling home anywhere, and no data leaving your server. It's also completely free and open source, the whole thing is on GitHub.

One binary, zero bloat

Most self-hosted analytics tools make you choose: either something lightweight with barely any real analysis, or something with real analytical depth that also means Docker, a multi-service stack, and its own ops burden just to keep running.

Gnat is a single Go binary. No CGO, no Docker, no multi-service stack. The dashboard (HTML, CSS, JS) is compiled directly into the executable, so there's nothing else to install alongside it.

tar xzf gnat-vX.Y.Z-linux-amd64.tar.gz
cd gnat-vX.Y.Z-linux-amd64
./gnat

That's the whole install.

Real analytical depth

Despite being one binary, Gnat doesn't skimp on features:

Pageviews and custom events via a simple JSON POST (no per-language SDKs needed)
Funnels, cohorts, retention curves
Auto-discovered path analysis
SQLite by default, or Postgres/MySQL if that's how you already run infrastructure
CSV, JSON, jsonL, it's your data

Built for the agent era

The part I had the most fun building: Gnat ships with a native MCP server. Claude or any MCP-compatible agent can query your analytics directly, ask what changed, pull a report, whatever you need.

Where it's at

This is early, v0.1.0, and I'm actively building on it. Multi-site support, SSO/team access, and managed hosting are all on the roadmap but not built yet, the README is upfront about what's real today versus what's planned.

Gnat is licensed under AGPLv3, free to self-host and modify for any purpose, including commercially, but if you run a modified version as a service for others, the source has to stay open under the same terms.

Repo: https://github.com/var-raphael/Gnat

Feedback, issues, and contributions are very welcome. I'd love to hear what you think, especially if you've felt the same Docker-and-multi-service fatigue trying to self-host analytics.

Firecrawl vs Apify vs Quorel

Samuel Raphael — Sat, 25 Jul 2026 12:04:12 +0000

Firecrawl and Apify both do the hard part well. They render JavaScript, dodge bot detection, and hand you back clean markdown or structured JSON. If you need a page turned into text, either one will do that job.

But most of the time the point isn't a single page. It's a page you need to check again tomorrow, and the day after, and the day after that: a price, a listing, a status, a set of rows that might quietly change. And that's where the workflow, not the extraction, starts costing you.

Firecrawl bills per credit, and the credit math is not what the pricing page implies. Scrape is 1 credit. Add JSON extraction and Enhanced Mode — both of which you probably want for structured data — and it's 9 credits per page. A $16/mo Hobby plan advertised at 3,000 credits works out closer to 300–500 pages once you're actually extracting fields, not just fetching markdown. Credits also don't roll over, so a quiet month doesn't bank anything for a busy one.

Apify flips the complaint around. Pricing is compute-unit based (RAM × time) plus separate line items for proxy and storage. It's flexible and it scales, but you can't look at a number on the pricing page and know what a job will cost until you've run it. For teams that want a predictable bill, that unpredictability is the whole problem.

Neither tool keeps history by default. You get the current state of the page. If you want yesterday's version to compare against today's, that's on you to build — a database, a diffing script, a place to store snapshots, a way to roll back if a refresh comes back wrong.

Quorel starts from a different question: not "how do I fetch this page," but "how do I keep a page as structured, versioned data over time." You describe the fields you want in plain English. Every refresh (nightly by default, or on demand via a ping URL, webhook, or your own cron) is saved as its own immutable version. Nothing is overwritten. You can diff any two versions to see exactly what changed, and roll back in one click if a refresh comes back bad.

Pricing is flat, not metered. Free tier is one dataset, 20 URLs, full version history included. Pro is $19/mo for 5 datasets and 100 URLs, with on-demand refresh and webhooks. No credit multipliers, no compute-unit math, no surprise bill.

This isn't a claim that Quorel replaces Firecrawl or Apify at everything they do. If you need to fetch an arbitrary one-off page as clean markdown for an LLM right now, Firecrawl is built for exactly that. If you need thousands of site-specific scrapers off a marketplace, Apify's Actor ecosystem has years of head start. Quorel is built for the narrower, recurring job: turning a page you check regularly into a structured, versioned API you can trust and query later.

Every dataset ships with an MCP server on every plan, free included, so an agent can query it directly — or pull the raw rows, fix nulls, and publish a new version itself.

If you've been burned by a scraper that silently broke, or wished you'd kept last week's data to compare against, that's the specific gap Quorel is built to close.

I'm building Quorel — happy to answer questions on versioning, pricing, or how the MCP integration works.

I Built a Job Listing + E-commerce API Dataset in 4 Hours, No Code

Samuel Raphael — Fri, 24 Jul 2026 21:27:44 +0000

The problem with "just build your own API"

If you've ever wanted a personal dataset (job listings, product prices, whatever) you know the real cost isn't the idea, it's the pipeline. Scraping. Cleaning. Storage. Refreshing on a schedule. Serving it over an API. Handling schema drift when the source site redesigns. Every one of those is its own mini-project, and each is its own source of friction.

I wanted a dataset combining Amazon products for developers and job listings scraped from multiple sites. Normally that's a multi-day side project. I built it in about 4 hours, without writing a single line of scraping or backend code.

Here's how.

The magic part: describing data instead of scraping it

I used a tool called Quorel. The workflow was almost embarrassingly simple:

I pasted in my source URLs (Amazon product pages, a few job board listing pages).
I described the schema and extraction intent in plain English: no selectors, no XPath, no code. Something like "product name, price, rating, whether it's in stock" for the Amazon side, and "job title, company, location, salary range, posted date" for the listings side.
I hit run and left it alone for 30–60 minutes.

When I came back, the data was fully structured and ready to query.

From there I asked Claude to build me a dashboard on top of the API so I could actually see and explore the data. That part took the rest of the 4 hours. The result is live here: diffboard.vercel.app.

The part that actually sold me: it's versioned like git

Structuring the data was the easy win. What kept me around is that Quorel treats datasets like a git repo for live web data:

Nightly refresh (or your own cron schedule): the data doesn't go stale.
Every refresh is a saved version, nothing overwrites the last run.
Diffing between versions: I can see exactly what changed, like prices that moved, listings that disappeared, new rows that showed up.
Time travel: I can pull the dataset as it looked days or months ago.
Rollback: a bad refresh doesn't mean starting over.
Forking: you can clone a public dataset the way you'd fork a repo, and extend it for your own use case.

That combination, structure plus history, is the part a plain scraper never gives you. Most tools hand you today's snapshot. This hands you the whole timeline.

Talking to your data instead of querying it

The other piece that stood out: Quorel ships an MCP server with every dataset, on every plan, including free. That means I can point an AI agent at my dataset and just talk to it, like "find listings with price drops this week" or "fix the null prices and publish a new version," and it does the pull, clean, and push cycle for me, still landing in the same versioned system. It's genuinely like chatting with your data pipeline instead of writing scripts for it.

Try it yourself

The datasets I built are public and free to use. You can query them directly, or clone one and extend it to fit your own project:

Quorel: quorel.vercel.app
Public datasets: quorel.vercel.app/datasets
My dashboard: diffboard.vercel.app

I've barely scratched what's possible here. Next I want to enrich the job listings with salary normalization and wire the Amazon dataset into a price-drop alert bot. If you build something on top of this, I'd love to see it.

I Found Firecrawl Too Expensive for My AI Agent's Knowledge Base, So I Built My Own

Samuel Raphael — Tue, 30 Jun 2026 22:47:51 +0000

Firecrawl is a great tool. I want to say that upfront, because what I'm about to describe isn't really Firecrawl's fault. It's just not what it was built for.

I was using it as the data layer behind an AI agent, and the costs kept climbing for a simple reason: every time my agent needed fresh context, I paid to re-fetch the same sources again. Same pages, same data, paid for again and again, just to get a raw dump I then had to clean up myself before my agent could even use it.

It felt wasteful. Not because the tool was bad, but because I was using a scraper to do a knowledge base's job.

The Difference That Was Costing Me Money

A scraper gives you data right now: raw, unstructured, one-time. A knowledge base gives you data that's always current, structured, and queryable.

I didn't need "fetch this page again." I needed something that already knew what I wanted, kept it fresh on its own, and let my agent ask for exactly the slice it needed, not the whole page dumped into context every single call.

Three things were actually breaking my workflow:

No structure. Raw scraped output meant my agent was spending half its context window parsing format instead of using data: nav menus mixed into content, tables that didn't parse cleanly, no consistency between pages.

No memory. I had no way to know what changed between fetches. Did a price update? Did a listing get removed? The only way to find out was diffing two raw dumps by hand.

No queryability. I couldn't ask for just the rows I needed. Every call meant everything, whether my agent needed it or not, straight into the context window, every time.

So I Built Quorel

Quorel turns any public website into a versioned, queryable API. You describe what you want in plain English, it extracts and structures the data against that schema, and it keeps itself current with a nightly refresh, no re-scraping on my end, no maintenance.

Every refresh that changes something meaningful creates a new immutable version, so I can see exactly what changed, roll back if something looks wrong, or freeze a version as a stable source while the live one keeps updating.

The part that actually changed my agent's workflow: Quorel ships with a native MCP server on every plan. My agent doesn't receive a page dump anymore, it calls a tool and gets back exactly what it asked for:

query_dataset(
  dataset_id: 1,
  keywords: "pricing tiers",
  filter_range: "updated_at:gte:2026-06-01",
  sort: "updated_at:desc",
  limit: 10
)

Structured, filtered, sorted, fresh: no glue code, no cleanup step, no paying again to re-fetch something I already had last week.

Why the Versioning Matters More Than I Expected

This is the part I didn't know I needed until I had it. An agent that reasons about change (what's different since last week, did this price move) genuinely can't do that with a plain scraper, because a fresh fetch only ever gives you now. It has no memory of then.

With versioned data, that comparison is just... there. Diff two versions, see what moved, done.

Where This Stands Right Now

I'll be straightforward: Quorel is early. I'm still building out the public dataset marketplace, so right now this is mainly for people who want to point it at their own sources and build their own structured, agent-queryable data layer, not yet a place to browse a library of ready-made datasets. That's coming soon, but I'd rather tell you where it actually stands than oversell it.

If you're building an AI agent that needs fresh, structured context from the web, and you're tired of paying to re-fetch the same pages over and over, it's worth a look. Free plan, no credit card, MCP server included from day one.

→ quorel.vercel.app
→ MCP docs
→ GitHub

Happy to answer questions about the architecture, the MCP server, or how the versioning works under the hood. This is genuinely the tool I use for my own agent's data layer now.