Why I Built IntentQL: A Protocol for AI Agents to Use the Web Natively

Lee Manley — Sun, 19 Oct 2025 13:42:26 +0000

For years, web scraping has been the duct tape behind AI agents trying to interact with websites. It works—until it doesn't. HTML changes, pages lag, and what should be a 100ms query becomes a 4-second screen-parsing mess.

I believe the web needs native infrastructure for agents. Just as robots.txt standardized how crawlers engage with websites, it's time for something similar for AI agents: a declarative, structured protocol.

So I built IntentQL.

🔧 The Problem: Scraping Is a Bottleneck

When an AI agent wants to book a hotel, it doesn't "understand" the page—it scrapes it. That means parsing layout-heavy HTML, guessing at form inputs, and praying nothing broke.

A recent test showed:

Scraping the hotel booking UI took 4.46 seconds

IntentQL took 0.049 seconds

That's a 91x performance improvement. But more importantly, it's stable. HTML can change without breaking the contract.

🌐 What Is IntentQL?

IntentQL is a declarative JSON endpoint websites can serve (like Agent.json) to expose agent-compatible actions. These actions are structured, timestamped, and composable.

An agent can ask: "What can I do here?" — and get a machine-readable list of intents like:

{
"intents": [
{
"name": "book_room",
"description": "Book a hotel room",
"parameters": ["check_in", "check_out"]
}
]
}

No scraping. No guessing. No fragility.

🚀 Live Demo

I built a live side-by-side demo:

https://widget-k3vj-a6qvo466v-lee-manleys-projects.vercel.app/

Left side: full HTML scraping

Right side: direct JSON query using IntentQL

Includes timestamps + graphics

Try it. Embed it. Share it. This is how agents should interact with the web.

🚧 Who Is This For?

Browser teams (Opera, Arc, Perplexity): no more reverse-engineering

Web devs: expose actions natively without API bloat

Standards bodies: time to start the conversation

If you're working on agent-first experiences, this is the missing layer.

👀 What Happens Next?

I'm sharing this across Hacker News, Twitter, and with the W3C AI Agent Protocol community. This isn't a product pitch—it's infrastructure. I'm here to help define the standard.

Spec: https://intentql.dev
Contact: lee@intentql.dev

Want to implement this on your site? There's a one-file guide. Need help? I offer consulting.

Let's give AI agents a better way to use the web.

Built in the open. Maintained at intentql.dev.

The Agentic Web Needs a Protocol

Lee Manley — Fri, 01 Aug 2025 09:29:54 +0000

"We built HTML for humans. But we never built a markup for machines."

Agentic browsers are coming.
Opera just announced Neon, their first browser that takes actions on your behalf. Others will follow. Everyone’s racing to build smarter interfaces that understand intent and can act autonomously.

But here’s the problem:

The web isn’t ready.
Even the smartest AI agents today are stuck scraping HTML, reverse-engineering buttons, guessing what elements do — and breaking every time a class name changes.

We’ve made browsers agentic.
But we’ve left the web opaque to agents.

Enter: IntentQL
IntentQL is a declarative, SQL-inspired protocol for describing how agents can interact with your site.

It flips the model:

Sites publish an agent.json file — like an API manifest for AI.

Agents send SQL-like queries to express what they want.

Responses come back as structured data — not HTML hacks.

sql
Copy
Edit
SELECT name, price, rating
FROM amazon.products
WHERE category = 'laptops'
AND price < 1000
ORDER BY rating DESC;
This isn’t scraping.
This is structured, intent-driven interaction — designed for agents, not eyeballs.

Why now?
Three forces are colliding:

Multimodal AI is better than ever at understanding human intent.

Agentic browsers are shipping (like Opera Neon).

The economic value of automating web tasks is massive — but blocked by fragile scraping.

We need a foundation.
A shared layer of intent between AI agents and websites.

This isn’t a product. It’s a protocol.
IntentQL is early. It’s not fully spec’d yet.
But it’s being built out in the open — designed to serve any agent, any site, any stack.

If you're a developer, a standard-maker, or just someone tired of fragile scraping…
You’re invited.

🔗 GitHub

🌐 intentql.dev

💬 Follow the journey — more soon

DEV Community: Lee Manley

Why I Built IntentQL: A Protocol for AI Agents to Use the Web Natively

The Agentic Web Needs a Protocol