Notes from building the harness behind a 200-venue field study, and one finding that surprised us two months later.
The question sounds simple until you try to answer it. Can an AI assistant book a table at this restaurant? You cannot ask a model, because the model will tell you what it believes rather than what it can do. You cannot read the website, because a page that talks about reservations is not the same as a page that accepts one. And you certainly cannot book two hundred tables to find out.
Here is how we ended up measuring it, what broke, and why the result got more interesting in September than it was in July.
The measurement is where you stop
The harness drives a real browser through a venue's public booking path: find the entry point, open whatever widget or form is behind it, fill date, time and party size, and advance as far as the flow allows.
Then it stops, at the exact moment the flow asks for the customer's own details or offers a final confirmation.
That stopping point is the measurement, not a shortcoming of it. "Reached the step where only the guest's own name and phone remained" is precisely what "an assistant could have completed this" means, and it costs the restaurant nothing. Booking two hundred phantom tables to produce a statistic would be a strange way to argue that software should be more considerate.
How to test an AI agent without booking a table: three barriers
We kept all three, because a study that accidentally creates real reservations is not a study, it is a nuisance.
No personal data is ever typed. No name, no email, no phone, no card. The flow structurally cannot complete, even if everything else failed at once.
No confirm-class control is ever clicked. A regex over button labels and link text keeps the agent away from anything that reads like committing.
A network backstop. Requests matching submission patterns are aborted at the browser level. If the first two barriers both failed, nothing leaves the machine.
The user agent identifies the study honestly, with a contact URL. A crawler that hides is a crawler that knows it is doing something it should not.
Judge from controls, never from prose
The first version of the harness was too credulous. A page that said "reservations" scored as a page with reservations. A restaurant describing its private dining room registered as a bookable flow.
So the confirmation detector was rewritten to look only at real controls. It counts input elements whose name, id, placeholder, aria-label or type match personal-detail patterns, and it only counts them if the agent actually reached a booking surface first. Text is evidence of intent. Fields are evidence of capability. Only the second one is being measured.
The same discipline applies to blockers. The harness records what it hit by name: cookie_wall, captcha, login_wall, form_not_parseable, no_booking_entry. When we re-ran the sample with a newer version that distinguishes a consent banner from an anti-bot check, the totals moved. That is a change in the instrument, not a change in the world, and it has to be said out loud every time the two runs are compared. Two numbers from two different instruments are not a trend.
Browser agent vs API: driving a page is not calling an interface
This is the distinction the whole study turns on, and it is easy to blur.
An agent driving a browser is pretending to be a person. It moves a virtual mouse, waits for scripts, guesses which div is a date picker. It works, sometimes, slowly, and it breaks whenever the layout changes.
An agent calling an interface is doing something else entirely: a request, a schema, a typed response. Availability comes back as data. A booking comes back with an identifier you can check later.
Both can end in a reservation. Only the second is reliable enough to be worth building on, which is why the platforms that assistants actually integrate with expose the second kind.
So the study reports two separate numbers, and they answer different questions. Sixteen of two hundred could be taken to the final step by a browser agent. Zero of the hundred and sixty-three working sites published an interface an assistant could call.
Is llms.txt worth it? What 163 websites did in two months
Two months after the first run we checked the same 163 domains for the artefacts that make a business legible to software: an MCP manifest at /.well-known/mcp.json, an OpenAPI description, an AI plugin descriptor, an llms.txt.
Callable interfaces: still zero.
llms.txt: thirty-four.
In July only nine of those sites had an llms.txt. By September one in five restaurants in the sample had a plain-text document explaining itself to a language model. Hours, cuisine, atmosphere, the things you would tell a guest.
And not one of them published a way for that model to reserve anything.
It is a good instinct aimed at the wrong layer. llms.txt improves how accurately you are described. It does nothing for whether you can be used. An assistant that reads your file and cannot act on it will describe you correctly to the customer and then complete the booking somewhere else.
If you are building for this, the shape of the answer is not complicated. Something that answers three questions with data rather than pixels: what do you offer, when are you free, and can you take this reservation now. It can be an OpenAPI surface, an MCP server, or anything else with a schema on both ends. What matters is that a machine gets a typed answer instead of a screenshot.
AI agents and captchas: the uncomfortable middle
There is a category of failure that is nobody's fault, and it is bigger than we expected.
At 26 of the 200 the agent ran into the venue's own protection: an anti-bot check, a captcha, a device verification. Those were installed on somebody's advice, for real reasons, and they do not distinguish a scraper from the customer's assistant.
The right conclusion is not "turn off your captcha". A consent banner is not security at all, since a scraper never renders it. A captcha is real security, but it is usually applied to the whole site, including the part that only reads opening hours and free tables. Those are public facts, given out over the phone all day long. Protect the write path, where damage is possible. The read path is answering a question a stranger could ask by calling.
Worth noting how varied this is in practice: two venues on the same booking system, in the same run, produced opposite results. One stopped the agent at a cookie banner; on the other the agent reached confirmation. Same software, different configuration. Which is oddly encouraging, because configuration is fixable.
An AI agent readiness checklist for a booking business
- A booking entry point an agent can find from your own website, not only from a marketplace.
- Date, time and party size as real form fields, not pictures or a chat.
- No captcha or device check on the part that only reads hours and free tables.
- A consent banner that does not cover the booking form.
- Typed answers to three questions: what you offer, when you are free, whether you can take this booking now (OpenAPI, MCP or another schema).
- An llms.txt if you like, knowing it describes you and does not make you bookable.
Method, aggregate data and the full funnel are public, and no venue is named as a low scorer: g-lab.studio/research/amsterdam-restaurants-2026
Disclosure: we also build G-Guest, a booking layer with a callable interface. The harness came first and told us what to build; the study names no venues and scores nothing in our favour.
Originally published at https://g-lab.studio/research/measuring-agent-bookability
Top comments (1)
That stop condition is a much better definition of capability than a yes/no claim. I’d record the exact UI state reached, the venue’s booking provider, and the reason the flow stopped, then replay a sample periodically to separate real regressions from site redesigns. It keeps the measurement useful without turning the benchmark into a source of unwanted reservations.