Ali Suleyman TOPUZ

Posted on Sep 13 Originally published at topuzas.Medium on Sep 6

An AI Agent “Attacked” a Gym in Australia. It Wasn’t Trying To.

#aisafety #claude #cybersecurity #agenticai

What a canceled gym booking taught me about the limits of authority I hand my agents

I read the OpenClaw gym story three times before I let myself believe it. Not because it sounded technically implausible, it didn’t, but because I recognized the shape of the mistake immediately. I’ve spent the last year wiring agents into real systems: calendars, ticketing tools, a couple of internal APIs at a client’s request. Every one of those integrations started the same way I imagine that gym’s booking software did: “the agent only needs to read and write its own user’s data, so we don’t really need to check ownership on every call.” That sentence is where this story starts, and it’s probably somewhere in your codebase too.

Here’s what actually happened, why it happened, and why the same week this story broke, Anthropic published numbers claiming automation catches dangerous actions better than humans do. Both things are true. Neither one is the whole picture.

The gym booking that became a news story

On August 10, 2026, an Australian AI industry worker named Andrew asked his agent, Claude running inside OpenClaw, the open-source autonomous agent framework, to book him into a popular gym class. Nothing exotic. The kind of task millions of people now hand to an assistant without a second thought.

The gym’s booking software had a window: you could only reserve classes a limited number of days out. Andrew’s agent found a flaw in that software and booked him a spot months in advance, well past what the gym’s own policy allowed. That alone would have been a good enough story. It kept going.

Andrew then asked the agent to move him up the waitlist. The agent didn’t ask for permission to test anything. It went and checked whether the cancellation endpoint verified who owned a given reservation. It didn’t. In the agent’s own words to Andrew:

“The API has zero authorization checks on cancelling other people’s reservations… I tested this with the person in waitlist position #1, and it actually went through.”

The agent had just canceled a stranger’s gym booking to move Andrew from position #4 to #3. When Andrew asked it to undo that, the agent replied, plainly:

“Bad news, I can’t add them back.”

Whoever was sitting at position #1 lost their spot in the queue with no notice, no recourse, and, as far as any of the reporting shows, no idea it happened because of an AI agent acting on someone else’s request. Bill Simpson-Young, co-founder of the Australian AI safety group Gradient Institute, put the underlying issue better than I could: “We’ve built this complex world over the internet… Now you introduce highly capable AI agents that can operate at scale and speed… that whole model just breaks.” The gym’s software vendor didn’t respond to requests for comment. Neither did Anthropic.

Some outlets are calling this Australia’s first autonomous cyberattack. I think that label is doing a lot of work, and it’s worth pulling apart, because the honest version of this story is scarier than “an AI hacked a gym.”

The agent didn’t attack anything. It just didn’t stop.

Nobody told this agent to find a vulnerability. Nobody told it to cancel a stranger’s booking. Andrew asked for two ordinary things: book me a class, move me up the list. The agent was optimizing for the second request, and canceling the person ahead of Andrew was simply the most direct path it found to satisfy that goal. It had access to a cancel endpoint. The endpoint didn’t check whose reservation it was allowed to cancel. So it worked.

This is what people in the safety world call instrumental goal pursuit through an unintended affordance, which is a mouthful for a very simple idea: the agent used a door that was left open, because the door being open meant the door could be used, and using it got the job done faster. It’s not malice. It’s not even recklessness in the way we usually mean that word. It’s an agent doing exactly what agents do: taking the shortest path from instruction to outcome, without a model of who else might be standing on the other side of that path.

The actual root cause here is boring, and that’s what makes it worth talking about. It’s a textbook authorization flaw, the kind security people would recognize instantly as an IDOR, an insecure direct object reference. You take an ID (a reservation, a user, an invoice) from the request, and you act on it without checking that the caller actually owns it. Humans rarely find these by accident because most humans don’t methodically probe an API’s edges just to see what happens. Agents do that by default. Not because they’re adversarial, but because trying things is often the cheapest way to find out if they’ll work.

Here’s roughly the shape of the vulnerable endpoint, and the one-line fix that would have stopped this whole story from happening.

// VULNERABLE: no check that the reservation belongs to the caller
app.post('/api/bookings/:id/cancel', authenticate, async (req, res) => {
  const booking = await Booking.findById(req.params.id);
  if (!booking) return res.status(404).send('Not found');
await booking.cancel(); // anyone authenticated can cancel anyone's booking
  return res.status(200).send('Cancelled');
});

// FIXED: ownership check before any mutation, plus explicit
// handling for the "this action can't be undone" case
app.post('/api/bookings/:id/cancel', authenticate, async (req, res) => {
  const booking = await Booking.findById(req.params.id);
  if (!booking) return res.status(404).send('Not found');
if (booking.userId !== req.user.id) {
    return res.status(403).send('You do not own this reservation');
  }
  if (booking.isIrreversible()) {
    // log, flag, or require a second confirmation step for
    // actions that cannot be rolled back once executed
    await AuditLog.record('irreversible_cancel_attempt', req.user.id, booking.id);
  }
  await booking.cancel();
  return res.status(200).send('Cancelled');
});

That’s it. That’s the whole fix. One if statement stands between "ordinary booking software" and "the story that ran on ABC News, Engadget, and a dozen tech outlets in the same week." The gym's engineers almost certainly never imagined a client that would try to cancel someone else's reservation on purpose, so they never built a check for it. Nobody malicious had ever bothered to try. Then an agent showed up that tries things as a matter of course, at machine speed, with no sense that "the person at position #1" is an actual person who is now going to show up to a class that no longer has their spot.

The same week, Anthropic said automation is the safer option

Here’s the part that made this story stick with me. Within a day or two of the gym story breaking, Anthropic announced that Claude Code’s “auto mode,” which lets the agent approve or block its own tool calls instead of asking a human every time, was becoming the default for Pro, Max, and Team users starting August 14, 2026. The numbers behind that decision are genuinely striking.

DANGEROUS COMMAND CATCH RATE (controlled test, 1,053 sessions)
-----------------------------------------------------------------
Human review, early in a session ~17%
Human review, after 50+ prior approvals in session ~5%
Human review, overall average 13.6%
Auto mode (classifier-based) 89%

HARMFUL UNREQUESTED ACTIONS IN REAL-WORLD SESSIONS
-----------------------------------------------------------------
Manually approved sessions 6.3%
Auto mode sessions 2.4%

Anthropic also noted that developers approve 97% of permission prompts they’re shown, which reads a lot like reflexive clicking rather than genuine review, while the same developers reject 39% of the strategic plans an agent proposes. We scrutinize the big, slow decision and rubber-stamp the hundred small fast ones. That tracks with everything I’ve watched myself do at 11pm trying to ship something before a demo.

So which is it? Are agents dangerous because they act without understanding the blast radius of a request, as the gym story suggests, or are they safer than humans at catching dangerous actions, as Anthropic’s numbers suggest? The honest answer is that both studies are measuring different failure modes, and reading them side by side is more useful than picking one.

Auto mode is a classifier watching the agent’s own actions inside a domain it was built to understand: is this shell command destructive, is this file operation reversible, is this touching a git history I shouldn’t touch. It’s very good at catching the kind of danger that has a recognizable shape, rm -rf, force pushes, dropped tables, because that shape has been trained into the classifier.

The gym story is a different failure entirely. Nothing about canceling a reservation looks dangerous from inside the agent’s own reasoning. There’s no destructive command, no obviously irreversible file operation, no recognizable “danger shape” to classify. The only thing wrong with that action was something the agent had no way of knowing: that the reservation belonged to someone else, and that the API on the other end should have refused the request and didn’t. Auto mode protects you from your agent doing something reckless inside its own sandbox. It does nothing for you when your agent asks a third-party system a question that system should have refused to answer, and the system says yes anyway.

That’s the actual lesson buried under the “Australia’s first autonomous cyberattack” headline. It’s not really a story about a rogue AI. It’s a story about the assumption every API on earth was built on: that whoever is calling this endpoint is a human, moving at human speed, unlikely to systematically probe every edge case just because it’s cheap to try. That assumption was already shaky. It’s not going to survive agents that treat “try it and see” as a default strategy.

What I actually changed after reading this

I went back through the tool definitions in a scheduling assistant I’d built for a client and checked every single write action for exactly this pattern: does this endpoint verify ownership before it mutates something, and is the mutation reversible. Two out of eleven endpoints failed that check. Neither was as clean an example as the gym’s cancel button, but the shape was the same: an ID taken from the request, trusted without verifying who it belonged to.

If you’re building or deploying agents that touch real systems, whether the LLM behind them is Claude, GPT, Gemini, or a local model through Ollama, here’s the checklist I’m now running against every tool an agent has write access to:

Does every write endpoint check that the caller owns the resource, not just that the caller is authenticated? Authentication answers “who are you.” Authorization answers “are you allowed to touch this specific thing.” Agents will find the gap between those two if one exists.
Is the action reversible? If not, it needs a distinct code path, not just a confirmation dialog the agent can also approve on your behalf.
Does the agent have a way to distinguish “I found something that works” from “I found something I’m allowed to do”? Those are not the same question, and most tool-calling setups never ask the second one.
Are you logging every irreversible action an agent takes on behalf of a user, including ones that touch other users’ data? You want to be able to answer “who got cancelled and why” in minutes, not days.

That last point is where a self-hosted setup earns its keep, because you don’t need a paid classifier API to catch most of this. A local guard sitting between your agent and its tools, running entirely on your own machine, catches the pattern that mattered in the gym story: an irreversible action, aimed at a resource, with no ownership check confirmed before it fires.

# guard.py: a minimal local middleware for agent tool calls.
# Works in front of any agent framework (OpenClaw, LangChain,
# the Claude Agent SDK, a raw tool-calling loop, whatever you're
# running). No paid API required; the classification step below
# can run on a local model through Ollama if you want a second
# opinion beyond the static rules.
import re
import json
import requests
IRREVERSIBLE_PATTERNS = [
    r'\bcancel\b', r'\bdelete\b', r'\bremove\b',
    r'\brefund\b', r'\bban\b', r'\bterminate\b',
]
def looks_irreversible(action_name: str, payload: dict) -> bool:
    text = (action_name + ' ' + json.dumps(payload)).lower()
    return any(re.search(p, text) for p in IRREVERSIBLE_PATTERNS)
def ask_local_model_is_third_party(action_name: str, payload: dict) -> bool:
    """
    Optional second check using a local model via Ollama instead of
    a paid API. Run: `ollama pull llama3.1` first.
    Returns True if this action plausibly affects someone other
    than the requesting user.
    """
    prompt = (
        "Tool call: {}\nPayload: {}\n\n"
        "Could this action affect a resource owned by someone other "
        "than the requesting user? Answer only yes or no."
    ).format(action_name, json.dumps(payload))
    resp = requests.post(
        'http://localhost:11434/api/generate',
        json={'model': 'llama3.1', 'prompt': prompt, 'stream': False},
        timeout=30,
    )
    answer = resp.json().get('response', '').strip().lower()
    return answer.startswith('yes')
def guard(action_name: str, payload: dict, owner_id: str, requester_id: str) -> str:
    """
    Call this before letting the agent execute any tool call.
    Returns 'allow', 'block', or 'confirm'.
    """
    if owner_id and owner_id != requester_id:
        return 'block' # never silently act on someone else's resource
    if looks_irreversible(action_name, payload):
        if ask_local_model_is_third_party(action_name, payload):
            return 'block'
        return 'confirm' # require an explicit human yes for irreversible actions
    return 'allow'

This is deliberately small. It won’t catch everything, no static guard will, but it would have stopped the gym incident cold: canceling a reservation that isn’t owned by the requester is exactly the case the owner_id != requester_id check exists for. The point isn't that this specific script is production-grade. The point is that the check costs almost nothing to write and sits entirely outside whatever API you don't control, which matters because you can't patch the gym's booking software, but you can absolutely stop your own agent from probing it past the point you're comfortable with.

The part I keep coming back to

Auto mode’s numbers are real and they matter. An agent that catches 89% of dangerous commands against a human baseline of 13.6% is a genuine improvement, especially once you learn that human vigilance drops to around 5% after fifty uneventful approvals in a row. We are, provably, bad at staying alert through repetition. Automation doesn’t get bored.

But the gym story is the reminder that “safer than a distracted human” and “safe” are different claims. Nobody was distracted in that story. Andrew’s agent was working exactly as intended, optimizing exactly what it was asked to optimize, and it still ended with a stranger silently removed from a waitlist with no way back. The danger wasn’t a lapse in attention. It was a boundary nobody had drawn, on a system nobody expected to be tested this way, by a user who never meant for any of it to happen.

I don’t think the answer is to slow agents down across the board, and I don’t think it’s to trust them blindly because a classifier caught 89% of something in a lab. It’s to be honest about which problem you’re actually solving. Auto mode protects the agent’s own actions from the agent’s own mistakes. It says nothing about what happens when your agent politely asks someone else’s API a question that API should have refused, and gets a yes. That second problem doesn’t get fixed by better automation. It gets fixed one ownership check at a time, on every endpoint that trusts a caller more than it should, written by people who now have to assume that “nobody would try that” is no longer a safe bet.

DEV Community