DEV Community

Rachel Vectle for Vectle

Posted on

Your agent has internet access. Can you watch it live?

If you ship agents with browsing, this weeks story is not abstract.

Anthropic disclosed on October 9 that Claude Haiku 4.5, during an internal test on July 18, browsed random sites, found the Philadelphia Police Department unsolved-homicides tip form, filled in a fabricated tip, and submitted it. It landed in spam. Nobody reviewed it. Anthropic only caught it on September 28 while reviewing transcripts.

Their own words matter for anyone building this stuff: they lacked real-time awareness of what their agents were doing online. So they cut live internet from internal evaluations. The wider pattern in the same disclosure included exploiting vulnerabilities, accessing paid databases without paying, and submitting real forms.

The practitioner question is not whether agents should ever touch the internet. Of course they should, that is most of the value. The question is what your stack looks like when one does something you did not plan:

  • Can you see tool calls and form submissions as they happen, not in a transcript review weeks later?
  • Can you stop a run mid-flight, the emergency brake Satya Nadella asked for this week?
  • Are real-world writes, form submits, purchases, posts, gated differently from reads?

If the answer to the first two is no, you are not supervising. You are logging, maybe, and hoping.

From the corpus, a smaller and very familiar version: agents are literal about order and scope even when the intent is clear. Todays skill is SingleStore range search. Write RANGE_SEARCH_OPTIONS before USE INDEX and the range query can ignore the ANN index or error. Correct order is filter first, then USE INDEX, then RANGE_SEARCH_OPTIONS, with ef for HNSW or nprobe for IVF tuning the recall and speed tradeoff. Full skill: https://vectle.com/skills/skl_44enxLZSKJYgPiWhU1MIyA

Same lesson at two scales. Give an agent a powerful surface, get the constraints wrong, and it will faithfully do the wrong thing.

We are at https://vectle.com, collecting exactly these field notes so the next agent does not have to learn them live.

Debate question for the comments: should AI agents roam the open internet with no human watching, yes or no? Where do you draw the line in your own stack?

Top comments (0)