Not looking for a highlight reel of prompt injection screenshots, more interested in the mundane stuff: the agent that called the wrong tool with confidently valid-looking arguments, the multi-turn conversation that drifted somewhere nobody planned for by turn six, the endpoint that worked fine in every test case and then did something strange the first week it saw real traffic.
If you've shipped an agent, something has probably slipped through. A few prompts in case they help:
What kind of test coverage did you have going in, unit tests on tools, eval sets, manual QA?
Where did the actual failure show up: a tool call, a conversational turn, an integration between two systems that looked fine in isolation?
Once you saw it, did you write a test for it? Did that test generalize, or did it only catch that exact case again?
At Humanbound, we build agent-native testing (endpoint configs, multi-turn, tool misuse), and we keep running into the same pattern: most of the existing OSS tooling is really good at catching what you already thought to test for, and the interesting failures are always the ones nobody wrote a test for yet.
Our goal here is to turn the replies into a big, running list of issues we can actually work through, and to have it live somewhere the community can see all of it in one place rather than scattered across DMs and one-off conversations. So the more specific and rough, the better, this is meant to become a shared backlog, not just a comment thread.
Drop your worst one below, we'll be reading every reply.
We've also got a thread going on GitHub Discussions if you'd rather post there or want to see what's already been shared: https://github.com/humanbound/humanbound/discussions/95
Top comments (0)