SWARM: What Happens When AI Users Collide?

Yash Vasudeva — Wed, 04 Nov 2026 10:46:41 +0000

This is a submission for the MLH x DEV Writing Challenge.

What I Built

Most apps get tested one user at a time. Real apps don't get that luxury.

Picture an online marketplace with exactly one item left in stock. Two buyers hit "purchase" at the same moment. Both checkouts go through, and now the store has sold two items it only had one of.

Look at either buyer's session on its own and nothing is wrong. The bug only exists when the two of them show up together.

That's the problem I wanted to dig into at HackCBS 9.0, and it's how I ended up building SWARM.

SWARM is a multi-agent AI testing system. It drops several goal-driven AI users into the same running application and watches for failures that only happen because those users interact.

Rather than leaning on a fixed set of test scripts, SWARM spins up AI actors that each want something different:

Buyers fighting over limited inventory.
Bargain hunters trying to squeeze every last bit out of a coupon.
Fraud-minded users poking around refunds and disputes.
Sellers managing orders and blocking people.
Other actors testing the edges of what the app lets them do.

They all work on the same app and share the same state. That's where the interesting collisions come from, the kind a single-user test would probably never trigger.

Spotting something suspicious is only half the job, though. SWARM records the requests, checks the app's invariants, replays suspicious sequences on a fresh copy of the application, and then removes actors one at a time to see whether the failure really depends on them interacting.

The core idea: AI agents explore, executable checks verify.

I ran a controlled evaluation with Groq's openai/gpt-oss-120b: 10 AI sessions across three held-out seeds. SWARM found three distinct, verified interaction failures that the isolated-agent baseline missed:

Inventory overselling: Two buyers bought an item when only one unit existed.
Duplicate coupon redemption: Two sessions sharing an account redeemed the same coupon twice.
Blocking bypass: A buyer who'd been blocked could still message the seller through item questions.

Each of the three was independently reproduced, and in each reduced reproduction the bug needed interacting actors to appear.

To be clear, this is a controlled evaluation of a prototype. I'm not claiming it finds every bug.

Demo

GitHub Repository: https://github.com/yashvasudeva1/SWARM

The SWARM dashboard showing benchmark results.

Bazaar, the marketplace I built to test multi-user interactions.

The repo has the implementation, setup instructions, evaluation reports and the benchmark configuration, so you can run it yourself.

Partner Technologies

The main AI infrastructure behind SWARM is Groq, which I used through its OpenAI-compatible API to run openai/gpt-oss-120b.

Groq handles the thinking for every agent. Each one gets its role, its objective, its available tools and the relevant application context, and then decides what to do next.

A few implementation details worth sharing:

Tool-based interaction: Agents talk to the app through validated tools generated from its OpenAPI spec.
Controlled concurrency: Python's async execution plus coordination barriers let several agents act on the shared state at the same time, in a way I can control and measure.
Context management: Agents keep the history that matters without their prompts growing forever.
Budget management: Shared token limits, request pacing and cost tracking keep API usage in check.
Independent verification: Suspicious sequences get replayed on a separate app instance, with no extra LLM decision involved.

The hardest and most fun engineering problem was getting independent AI decisions to land on shared state in a way that was controlled and repeatable. Agents that all do their own thing don't give you a result. Agents that collide on purpose, and do it measurably, do.

The rest of the stack: Python, FastAPI, SQLite, SQLAlchemy, React, TypeScript and Tailwind CSS.

Hackathon Experience

HackCBS 9.0 gave me a chance to take a problem I found genuinely interesting and turn it into a working system.

My biggest takeaway: getting multiple AI agents to make decisions is the easy part. The real work is coordinating what they do, managing shared state, handling model failures, and reproducing a bug reliably enough that you can trust it. That's a whole different level of engineering.

I also learned that a convincing AI demo isn't the same as a convincing engineering result. It's why I spent my time on the verification pipeline instead of just showing off AI-generated claims and calling them discoveries.

If I had to sum it up in one line: building something that works matters, but being able to show why it works matters just as much.

Huge thanks to the organisers and to everyone else who hacked alongside me. It made the whole weekend.

SWARM is still a prototype, and there's a lot left to explore. I'm looking forward to improving it and pushing the idea further.

Project: https://github.com/yashvasudeva1/SWARM

Thanks for reading!