118 Green Tests. One Client Still Hanging.
mosquitto_pub -q 1 doesn't print much when it works. Connect, publish, get an ack, exit. Half a second, if that.
Ours took four seconds. Then it timed out.
Nothing had crashed. Nothing had printed an error. Our broker was up, accepting connections, parsing every byte correctly. 62 tests passing at that exact commit, every one of them green. A real MQTT client just sat there, waiting for an acknowledgment that was never going to come, because we had built half of a feature and tested only the half we'd built.
That gap between "the tests pass" and "the thing works" turned out to be the actual subject of this project, more than the protocol was.
What we were avoiding
Zero Dependency's rule is simple to state and expensive to keep: an empty manifest. No crates. We picked Track C, that is, Web & Network, and built BlitzBroker, an MQTT 3.1.1-subset pub/sub broker in Rust, std only.
MQTT, if you haven't touched it: clients connect to a broker, subscribe to named topics, publish messages, and the broker fans each message out to every current subscriber of that topic. It's the protocol under most "smart home" and IoT stacks: lightweight, binary, built for flaky connections. Normally you get it by running Mosquitto or EMQX, or by pulling in a client library and pointing it at someone else's broker. We wrote the broker.
The honest reason to pick Rust for this specific event, rather than for its own sake: Rust's std gives you almost nothing for the "Web & Network" track. No async runtime. No HTTP. No JSON. No MQTT, obviously. Real Rust network code leans on tokio and serde almost by reflex, avoiding them here isn't a small flex, it's undoing the thing most Rust web code assumes on day one.
The shape of the thing
One broker thread, later four, each owning a disjoint slice of the topic registry, talking to connection threads only through channels. No shared-state locking on the registry, a topic either belongs to a thread or it doesn't, so there's nothing to race. Register and Disconnect broadcast to every shard, since a single client might end up with subscriptions scattered across more than one; everything else routes to exactly one shard, chosen by hashing the topic.
That routing decision is also where the more interesting bug lived, but first: the queue.
Every subscriber gets a bounded outbound queue. When it fills, a slow client, a burst of traffic, something has to give. std::sync::mpsc::sync_channel(N) looked like the obvious tool here, until we actually read what it does under pressure: it blocks the sender once full. That's exactly backwards for a broker. A stalled subscriber blocking the thread that's trying to serve every other subscriber isn't backpressure, it's a single client taking the whole broker hostage. We wanted drop-oldest — discard the stalest buffered message, keep moving and std doesn't hand you that. So it's a Mutex<VecDeque<T>> and a Condvar, maybe forty lines, and it's the one piece of this project I'd point to as "yes, this is genuinely what the stdlib made painful": not that the primitives were missing, but that the one primitive that was there had the wrong policy baked in, and there was no way to ask it for a different one.
The bug that actually ate an afternoon
Wildcard subscriptions in MQTT let a client ask for sensors/+/temp and receive publishes to sensors/kitchen/temp, sensors/garage/temp, anything matching that shape. We wrote the matcher iterative, deliberately not recursive, since both the topic and the filter are attacker-controlled strings arriving off the wire and a recursive matcher is a stack-exhaustion attack waiting to happen, tested it thoroughly against the spec's own worked examples, wired it into the broker's publish path. Subscribed with a wildcard, published to a matching topic, watched it arrive. Shipped it.
Then we added sharding, and it broke, the same way the QoS1 gap did. No error, no panic, just a wildcard subscriber that stopped receiving anything.
The routing rule hashes the topic string to pick a shard. That's correct for exact matches: a subscription to "sensors/temp" and a publish to "sensors/temp" are the same string, same hash, same shard, every time. It's wrong for wildcards, and it took an embarrassingly long time to see why: "sensors/+/temp" and "sensors/kitchen/temp" are different strings. They hash to different shards, almost certainly. The subscription succeeds. The shard holding it just never sees the publish that should have matched it, because that publish landed on a different shard entirely, one that has no idea the wildcard filter exists.
The fix, once we saw it clearly, was one sentence: treat a wildcard subscription like Register or Disconnect, broadcast it to every shard, not just one. Exact-match subscriptions keep routing by hash; wildcard ones go everywhere, so whichever shard eventually gets the matching publish already has the filter to check it against. We verified it the only way that felt honest after the first miss: real mosquitto_pub/mosquitto_sub, eight different retained topics deliberately spread across shards, one wildcard subscriber, all eight arriving. Not because a unit test said so, because we watched it happen.
The half-built feature
Which brings back the four-second hang. MQTT's QoS 1 means "the broker must acknowledge receipt of this publish." We built the wire format for that acknowledgment: encode it, decode it, round-trip it and wrote tests proving the bytes were exactly right. What we hadn't built was the broker actually sending one when a client published at QoS 1. Every test we'd written was protocol-layer: bytes in, struct out, struct in, bytes out. None of them ever ran the connection-handling code that's supposed to notice "this was QoS 1, better say something back" because that logic didn't exist yet, and a test can't fail to cover code that isn't there.
mosquitto_pub -q 1 was the first thing that ever actually waited for that acknowledgment instead of just checking whether the bytes we produced were well-formed. It hung, because we'd never asked anything to wait before.
Same lesson, twice, in one project: encode/decode tests prove your wire format is correct in isolation. They prove nothing about whether the broker behaves correctly end to end. The only way we found either gap was pointing a real, independent MQTT client at the compiled binary and watching what it actually did not what our own code claimed or the AI agent claimed it would do.
The bonus we didn't take
Zero Dependency offers a "Single File" bonus, that is, the whole submission as one source file someone could read top to bottom. We tried. A branch exists, ~3,885 lines, compiles, passes the full test suite.
It's not a single-file program. It's a single file. Every mod boundary from the multi-file layout got preserved wholesale, the module walls are all still there, just copy-pasted into one document instead of split across several. That's a stapled concatenation, not the thing the bonus is actually asking for, and we knew the difference the moment we looked at it honestly.
We didn't claim it. It's disclosed in the README as attempted and not achieved, the branch stayed public, and the bonus tally doesn't include it. It would have been trivial to point at "one file, compiles, tests pass" and call it done and nobody skimming a submission list would have caught the difference on the first pass. But this whole event is built around the idea that a naive, honestly-disclosed gap is worth more than a confident claim that doesn't survive a second look, and once you've internalized that watching your own QoS1 hang, it's hard to selectively unlearn it for the bonus you'd rather have.
What actually got killed
No tokio, thread-per-connection plus a channel-based actor model instead of an async runtime. No serde, the entire MQTT codec, every packet type, hand-rolled against the spec. No crossbeam, the drop-oldest bounded queue above, built because the stdlib's one bounded-channel primitive had a backpressure policy we didn't want, not because a policy was missing entirely. No mqtt/aedes, the packages you'd actually reach for to do exactly this locally, without thinking about it, in about four lines and one npm install.
The decision I'd take back
Small, and it cost real time rather than teaching us anything: our decision log kept colliding. More than once, someone edited it from a copy that predated the last person's changes, and a decision we'd already written down quietly vanished, replaced by someone else's entry claiming the same slot. Not a code bug but rather a coordination one, which is anyways understandable for a team of 4. The fix was never clever, just "pull the actual current version of a shared file before you touch it," and we should have said that out loud on day one instead of relearning it three separate times.
The numbers, honestly
118 automated tests, all passing, none of them the reason we found either real gap, live testing against actual clients was. A four-shard broker pushing roughly 495k msg/s against a single-shard broker's 402k at high concurrency across 64 topics. A real improvement, and a modest one, because at this scale the network syscalls dominate over how many threads are splitting the registry, and we'd rather say that plainly than let a comparison table imply more than it earned. Thirteen documented stdlib substitutions. Zero entries in cargo tree.
The bug that mattered most wasn't in the MQTT spec, and it wasn't really in our code either. It was in trusting a test suite to tell us something it was never actually testing.
Finally
Win or not, this was again one of the most fun 72 hours I've spent building something on a Hackathon Raptor's Hackthon. I cannot thank enough to my teammates at Team CodeBlitz for the experience and knowledge we shared together. We're eagerly waiting for results and feedback from the esteemed judges.
Repo: https://github.com/Dev-Am12/BlitzBroker
Demo: https://drive.google.com/drive/folders/1ScvHPNuxs_JTPaasKMEvzKrL9Wo5OLah?usp=sharing
Built for Zero Dependency 2026, run by Hackathon Raptors.
Top comments (2)
The "62 green tests, real client hangs" gap is the honest center of this write-up. Tests encode the protocol as you understood it; the real client encodes it as it is. The four-second hang with zero errors is what that mismatch looks like from outside — the broker isn't failing, it just never sends the one packet the client is blocking on.
QoS1 makes it sharper: mosquitto_pub won't exit until the PUBACK for that specific packet id comes back. If the ack path depends on the owning shard's queue state, any path where the shard drops the message before the ack — bounded queue full, a client racing a disconnect — leaves the publisher waiting forever while every internal test stays green.
Two things I'd probe: does the hanging client get its PUBACK when you publish to a topic with no subscribers (fan-out trivially empty)? And how are you stressing the queue-overflow path? That's the state the 118 tests almost certainly don't cover, because it needs a slow consumer, not a fast one.
On the PUBACK question: Yes, the acknowledgment is generated and queued before the publish is even forwarded to the broker for fan-out:
if publish.qos == 1 {if let Some(packet_id) = publish.packet_id {
outbound.push(OutboundEvent::Packet(MqttPacket::PubAck(PubAckPacket { packet_id })));
}
}
broker_tx.broker_send(BrokerMessage::Publish { from: id, packet: publish });
There's no dependency between the ack and fan-out results, it can't be conditioned on subscriber count because it's emitted a line before the broker even sees the topic. PUBACK acknowledges broker receipt, not downstream delivery, and that's true by construction here, independent of whether zero or a hundred clients are subscribed.
On the queue-overflow question: you're right. Our existing stress test publishes 148 messages against a 128-capacity queue with nothing draining it concurrently, then checks the count after the fact that validates the queue's own bounded-capacity bookkeeping, but it's the same thing our unit test already proves by calling
push()directly. Neither exercises a real writer thread against a client that's actually behind in real time.So we built that reproduction just now: a raw client that subscribes and then never reads. 300 messages in, still no drops. We pushed to 2000 messages with the client's receive buffer deliberately shrunk and still zero drops. The reason turns out to be more interesting, the OS's own TCP send buffer (200KB+ default on Linux, autotuning into the megabytes) absorbs everything long before our 128-item application queue ever sees backpressure. The writer thread never blocks, so it never stops draining the queue, so drop-oldest never gets a chance to engage. "Don't read from the socket" and "actually stall the writer" turned out not to be the same thing.
So: no, 118 tests don't cover this path. Reproducing it for real needs either a deliberately broken connection or payload volume well past what a normal client would send. That's flagged now as a genuine follow-up, thanks for pointing this out.