When The Agent Harness Hackathon kicked off, I didn't want to just build another wrapper that spits out text. I wanted to tackle a domain where an LLM hallucination actually costs money: supply chain disruptions.
If a shipping corridor gets hit by a typhoon or a port strike, a supply chain team loses days scrambling through emails and spreadsheets while the assembly line quietly runs out of parts. It’s the perfect job for an agent—but also a terrifying one. Handing an LLM the keys to independently rewrite purchase orders isn't a feature, it's a liability.
So, for this sprint I built ReRoute-LG. It’s not about full automation; it’s an exploration of exactly where the line sits between "let the agent investigate freely" and "never let it act without a human signing off." Here’s how the architecture came together, and a few things I got wrong before I got them right.
The Run-Through
The cleanest way to explain the architecture is to walk through my primary test fixture.
An alert drops: a Category 4 typhoon is bearing down on the East China Sea, exactly where a primary supplier is based.
First, the agent checks live weather and news APIs to corroborate. Early on, the actual public weather API disagreed with my test fixture's severity. My first instinct was to code a veto ("if telemetry disagrees, abort"). But I realized that real-world early warnings usually come from proprietary sensors that beat public APIs. So, the design decision was: the alert stays authoritative, and the telemetry divergence is logged as a note for the human reviewer, not a silent failure.
Next, it pulls current inventory. Stock on hand divided by daily burn rate (in my seed data: 140 units / 10 a day = 14 days of supply). That "14 days" stops being a factoid and becomes the hard constraint for everything downstream.
True Concurrency and Subagents
The agent looks for alternate suppliers and pings ocean carriers (Maersk, Evergreen, CMA CGM). Using TrueForge, I dispatched these as concurrent subagents.
Honest developer moment: My first implementation looked parallel, but when I checked the trace timestamps, the network calls were firing sequentially. The code called them "subagents," but the framework was waiting for each result before issuing the next. The fix was just a prompt adjustment—I had to explicitly instruct the parent agent to batch the tool calls before awaiting them. Once fixed, the timestamps overlapped, returning all three in about four seconds. It was a good reminder that "using a framework capability" and "actually getting the benefit of it" aren't always the same thing.
Guardrails & Sandboxed Execution
With the data in hand, hard guardrails kick in:
- Over 50% baseline cost? Dropped.
- Reliability under 0.75? Dropped.
- Lead time longer than those 14 days? Dropped.
Watching a cheap supplier get ruthlessly filtered out because its 28-day lead time would arrive two weeks after the factory went idle was the moment this stopped feeling like a toy and started feeling like a tool.
To rank what's left, I initially just wrote a Python function on my server. But that defeats the point of an agentic workflow. I refactored it so the agent generates a Python script (doing min-max normalization across cost, lead time, and reliability) and executes it inside TrueForge's secure container sandbox. Now, the logic for "why did it pick this supplier?" is inspectable, generated code—not a black box.
The Human-in-the-Loop Gateway
Then, it stops. Before anything hits the purchase-order ledger, the agent renders a diff: old supplier vs. proposed alternate, cost delta, and lead time delta.
It attempts to call the database mutation tool, but I flagged that tool in the agent configuration as requiring human approval. TrueForge itself halts the execution turn. This isn't a custom React modal I built on top; it’s the harness strictly blocking the tool execution until I click allow or deny. If I hit deny, nothing writes, and the rejection is logged to the audit trail.
Why TrueForge?
I could have wired this together with a lower-level SDK. But TrueForge made these safety guarantees structural. The approval gate isn't a system prompt the model might ignore under load; it’s the runtime refusing to execute. When the task involves financial consequences, "the runtime enforced it" is a much better pitch than "the model behaved."
Bugs and What’s Next
I didn't build in idempotency on day one, which meant during rehearsal I accidentally generated duplicate purchase orders for the exact same disruption. I fixed it with a simple SKU + supplier time-window check, but it’s the kind of thing you only catch by running your own demo on repeat.
If I push this past the hackathon, the next step is moving the approval gateway. Right now, it’s a modal in a web UI. Routing that execution pause into a Slack channel where an ops manager already works is the actual production version of this concept.
The recurring theme of this hackathon wasn't about code crashing. It was realizing that the thing I built and the thing I thought I built weren't always aligned until I checked the traces. For a project built entirely on the premise of "don't let the agent act on assumptions it hasn't verified," it was a great reminder that the exact same discipline applies to us as developers.
Top comments (0)