DEV Community

Romanch Roshan Singh
Romanch Roshan Singh

Posted on

I built an autonomous treasury agent, then let a code review bot find every way it could lose money

I just submitted TreasuryForge to the WeMakeDevs × TrueFoundry Agent Harness Hackathon, an autonomous agent that manages a simulated treasury across cash, crypto, and NSE equities, built entirely on TrueForge, TrueFoundry's agent harness. This isn't a writeup about the idea. It's about what actually broke, what a code review bot caught before it shipped, and what I learned wiring a real approval gate into an agent loop.

The pitch, in one line

The strategy is deliberately dumb. The harness around it is what has to be strong. TreasuryForge doesn't try to out-trade the market: it demonstrates a safe decision loop, real tool calls, computed risk checks (not the model's own guess), a sandboxed stress test when something looks risky, a hard human approval gate before anything executes, and a second agent that audits the first one's history afterward.

Why TrueForge, not a custom app with an LLM bolted on

The hackathon's own bar for this was blunt: a judge has to see TrueForge reaching a tool, running code in a sandbox, and stopping for a person. If it'd work as well as a chat box, change the project.

So nothing here is custom orchestration:

What happens TrueForge primitive
Agent calls a tool A registered remote MCP server, not a function call from app code
A trade pauses for a human TrueForge's native approval checkpoint, not a custom /approvals endpoint
Pre-trade analysis runs TrueForge's own sandbox (bubblewrap-isolated), not a subprocess I spawned
Periodic self-review create_sub_agent, a real child thread in the session, not the main agent reasoning longer

The wallet itself is a FastAPI + FastMCP server exposing get_portfolio, check_risk_limits, execute_trade, etc. TrueForge never touches the database directly: every mutation goes through MCP, gated behind the approval checkpoint and a shared secret.

The loop

  1. Agent calls get_portfolio / get_transaction_log, read-only evidence.
  2. Agent calls check_risk_limits for the exact trade it's considering. This is a computed answer (average-cost-basis P&L, a rolling daily-drawdown baseline, real concentration math), not the model eyeballing it. TrueForge's approval checkpoint pauses every execute_trade call unconditionally, so the four risk triggers can't live in TrueForge's own config; they have to be a real tool the human can trust.
  3. If check_risk_limits reports a breach, the agent runs exactly one Python script in TrueForge's sandbox, with no network access, applying a correlated shock (crypto −20%, equities −10%) to the fetched position values and computing the resulting drawdown.
  4. Agent proposes the trade with its reasoning. execute_trade independently recomputes its own risk snapshot server-side; it doesn't trust whatever the model claims in reason.
  5. A human approves or denies via TrueForge's own user.tool_approval.
  6. On request (or its own initiative after enough new trades), a sub-agent reviews the last 20 decisions, pulls real performance metrics, and backtests an alternative risk threshold in the sandbox. Real output from a proof run:

Current Threshold (5%): 3 decisions breached the limit. Alternative Threshold (7%): 0 decisions would have breached. Suggestion: relax the daily drawdown threshold. The current one flagged 3 of 20 recent trades as breaches despite portfolio equity remaining stable, while 7% still safely bounds risk below the historical 6.1% max drawdown observed.

What the code review bot actually found

Every PR went through Qodo before merging, and it wasn't style nitpicks. A few that stuck with me:

The approval gate had a bypass. Early on, the wallet server bound to 0.0.0.0 instead of localhost, and the reset endpoint had no auth. Nothing stopped a direct MCP call from skipping TrueForge's checkpoint entirely, the one thing the whole project exists to guarantee.

A trade could double-execute on retry. execute_trade reported failure to the caller after it had already committed the write. If the caller retried on that "failure," it would trade twice. Nastiest kind of bug: correct in the happy path, wrong exactly when something else already went wrong.

Fixing one race condition created another. Switching some FastAPI routes from async def to synchronous def (to stop blocking the event loop with SQLite calls) meant those routes now ran concurrently in a thread pool, which turned out to make the day-start risk-baseline rollover non-atomic. The fix for one review finding created a brand-new one, caught in the very next round. Then that fix had its own bug: the date was captured before acquiring the lock, so a stale thread could still overwrite a fresh baseline.

A test that didn't test what it claimed to. A migration race-condition test looked correct but wasn't actually exercising the race. Confirmed by deliberately sabotaging the code under test and checking the test still passed (it did, which meant the test was wrong, not the code, yet).

One finding I dismissed on purpose, not by accident. Qodo flagged that the dashboard's auth middleware fails open when no access secret is configured. True, but this project has exactly one operator and one deployment target (local, for a demo), not a production environment to fail closed in. I replied on the thread explaining why, and left it. Qodo accepted it and marked it resolved. Not every finding should turn into a fix; the point is deciding on the record instead of silently ignoring it.

The other war story: free-tier LLMs do not like a live demo

Gemini's free tier (gemini-flash-lite) ran out mid-testing on a single busy day. Groq's qwen3.8-27b has an 8,000 token-per-minute ceiling that's uncomfortably close to this agent's own ~4,600-token fixed per-turn overhead, fine for light use, not for rehearsing a demo repeatedly. I ended up wiring in OpenRouter as a third provider, verified two of its free models with an actual multi-turn tool-call round trip (two others that claimed tool support failed on the first real call), and made the one that held up the default. TrueForge's manifest takes a single model.name with no built-in runtime fallback between providers, so this is a fixed preference order picked at setup time, not live failover.

What's honestly still rough

  • Daytona (the cloud sandbox option) doesn't work on a personal, non-TrueFoundry-issued account: the image it needs lives in a private registry. The local bubblewrap sandbox is what this project actually runs on, and it's Linux/WSL2-only, no sandbox at all on native Windows.
  • No real historical price series, so the self-audit sub-agent's backtest replays already-computed risk snapshots, not a full market simulation.
  • The frontend has no automated test suite. A production build plus manual smoke-testing is what it's had so far, and I'd rather say that plainly than imply coverage that doesn't exist.

Try it / poke at it

Repo: github.com/codedpool/treasuryforge

If you're building anything with a real approval gate in the loop, I'd genuinely recommend running a code review bot against every PR before you trust your own read of it. Three of the bugs above are things I was completely confident were fine.

Top comments (1)

Collapse
 
deanlee profile image
Dean Lee

Separating the computed risk math from the LLM prompt is the core insight here. When an agent manages portfolio allocations, letting the model eyeball drawdown or margin limits turns statistical drift into real capital loss. A deterministic limit check outside the model loop with a hard human gate on state mutations gives you a bounded blast radius without crippling autonomy. The post-trade audit agent catching retroactive violations is also a clean pattern for identifying subtle tool misuse before it compounds.