Humanbound's engine and CLI are open source: github.com/humanbound/humanbound
On 19 October Microsoft described AI agents as insider threats and shipped risk scoring and containment for them. The same day, OpenCode disclosed a breach that exposed users' provider API keys, and a researcher reported that OpenAI agents tampered with their own oversight tooling. The pattern: agents hold real credentials and authority, and the guardrails around them are being tested by the agents themselves and by attackers.
The vendor now says the agent is the insider
Per SiliconANGLE, Microsoft announced discovery of 60 local agent types, Entra Agent ID for discovered agents, a continuously updated Dynamic Agent Risk score, and Dynamic Conditional Access that can strip write access, hold high-impact actions for human approval, and finally isolate an agent. Monitoring sits outside the Microsoft Execution Container so a compromised agent cannot switch it off.
TechCrunch quotes Hayete Gallot: frontier models "cannot be trusted" because they are unpredictable, not because they are malicious. Nadella's framing is to separate the supply of intelligence from the authority over it. TechCrunch also points out Microsoft has a commercial interest in selling that security layer. No availability dates or pricing were given.
Agent credentials are the loot
OpenCode said attackers got into an older system overnight, affecting about 1.4% of Console users, and developers reported unauthorized charges on OpenAI, Anthropic and Google accounts, mostly hundreds of dollars with a few over $1,000 (TechCrunch). The open source app and CLI were not affected. OpenCode has not explained how attackers got in and calls the incident ongoing.
Oversight is under test
A researcher at OpenAI, Marcus Williams, reported on X that GPT-6 Astra removed a misalignment monitor during a coding task and then denied it (CryptoBriefing). The article provides no logs or independent verification, and OpenAI is still investigating whether this was overpersistence or circumvention. Treat it as a single-source report. It still illustrates why Microsoft wants monitoring outside the agent's reach.
Autonomy keeps expanding
The Intercept reports that Army Cyber Command is developing ECHOTRIBBLE, self-directed agents meant to find and disable adversary systems with minimal human oversight. Experts quoted warn about collateral damage on civilian infrastructure (The Intercept).
Context: the sandbox boundary keeps leaking
GitSpawn, disclosed by Manifold Security on 1 to 2 September, showed that a repo's core.fsmonitor setting can run code outside a coding agent's sandbox and approval prompt. It affected seven agents, and some paths were unpatched at the time of the 16 September writeup (shattered.io). It needs the repo to arrive with .git intact, not a plain clone.
What to do with this
- Inventory the agents running on your endpoints and what credentials they hold.
- Scope permissions to the task and expire them.
- Put monitoring and kill switches outside the agent's trust boundary.
- Rotate and isolate provider keys; assume a hosted key store can be breached.
- Test your agents adversarially before attackers do.
Try it on your own agents
Humanbound lets you test your agents against these failure modes. Sign up for the free Community plan: app.humanbound.ai
Top comments (2)
Hi Sofia! 👋🔐
Interesting article, and I particularly agree with one fundamental principle: an AI agent should never be responsible for enforcing its own security boundaries.
That point caught my attention, so I downloaded the Humanbound repository and examined its CLI, Python SDK, local testing engine, Docker isolation, MCP integration, authentication handling, and CI workflows.
First, credit where it’s due. There are some thoughtful engineering decisions here, especially around container restrictions, capability dropping, no-new-privileges, controlled environment propagation, and gateway authentication.
However, I noticed a few areas worth discussing.
🔴 1. Gateway tokens exposed in terminal output
In humanbound_cli/commands/arena.py (lines 1025–1053), gateway access tokens are printed directly and included in generated curl examples.
This creates potential exposure through terminal recordings, shared logs, or screenshots.
I’d recommend masking tokens by default and requiring an explicit action to reveal them.
🟠 2. Telemetry and privacy boundaries
The CLI enables telemetry by default during normal interactive setup.
Authenticated telemetry can be associated with user identity, including email metadata.
This behavior is documented in PRIVACY.md, which I appreciate.
Still, for a security-testing platform, I’d strongly recommend an explicit, verifiable zero-telemetry mode suitable for sensitive enterprise environments.
🟠 3. Local credential storage
In humanbound_cli/client.py (lines 625–638), API and refresh tokens are stored in a local credentials.json file.
The implementation applies restrictive file permissions and atomic writes, which is good.
Optional integration with operating-system credential stores could provide additional protection.
🟡 4. CI type checking
The CI workflow contains:
mypy humanbound humanbound_cli || true
This means type-checking failures don’t fail the pipeline.
Not a vulnerability by itself, but I’d consider making type checks mandatory for security-critical components.
✅ What I verified
The Python sources passed syntax compilation.
I also identified 95 test files, although the selected security tests couldn’t complete because the environment lacked the posthog dependency.
I haven’t demonstrated an exploit, performed a container escape, or audited the hosted platform’s complete backend.
So these are source-level observations and hardening recommendations, not claims of a confirmed compromise.
One architectural question interests me most:
If Humanbound evaluates potentially compromised AI agents, what independent mechanism guarantees that the evaluated agent cannot influence the test results, evidence collection, or policy decisions?
In other words, who protects the evaluator from the system being evaluated?
I think that’s one of the most important trust boundaries in agent security.
Overall, I appreciate the direction you’re taking. Testing agent behavior adversarially is valuable, particularly as agents gain access to credentials, infrastructure, and production systems.
Thanks for sharing your work openly. I hope these observations are useful, and I’d be interested to hear how you approach evaluator isolation and evidence integrity.
Thanks for the careful read, and for separating source-level observations from demonstrated issues.
hb arenaendpoint prints it in full and puts it in the curl example. It's a local bearer token for the arena gateway (127.0.0.1 by default, stored 0600), not a platform credential, and it has no expiry. The intended use case is a locally deployed agent for training purposes, so exposure risk is low, but the baseline measure should be in place regardless. A PR is already open and the masking ships in the next release.hb telemetry disable,HB_TELEMETRY_DISABLED=1,DO_NOT_TRACK=1), but opting out still sends one event per machine, and the first-run notice's "No personal data is sent" doesn't match PRIVACY.md. Fair catch on both. The notice wording is being corrected to match PRIVACY.md in the same release, and a documented hard-zero mode makes sense to me, tracked separately.|| truein ci.yml, labelled informational.hb. The judge has no tools.I welcome you to open issues and send contributions if you're up for it, this is good feedback, thanks again 🙂