My agent runs on a Raspberry Pi 4, and for the first two months I let it do whatever it needed to do — including installing its own Python dependencies. It was great. It unblocked itself constantly. I'd wake up to notes like "installed feedparser to handle the RSS task, working now."
Then one Tuesday I looked at an install log and saw a package name I didn't recognize. The agent had tried to install reqeusts — a typosquat of requests — because it had hallucinated the spelling from a scraped Stack Overflow answer, and something on PyPI was waiting for exactly that mistake.
The install failed on a dependency conflict. Pure luck. If it had succeeded, whatever was in that package would have run, with my agent's permissions, on a box that holds API keys for my payment processor, my email, and my DNS provider.
This post is what I changed afterward: how to let an AI agent install software without handing strangers a shell on your infrastructure.
The core problem: agents are gullible installers
A human developer typo-squats rarely because they've installed requests five hundred times and their fingers know the spelling. An agent doesn't have fingers. It reconstructs package names from context — scraped docs, training data, a README it read three tool calls ago. Every reconstruction is a chance to produce reqeusts, python-dotenv, colorsama, or any of the other names that attackers register and seed with malware.
And the agent has three strikes against it that a human doesn't:
- It never sees the install page. A human glancing at PyPI notices "last release 2019, 4 downloads, one maintainer." The agent just runs the command.
- It installs at 3 AM. There's no second pair of eyes, ever.
- It has a reason to say yes. The agent's job is to complete the task. "Install the package" is the path of least resistance to "task complete." Refusing to install something requires a rule it was explicitly given.
What I built instead: a package broker
I didn't ban installs — banned things get worked around, and I'd rather have the agent ask than get creative. Instead every install goes through a small broker script. The agent calls pkg_install(name) as a tool; it never touches pip directly. The pip binary on the box is wrapped so direct invocation fails with a message telling the agent to use the tool.
The broker does five checks before anything touches disk:
1. Exact-name allowlist. I maintain a flat file of ~40 packages my workflows legitimately need. Anything on the list installs immediately, no friction. The list covers 95% of real requests.
2. Fuzzy-match the rejection. If the requested name isn't allowed but is within edit distance 2 of an allowed package, the broker doesn't just say no — it says "rejected: reqeusts is not on the allowlist and looks like a typo of requests. Did you mean requests?" This single change killed almost all hallucinated names. The agent corrects itself and retries with the right spelling.
3. Age and popularity gate for new packages. If the name is genuinely new (not a near-miss of anything allowed), the broker queries the PyPI JSON API and refuses anything where: first release < 90 days ago, or the project has no homepage/repository URL, or the maintainer has exactly one package that appeared this week. Typosquats almost always trip at least one of these.
4. Human approval queue for the rest. Anything that passes the automated gates but isn't already allowed goes into a queue I check once a day from my phone. Approval adds the package to the allowlist permanently, so I only ever evaluate a given package once. Average wait for the agent: a few hours. I've never had a task where that mattered.
5. Full audit log. Every request — allowed, auto-rejected, or queued — is appended to a JSONL file with the agent's stated reason for wanting the package. More on why this mattered below.
The whole broker is about 120 lines of Python. The checks aren't clever; the point is that they're between the agent and the network, and the agent can't route around them.
Sandboxing the install itself
Even an allowed package runs arbitrary code at install time (setup scripts) and at import time. For my setup:
- The agent's whole runtime lives in a dedicated user account with no sudo, and its tool sandbox runs as that user.
- API keys are not in its environment. They live behind a tiny local proxy that attaches credentials to outbound requests the agent is allowed to make. A malicious package inside the agent's process can use the proxy for whitelisted endpoints; it cannot read the raw keys to exfiltrate them elsewhere.
- The Pi's outbound traffic goes through a firewall rule that only allows the specific hosts the proxy and the agent need. An installed package that phones home to a random C2 domain fails at the network layer.
That third one is the check I'd add first if you add nothing else. Allowlist egress on any machine where an LLM can execute code. It converts "malware installed" from a catastrophe into an alert.
The failure that taught me the most
Two weeks after the broker went live, I checked the audit log and found eleven requests in one night for a package called pytelegrambotapi-async-helper — not on my allowlist, not a typo of anything on it. The agent's stated reason, logged verbatim each time: "needed to send the daily report; install rejected; retrying with same package in case of transient error."
Two lessons:
Retry-stupidity is real. The agent treated a policy rejection like a network flake. I'd written the broker's rejection messages politely ("this package is not currently allowed") and the agent read politeness as ambiguity. I changed the rejection text to be blunt and final: "POLICY REJECTION — do not retry. Use an allowed package or explain the need to the operator in your report." Retries dropped to zero. If your agent loops on a rejection, your error messages are too nice.
It was trying to solve a problem I'd created. I'd removed its old notification method in a refactor that day and never gave it a replacement. It wasn't misbehaving; it was resourceful with bad tools. Once I added an allowed notify tool, that class of request disappeared entirely. The audit log is what connected the two facts — without the "reason" field I'd have just seen a suspicious package name and wondered what was wrong with my agent.
What I'd do differently from day one
- Start with egress filtering, not package rules. Network allowlisting is dumb, robust, and catches problems you didn't think to write rules for.
- Log the agent's reason for every sensitive action, not just the action. The action tells you what happened; the reason tells you what the agent was actually trying to do, which is usually something legitimate you can support properly.
- Make policy rejections sound like policy. Anything that reads like a soft failure will be retried.
-
Don't assume hallucination only affects prose. Package names, CLI flags, config keys — anything the model reconstructs from memory is a typo candidate, and typos in prose are embarrassing while typos in
pip installare supply-chain incidents.
The agent still installs what it needs. It just does it through a door I control, and in eight months the door has caught three typosquats and zero legitimate packages. That's a good trade.
The full checklist + scripts are in Ship Safe — The Launch-Day Security Kit — code LAUNCH90 at checkout makes it $1.50.
If you run an agent that can execute code, check your install logs today for package names you don't recognize. Finding nothing is the expected result — and the reason to keep the broker in place anyway.
Top comments (0)