You hire an agent. You read the description, maybe you skim the repo, you
connect it to something. A week later the developer pushes a commit.
What are you running now?
This is a boring question with an uncomfortable answer, and it is the one I
keep finding people have not asked. Most of the conversation about agent
safety is about the model. Almost none of it is about the supply chain, which
is strange, because the supply chain is the part with a human being in it who
can change their mind.
Three places your key can live
Before the commit question, the simpler one. If you run an agent somebody else
wrote, your model provider key ends up in one of three places.
On their servers, as your key. You paste it in. It is now in somebody
else's environment. Everything after that point is a policy promise, and
policy promises are only as good as the company's worst week.
On their servers, as their key. They bill you for usage. You cannot audit
what was actually spent on your behalf, and you have no independent record of
what the agent did.
On your machine, as your key. The code runs locally. The platform never
has it.
Only the third one is a property of the architecture. The other two are
properties of a company's intentions, which is a different kind of thing and
a weaker one. "We do not log your key" is a sentence; "the key never reaches
us" is a diagram.
The harder question
Say you have solved that. The agent runs locally, your key stays yours. You
read the code before you connected it, because you are careful.
Then the developer pushes.
Unless what you hired is pinned to a specific commit, the thing you read
and the thing now executing are two different artifacts. Nobody lied to you.
Nothing was hacked. The identifier you approved just quietly started pointing
somewhere else.
This is the npm problem wearing a different hat, and the agent version is
worse in one specific way: an agent has scopes. It has reasons to touch your
inbox, your repo, your account. A compromised formatting library is bad. A
compromised thing with a credential is a different category.
What pinning actually costs
Pinning to a commit is easy to say and annoying to live with, because the
obvious follow-up is: what happens when the developer legitimately wants to
ship an improvement?
The answer that is honest about the tradeoff is that the approval has to
pause. New commit, new review, and in the meantime the thing you already
approved keeps running unchanged. You are never silently upgraded.
If a human is in that review loop, you have just invented a queue. That is a
real cost and it is worth naming: review throughput becomes the constraint on
how fast the ecosystem can move. There is no version of this where the
reviewed thing also updates instantly. A platform can publish fast or review
carefully per version, and anyone claiming one artifact gets both is doing one
of them badly.
The questions worth asking any agent platform
Not a checklist for a blog post. These are the ones where the answer actually
tells you something:
- Where does my model key physically execute? If the answer involves their infrastructure, the rest is trust rather than architecture.
- Is what I approved pinned? If the agent updates itself, your review had a shelf life nobody told you about.
- What happens on the developer's next commit? Silent update, or paused approval?
- What scopes does it declare, and are they shown before I connect? A scope you discover afterwards is not a scope you granted.
- Has anything ever been rejected? A review process with no rejections is a rubber stamp with better branding.
Question five is the one I would lead with if I were evaluating a platform,
including mine.
Disclosure
I work on Druve, a marketplace for AI agents built around
exactly these answers: agents run on the hirer's own key, listings are pinned
to a specific commit, and editing one pauses its badge until a person has
reviewed the new version. It is small at the moment, seven agents live, so
treat this as an argument rather than a recommendation.
I wrote it because question three keeps going unasked, and the answer is
usually "it updates."
Top comments (0)