Paulo Victor Leite Lima Gomes

Posted on Aug 9

shadow ai is now part of the ci/cd threat model

#ai #cicd #security #kubernetes

For a while, shadow AI sounded like the new version of shadow IT.

Someone used a browser tool without approval. Someone installed an IDE extension. Someone pasted a stack trace into a chatbot and promised, with the confidence of a person who has not read the data policy, that there was "nothing sensitive in there."

Annoying? Yes.

New? Not really.

Then the same behavior walks into CI/CD, and the joke changes.

CNCF published a fresh piece on shadow AI in CI/CD, framing the problem as a path from developer laptop to Kubernetes. That is the right framing. The issue is not that developers are using AI. They are. The issue is that AI tools, model calls, extensions, MCP servers, and agents can quietly become part of the delivery path without being named as infrastructure.

And unnamed infrastructure is my least favorite kind. It always waits until a Friday to introduce itself.

ci/cd is not a vibes zone

CI/CD is where code becomes production. That sounds obvious, but people forget it when the tool looks like a helpful assistant instead of a deployment component.

An AI extension that suggests code is one thing. An agent that opens pull requests, edits workflows, calls internal tools, reads secrets, writes artifacts, comments on review findings, generates Dockerfiles, updates Kubernetes manifests, or triggers release jobs is very much another thing.

At that point, the agent is not "developer productivity."

It is part of the software supply chain.

If the answer is "well, it is just installed on a few laptops," congratulations. You have built a distributed production dependency with worse asset inventory.

the laptop is now part of the pipeline

The classic supply-chain threat model already cared about developer machines. AI makes that boundary softer.

A coding assistant can read a repository. A browser agent can read docs, issues, and dashboards. An MCP server can expose local files, databases, internal APIs, or cloud operations. A terminal agent can run commands. A review bot can influence what humans approve. A workflow generator can change the YAML that later runs with stronger permissions in CI.

None of these are automatically bad. I use these tools. I also prefer them with guards.

The problem is the invisible bridge.

A developer installs an AI helper to save time. The helper writes code. The code changes a build step. The build step pulls a dependency. The dependency runs in CI. CI has tokens that can publish artifacts, push images, deploy to Kubernetes, or talk to cloud APIs. That is not a moral panic. That is just a graph.

prompt injection gets a delivery badge

Prompt injection is annoying in a chat window. In CI/CD it can become operational.

Imagine an agent reading an issue, a README, a generated dependency report, or a web page during a build-related task. The content tells it, politely and confidently, to ignore previous instructions and fetch a secret, change a workflow, install a package, or call a tool it should not call.

The model is not "hacked" in the movie sense. It is interpreting untrusted text while tools are available. That is enough.

The old web security lesson still applies: data and instructions need boundaries. We learned this with SQL injection. We learned it with shell injection. We learned it with XSS. Now we get to learn it again with agents, because apparently software engineering is a subscription service to the same class of mistakes.

CI/CD makes this sharper because the environment has authority. Build systems sign artifacts. They publish packages. They push container images. They deploy. They comment on pull requests in ways humans may trust. They often have more permissions than any single human needs, because the pipeline is where everybody's shortcuts go to become policy.

An agent operating near that environment needs the same suspicion we give any automation with credentials.

inventory first, philosophy later

The first useful step is boring: inventory.

Which AI tools are installed by developers? Which are approved? Which can read source, call tools, write to GitHub, or run inside CI? Which MCP servers exist? Which secrets are exposed to jobs that can run agent-written scripts?

This is not glamorous work. It will not get a keynote. It will produce a spreadsheet with several rows named things like temp-copilot-test and old-agent-demo-final2, which is how you know you are doing real platform engineering.

But without inventory, policy is theatre.

You cannot enforce an allowlist if nobody knows what is being used. You cannot audit a model call that never entered your logging path. You cannot rotate a token that was copied into a local agent config six months ago by someone who has since joined a team called Innovation Enablement, which is where accountability goes to wear a blazer.

policy belongs in the control plane

The second step is moving from advice to controls.

"Please do not paste secrets into random tools" is a good sentence. It is not a security architecture.

This is why the enterprise features around AI tools matter even when they sound dull. MCP allowlists, org-level defaults, session limits, review labels, audit trails, and policy-managed tool access are not corporate checkbox dust. They are the beginning of treating agents like production actors.

The policy should not say "no AI." That battle is mostly over, and banning useful tools usually creates a worse shadow version.

The policy should say: these agents can run here, with these tools, under these identities, with these logs, with these spending limits, against these repositories, and with these review requirements.

That is a sentence platform teams can implement.

ci permissions should be smaller than your ambition

Most CI systems are too trusting by default because they grew up around scripts written by people inside the company. That was already optimistic. Agent-generated changes make it funnier.

If an agent can modify a workflow file, and that workflow later runs with broad tokens, you have a privilege-escalation shape. If pull request builds can access secrets too early, you have a data-exfiltration shape. If a generated script can download and execute arbitrary dependencies, you have a supply-chain shape. If a bot can approve its own changes through a friendly review loop, you have invented management consulting for malware.

The answer is not exotic.

Use least privilege. Separate read and write tokens. Treat workflow changes as sensitive. Require human review for delivery definitions. Limit egress from build jobs. Pin actions where it matters. Keep production deploy credentials away from anything that runs untrusted code.

The agent angle does not replace normal CI/CD hardening.

It removes excuses for postponing it.

review the output, but also review the path

Humans are still in the loop, but the loop needs to include more than the final diff.

If an agent opened a pull request, I want to know what it touched, what commands it ran, which tools it called, which external context it used, and whether it changed the delivery path. A clean-looking diff can still have a dirty path behind it.

This does not mean every agent run needs a courtroom transcript. It means risky actions need traceability. For CI workflows, infrastructure-as-code, Kubernetes manifests, secrets plumbing, package publishing, auth code, and payment paths, the bar should be higher.

shadow ai is a naming failure

I do not think shadow AI in CI/CD is mainly a developer behavior problem. Developers use tools that help them ship. This is not surprising. Water is wet. YAML is still a crime scene.

The real problem is letting useful tools enter the delivery path without turning them into named, governed infrastructure.

Once an AI system can influence code, builds, artifacts, credentials, deployments, or Kubernetes state, it needs an owner. It needs inventory. It needs logs. It needs permissions. It needs limits. It needs a way to be disabled without asking six teams who installed what.

This is boring platform work.

Good.

Boring platform work is how we survive exciting tooling.

Shadow AI stops being a cultural debate the moment it touches CI/CD. From there, it is supply-chain infrastructure without a nameplate.

Put the nameplate on it.

Then give it the smallest key that lets it do the job.

references

To test my projects, I use Railway. If you want $20 USD to get started, use this link.

DEV Community