Intro
Every engineering org has a policy about which AI tools developers are allowed to use. Every engineering org also has a Shadow AI problem, whether leadership has noticed it yet or not.
The gap between the two is predictable. Security approves one internal tool, or none at all. Developers still have deadlines, still get stuck, still need to paste a stack trace somewhere and ask "why is this failing." If the sanctioned option doesn't exist, is slow, or is worse than what's publicly available, developers go find their own. Quietly.
What actually gets pasted into public chat windows
This is where it gets concrete. A developer debugging a failing integration test copies the full stack trace into a public AI chat window to ask what's wrong. The stack trace includes a database connection string. Nobody flagged it because nobody was watching, the tool has no audit log and no data retention policy the company controls.
A different developer is stuck on a tricky business rule and pastes the actual proprietary algorithm, the one the company patented, into a public model to get a "cleaner implementation." That algorithm may now sit inside a public training corpus, permanently, with no way to un-paste it.
A third developer, working through a customer support bug, pastes real customer records, including names, emails, and account details, into a chatbot to understand a data corruption issue. That's a compliance violation the moment it happens, whether or not anyone downstream ever notices.
None of these developers think of themselves as a security risk. They think of themselves as someone trying to ship on time with the tools that actually work.
Why banning the tools doesn't work
The instinct from leadership is usually to block the domains at the network level and call it solved. This doesn't remove the underlying need, it just removes the visibility. Developers switch to personal devices, personal accounts, or a browser extension nobody in IT has heard of. The paste-your-code-into-a-public-model behavior doesn't stop. It just becomes invisible to the company that's supposed to be accountable for it.
That's arguably worse than the original problem. An org with visible Shadow AI usage at least knows it has a problem. An org that's driven the behavior underground has the same exposure with none of the ability to see or measure it.
The actual failure is a tooling gap, not a discipline gap
Developers aren't circumventing policy because they're careless. They're circumventing it because the approved path is slower, worse, or nonexistent, and the unapproved path solves their actual problem in thirty seconds. Every time an organization treats this as a training or discipline problem instead of a tooling problem, it gets the same result: policy on paper, Shadow AI in practice.
The fix isn't a stricter acceptable-use policy. It's giving developers an approved tool that's actually good enough that going around it stops being worth the trouble, one that keeps proprietary code, schemas, and customer data inside a boundary the company actually controls, with the visibility to know it's being used instead of guessing.
Anyone else seeing this play out on their team? Is it a policy problem, a tooling problem, or both?
Top comments (3)
I feel that underneath it really is a mindset problem. Over the years I have witnessed countless times that employees or freelancers feel entitled to do anything to finish their task or to "safe the project". A capable, approved tool helps of course, but it relieves the symptoms, not more. The same is true for training and policies. At the end of the day it comes down to hiring, I guess. Find the right people who understand that they do not only act in technical and financial contexts, but also always in a legal context. Easier said than done of course.
Most companies fail because the AI Act creates confusion on what they can do and what not. Also the financial side of the AI use should be considered. I think it's the same debate we had 15 years ago with shadow IT.
Really good parallel, the Shadow IT comparison is exactly right and I think a lot of orgs are about to relearn the same lesson the hard way. The AI Act piece is interesting too, because the regulatory ambiguity actually makes the tooling gap worse, not better. Legal says "we're not sure what's allowed," security says "then nothing is allowed," and developers still have work to do, so they just quietly use whatever works and hope nobody asks. The financial angle you mentioned is the other half of it. Approved enterprise AI often costs 5-10x what a personal ChatGPT subscription does, so orgs end up either underprovisioning seats or restricting access to a small group, which guarantees everyone else finds their own way in. Same pattern as Shadow IT: the answer was never stricter policy, it was giving people sanctioned tools that were actually usable and priced in a way that made compliance the path of least resistance.