Pin a no-training endpoint in your team's .env
echo 'OPENAI_BASE_URL= >> .env
echo 'OPENAI_ORG_ID=org_xxxxx' >> .env
echo 'OPENAI_DATA_RETENTION=none' >> .env
The bug was a beast. Mangled routing code crashing under load, a release due at dawn, a developer alone at 1 a.m. He highlighted the file, pasted it into a public AI chat, and got the answer in about four seconds. The release shipped. Six months later, his company's exact error-handling logic — variable names, internal microservice structure, the works — showed up in a competitor's open-source library. The full story is in [the original incident write-up](https://medium.com/@mohiitlamba/i-pasted-our-company-code-into-a-chatbot-six-months-later-it-handed-it-to-a-competitor-fa04654a4f36).
That's the deal with shadow AI in software development: the productivity win is real, and so is the leak. The model didn't get "hacked." It just remembered.
## The 4 a.m. version of every dev team
Public AI chat boxes are the fastest debugging tool ever invented. Paste a stack trace, get a hypothesis, ship the fix. That part is genuinely good — no senior engineer can pretend otherwise. The model that diagnosed the routing deadlock in the article above did in seconds what would have taken a tired human another hour of squinting.
The problem is what happens after. LLMs are not standard software. Once your proprietary code is in the training pipeline, there is no `DELETE FROM weights WHERE prompt LIKE '%routing%'` you can run. The patterns are baked into the model's parameters as floating-point adjustments — millions of them — and once they're there, retrieval is the model's whole job.
So the developer in the article isn't a cautionary tale about incompetence. He's a cautionary tale about treating a capable tool like a private tool when it isn't.
## What "shadow AI" actually means
Shadow AI is the informal, unmanaged use of AI tools by people inside an organization. No procurement review. No security sign-off. No logging. Just a developer and a chat box at 1 a.m., getting work done.
The same pattern shows up in every team that hasn't yet drawn a line:
- A backend engineer pasting proprietary logic into a chat box for a quick review.
- A finance analyst uploading a CSV of customer data to summarize quarterly churn.
- A lawyer dropping a draft contract into a model to tighten the language.
The tool is the same one — a public, multi-tenant LLM that improves by training on its inputs. The risk is the same too. The developer who pasted his company's routing code thought he was talking to a search engine. He was actually talking to a sponge.
[[CONCEPT: A public chat box is a sponge — anything you drop in gets absorbed into training data, not stored like a row in a database you can `DELETE FROM`]]
## Why models remember what you told them
Neural networks store knowledge as weights — billions of floating-point numbers tuned during training. There's no row-level delete. No TTL on a token. The network "remembers" by adjusting the strength of patterns it now recognizes, and it can spit those patterns back out when a similar prompt comes along.
That's why the competitor in the article got the company's error-handling logic back: not as a verbatim copy, but as a familiar solution to a familiar problem. The model had learned the shape of that team's proprietary architecture well enough to reproduce it for anyone who asked the right question.
The article frames this as the moment the team realized LLMs "refuse to forget." That's accurate. It's also why confidential computing — running models inside hardware-isolated enclaves where the prompts can't leave — has become the loudest bet in the AI infrastructure world right now. The marketing pages talk about encryption at rest and in transit. The harder problem is encryption *in use*, while the model is reading your code. That's the gap confidential computing is trying to close.
## What you actually lose when code leaks
Three things, in roughly this order:
1. **Competitive advantage.** If your error-handling, your routing topology, or your internal microservice boundaries show up in a competitor's library, the moat narrows.
2. **Legal exposure.** Depending on jurisdiction and what was pasted — PII, regulated data, contractual code — the leak can trigger disclosure obligations.
3. **Trust.** The hardest to rebuild. Customers and partners who learn that internal code walked out through a chat box tend to walk out themselves.
The financial shape of a code leak is harder to quantify than a customer-data breach, because the loss is to the strategic edge, not a single legal line item. But the strategic edge is what most software companies are actually selling.
## How to use AI without giving away the store
You don't have to stop. You have to route. Here are the patterns that work today, in roughly increasing order of effort.
### Use the no-training endpoints
Major providers offer data-handling modes where prompts are not retained and not used for training. The CLI flag matters more than the marketing page:
bash
Hitting an enterprise endpoint with no-retention semantics
curl \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "OpenAI-Organization: $ORG_ID" \
-H "OpenAI-DATA-Retention: none" \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Review this snippet..."}],
"metadata": {"source": "internal-review"}
}'
The exact header set varies by vendor. Read the data-handling addendum — don't trust the blog post.
### Run the model yourself
For code that absolutely cannot leave your perimeter, run an open-weight model locally or inside your VPC:
bash
Ollama, one command, runs coder-class models on your laptop
ollama run qwen2.5-coder:14b "Review this Python snippet for race conditions..."
vLLM for a server-grade deployment behind your firewall
docker run --gpus all -p 8000:8000 \
vllm/vllm-openai:latest \
--model Qwen/Qwen2.5-Coder-14B-Instruct
The trade-off is hardware cost and a quality gap on the trickiest prompts. For the 90% of "explain this stack trace" questions, a 14B coder model on a single GPU is more than enough.
### Redact before you paste
Most proprietary bugs live in five lines inside a hundred-line file. Strip identifiers, replace microservice names with placeholders, and paste only the relevant slice:
python
Before — your architecture is right there in the names
async def route_request(req, ctx):
if req.service == "payments-orchestrator":
return await payments_orchestrator.dispatch(req, ctx)
After — the model gets the shape, not the secret
async def route_request(req, ctx):
if req.service == SERVICE_A:
return await service_a_dispatcher.dispatch(req, ctx)
It's tedious. It's also the difference between giving the model a hint and giving it your architecture.
### Keep an audit trail
Even with no-training endpoints, log every prompt that touches proprietary code: who, what, which model, which endpoint. The log is your defense when security asks why a particular snippet went out the door.
## The layer that doesn't depend on the model
Here's where the strategic question sits: what parts of your codebase are you willing to let an LLM see, and what parts are the actual product?
The answer, for most teams, is sharp. The UI layer — the components, the design tokens, the navigation patterns — is rarely the secret. The secret is the business logic underneath: the routing, the pricing rules, the recommendation graph, the proprietary algorithms.
That's the part worth structuring so the AI never needs to look at it. When your shared component layer is consistent across web, iOS, and Android — one API, one set of primitives, one place where design decisions live — the AI's job stays small. It polishes the components. It doesn't see the model. It doesn't see the moat.
[[CONCEPT: Structure the boring parts so well that AI never needs to see the secret parts]]
That's the durable layer underneath the model churn. The model in use will change every quarter. The architecture that decides what the model is allowed to touch is the part you actually own.
## What to do this week
If your team is shipping code against a Friday deadline right now, here is the minimum viable safe-AI setup:
1. **Pick one no-training endpoint.** Get an org-scoped API key. Pin it in your team's `.env`. The snippet at the top of this post is the shortest path.
2. **Run one open-weight coder model locally.** `ollama run qwen2.5-coder:14b` works on a modern laptop. Pair it with an editor extension that points at `
3. **Write a one-page rule:** what can be pasted, what must be redacted, who owns the logs. Keep it under 500 words. If it's longer, nobody reads it.
4. **Audit the last 30 days of shared chat prompts.** Yes, the chat boxes export. Yes, your security team can request the export. Better to know what's in there before a competitor writes a blog post that tells you.
None of this requires new tooling or a security review board. It requires the same instinct the developer in the article wishes he'd had at 1 a.m.: the model is fast, the model is useful, and the model is not yours. Treat it like a capable contractor in a shared office — helpful, sometimes brilliant, and absolutely not where you leave your notebook open.
Top comments (0)