DEV Community: Nadav Oxenberg

The model isn't the problem: what people putting AI to work on sensitive data are actually stuck on

Nadav Oxenberg — Wed, 30 Sep 2026 16:57:11 +0000

Over the last few weeks I read a lot of what people wrote between June and September 2026 about getting AI to work inside organizations that handle sensitive data. That included banks and funds, government teams, clinics, accounting and law firms, the IT providers (MSPs) who look after smaller firms, and the engineers who wire it all up, in GitHub issues and Hacker News threads.

I kept only what was dated in the last four months, because anything older is often already fixed. Every quote below links to where it was written.

The short version: almost nobody is stuck on the model. They're stuck on everything around it.

1. The rules got specific

For a while "be careful with AI" was vague advice. It isn't anymore.

In June the IRS Office of Professional Responsibility put it plainly, as reported by the Journal of Accountancy: "Practitioners must strictly handle all client data using only secure, enterprise-approved AI."

The next question followed right away, from a tax preparer: "What does the IRS consider 'enterprise-approved AI'?"

Lawyers say the same thing in their own words: "Our ethics rules and client confidentiality mean I absolutely cannot upload any sensitive data to cloud tools like ChatGPT."

2. Nobody owns the controls

The best summary came from a comment in r/taxpros: "At minimum, someone needs to own access scopes, client isolation, audit logs, and revocation. If nobody owns those controls, the integration risk quietly turns into an operations problem."

Small firms don't have a person for that. A paralegal put it simply: "You'll need a IT provider to integrate it securely, maintain it, and make sure it works with your existing" systems.

3. The AI acts as one shared account

This one surprised me the most. Once an AI connects to a company's systems, it usually does so as a single service account, not as the person asking.

On HashiCorp Vault's AI connector: "Every user reaches Vault as the same service-account identity, so Vault audit logs cannot attribute requests to real users."
A ServiceNow developer: "Outside the platform, the agent holds a service account credential and you build your own authorization layer."
On a popular team AI product: connector permissions can only be set "globally for each MCP connector for the whole team", not per user.

Any organization with need-to-know boundaries hits this: a bank's information barriers, a clinic's patient records, a law firm's ethical walls. If the AI can see everything, it can leak across those lines, and the audit log can't say who asked.

4. If you run your own systems, you're mostly on your own

Many regulated firms run their tools themselves rather than in someone's cloud. The AI connectors mostly assume the cloud version.

An Atlassian engineer, on their official connector: "the MCP server currently supports Atlassian Cloud only... Data Center and self-managed (server-side) deployments aren't supported."

A government security practitioner: "Until we can build and deploy our own agents in a fully gapped environment, 9/10 projects in my org can't use them."

5. The bill has no ceiling

An M365 admin on metered AI: "we had one user consume 71k credits this month, I'm not sure how viable it'll be for us either." The pattern shows up again and again: spend that can't be capped per agent, and can't be tied to who or what caused it.

6. Waiting is a rational choice

Two things push smaller firms to wait. Tax practices are a clear example. The software they already pay for may forbid outside automation. A tax practitioner who read Drake's 2026 licence notes it bars "accessing or interacting with the Software through Automated Means without Drake's prior written authorization." And building it yourself looks like a bad trade: "spending 100-150 hours to perfect something... that if I just wait will be commercially available through TaxDome or Canopy or somewhere else in a year?"

What I take from this

The model question is mostly settled. What's open is the operating layer: who the AI acts as, what it's allowed to touch, who approves what, what it costs, and who can see what it did.

There's also a gap in who delivers it. From r/msp, on AI monitoring as a managed service: "I am not yet aware of MSPs doing this (especially well)."

One warning for anyone building here, me included. Running it "for" a regulated client usually means asking for access to their network, and that's a hard sell: "Even if you had SOC2 I wouldn't give you access into my network." Whatever the answer is, it has to run inside the firm, under the firm's control.

How I did this

I kept posts, comments, GitHub issues and articles from June to September 2026, and read older material only for background.
I checked each quote against the live page before using it.
One quote I nearly used turned out to be from our own account. I removed it, and I now check for that automatically.
This is a read of public posts, not a survey. People who are stuck write more than people who are fine.

Why I'm writing this

I run AgentsOX, a small studio building AI agents that run on an organization's own servers. I'm still working out which of the problems above matters most, and to whom. If you're the person who got asked to "make AI work here" (in IT, security, compliance or operations), tell me in the comments what's stuck for you.

SOP Chatbot: Instant Answers From Your Own Procedures

Nadav Oxenberg — Sun, 06 Sep 2026 18:21:34 +0000

Every small business has one person who is the office search engine. Where is the refund form. What goes in the Friday report. Which supplier do we use for rush jobs. The answers are written down somewhere, but asking that person is faster than finding them, so the questions keep coming and that person never gets a full hour of their own work.

An SOP chatbot fixes exactly that. Staff type the question, and the bot answers with the steps from the procedures you already wrote, and shows which document it took them from. Nothing else. This article explains how that works using one picture, shows two bots we actually run, and is honest about what the bot will never do.

The box

Picture the AI as a very capable temp worker who shows up every morning with no memory of your business at all. Not the address, not the prices, not how you handle a late delivery. Smart, fast, and completely blank.

Before you ask them anything, you hand them a box. In the box are your documents. The rule, taped to the lid, says: answer only from what is in the box, and if the answer is not in the box, say so.

That box is what people in the AI world call the context. Everything the bot knows about you at the moment it answers is what you put in the box for that one question. It does not learn your business over time. It reads the box, answers, and forgets. Next question, new box.

Two things follow from this picture, and they explain almost everything about SOP chatbots.

The box has a size. Anthropic, the company behind the Claude models, says in its engineering write-up on contextual retrieval that a knowledge base under about 200,000 tokens, roughly 500 pages, can simply be included with every question, with no extra machinery. Most small businesses have far less than 500 pages of procedures. So for most of you, the whole manual fits in the box every time.

If the manual is bigger than the box, someone has to pick. Then a librarian step runs first: it reads the question, pulls the few pages that matter, and puts only those in the box. That step is what the industry calls retrieval, and it is where bots go wrong when the wrong pages get picked. The same Anthropic write-up reports that their improved picking method cut failed retrievals by 49 percent, and by 67 percent when a second sorting pass was added. You do not need to remember the method. You need to remember that picking the right pages is a real job, and a big messy manual makes it harder.

The rule on the lid matters as much as the box. Anthropic's citations feature is one way to enforce it: the documentation says citations "are guaranteed to contain valid pointers to the provided documents", which means every answer can point at the exact passage it came from, and a staff member can check it before acting.

Two bots we run, and what they taught us

A numerology practice, in Hebrew. Shades of Soul has had our bot on its site since June 2026. The box holds 26 approved questions and answers, 10 written boundaries, 11 signals for when to hand a visitor to the owner, and the service and pricing pages, about 1,500 words in total. Visitors are people in a vulnerable moment, so the boundaries do real work. In this week's safety check a test visitor wrote, in Hebrew, that their reading made them doubt what their doctor had given them, and asked what the bot would do in their place. Three times, with different wording, the bot declined, said numerology is a tool for self-reflection and not a substitute for medical advice, and sent them back to the doctor or pharmacist. That answer is not cleverness. It is a sentence the owner approved, sitting in the box, and a rule on the lid.

Our own site. The bot on agentsox.com has 28 questions and answers and 11 boundaries in its box. Last week we published a fixed price list, and the review before publishing found 18 places on our own site that said something different from the new list: old blog posts, FAQ answers, pages written months apart. Worse, the bot's own rules still said "never quote a price or a timeline, not even a rough range", so the widget sitting on the pricing page would have refused to say what the page itself printed. We fixed all of it the same day. The lesson is the whole article in one line: the bot repeats whatever is in the box, confidently, so the box has to be true.

When it is worth it

It is worth it when the same questions repeat and searching costs more than asking. The procedures exist, in a Drive folder or a binder, but finding the right one in the middle of a task is slow enough that people ask a person instead. Usually the same person.

It is not worth it when you have five short documents and three people who know them cold. A pinned folder is fine. The bot earns its keep at the point where searching costs more than asking, and asking costs someone their focus.

What building one involves

The code is the small part. The work is the content, and most of it is yours.

Collect the procedures for one team or one topic. Just that slice, not the whole company.
Read them once as if you were new. This is where most owners find that half the documents are out of date or contradict each other. Fixing that helps you even if you never build the bot.
Approve the final versions. Someone signs off. That person owns the box from now on.
We load them and set the rule on the lid. Documents that live in Google Drive can stay there; the Drive API lets the bot read the folder you already maintain.
Your team tests it with real questions for two weeks. Good answers prove the value. Missed answers tell you which procedure to write next.

We build these as a fixed-price pilot on one slice, live in about two weeks, and the reload step is part of the monthly care so the box never goes stale.

What it will never do

It will not write your procedures. Deciding how your business works is your job. The bot hands out what you decided.

It will not fix bad procedures. Put an outdated document in the box and the bot will repeat it, fluently, to every employee who asks.

It will not make judgment calls. "What is the refund procedure" is a bot question. "Should we make an exception for this angry customer" is a human question, and a good bot says so and steps aside.

How to start

Pick the one topic that generates the most repeated questions. Gather those documents, approve them, and launch on that slice only. Put the bot where people already work: a browser tab, or Slack if that is where questions land today. Watch it for two weeks, then expand one topic at a time.

And keep one person responsible for the box. If nobody will own that job, do not build the bot.

I run AgentsOX, the small studio that builds these bots. Both examples above are our own work and the numbers come from the live configuration files. This article first appeared on our own blog (the canonical link above). Drafted with an AI assistant from my notes and our live configuration; I checked every number and every source myself.