When a clinic administrator asks "is ChatGPT HIPAA compliant?", the honest answer is no by default. Not because the technology can't handle it. Because compliance depends on configuration, contracts, and what data you feed it.
I've spent the last three years building AI systems for a Medicare Advantage primary care organization. Here's what I've learned about which tools you can use with PHI, which ones require a BAA, and which ones will get you fined.
HIPAA compliance is not a property of software
HIPAA doesn't certify products. It regulates how covered entities handle protected health information. A tool is "HIPAA compliant" when:
- You have a signed Business Associate Agreement (BAA) with the vendor
- The vendor agrees to safeguard PHI per HIPAA Security Rule requirements
- You configure the tool to not retain or train on your data
A product can have every security certification in the industry and still produce a HIPAA violation if your staff paste patient names into it without a BAA in place.
Which AI tools have signed BAAs (as of 2026)
I'll be specific here because vague answers are what get clinics in trouble.
Microsoft Azure OpenAI: Yes. Microsoft offers a BAA through their Online Services Terms. Azure OpenAI does not train on your data by default. This is the most common path I see for healthcare organizations that want GPT-4 class models.
Google Cloud Vertex AI (Gemini): Yes. Google Cloud offers a BAA and will sign one for Vertex AI. Gemini data is not used for training when accessed through Vertex AI.
Anthropic Claude (via AWS Bedrock): Yes. AWS offers a BAA for Bedrock. Claude models accessed through Bedrock are covered. Claude accessed through the consumer website is not.
OpenAI ChatGPT: Not by default. The consumer ChatGPT product does not have a BAA. OpenAI does not offer a BAA for ChatGPT as of mid-2026. If you need GPT-4 for healthcare work, use Azure OpenAI instead.
Otter.ai: No BAA available. People search for this constantly (110 searches/month for "is Otter AI HIPAA compliant"). The answer is no. Do not use Otter for meetings where PHI is discussed.
Fireflies.ai: No BAA available. Same situation.
Plaud.ai: No BAA available.
Fathom: No BAA available as of mid-2026.
Granola: No BAA available.
The pattern: note-taking and transcription AI tools almost universally lack BAAs. This is a gap in the market, and it's why clinics keep asking.
How to deploy AI with PHI safely
The architecture I use in production:
Option 1: Azure OpenAI with a BAA. Route API calls through a backend proxy that strips identifiers before sending prompts. Configure Azure to disable data retention and training. Monitor usage logs for PHI leakage.
Option 2: Private LLM deployment. Run an open-weights model (Llama, Mistral) on infrastructure you control. No data leaves your network. No BAA needed because no third party processes the data. The tradeoff is higher operational cost and you need someone who can manage the infrastructure.
Option 3: LiteLLM routing to HIPAA-covered endpoints. Use LiteLLM as an API gateway that routes requests to Azure OpenAI or AWS Bedrock (both BAA-covered). LiteLLM handles the routing, rate limiting, and fallback. Your application talks to one endpoint. The data stays within BAA-covered infrastructure.
What actually gets clinics fined
It's almost never the AI tool. It's the workflow.
A clinician pastes a patient's medication list into ChatGPT to check for interactions. That's a PHI disclosure to a vendor without a BAA. The tool didn't fail. The process did.
I've seen this happen at organizations with otherwise strong compliance programs. The fix is not better AI. It's blocking access to non-BAA AI tools on work devices, providing a sanctioned alternative (Azure OpenAI proxy), and training staff on what they can and cannot type into an AI prompt.
Questions I get asked most
"Can I use Claude for clinical notes?"
Through the consumer website, no. Through AWS Bedrock with a BAA, yes. The model is the same. The compliance posture is entirely different.
"Is Gemini HIPAA compliant?"
Through Google Cloud Vertex AI with a BAA, yes. Through gemini.google.com, no. 20 people per month search this exact question. The distinction matters.
"Do we need an AI governance committee?"
Not a formal one on day one. But you need a named owner, a written policy on approved tools, and a vendor evaluation checklist that covers BAA terms, data retention, subprocessor disclosures, and breach notification windows.
Bottom line
The question isn't whether AI is HIPAA compliant. It's whether your deployment of it is. Microsoft, Google, and AWS all offer BAA-covered AI services that can handle PHI. The consumer tools you've heard of (ChatGPT, Otter, Fireflies) do not.
If you're a healthcare organization figuring out which AI tools you can actually use, that's the work I do. PumaTech builds and deploys HIPAA-compliant AI systems, data platforms, and Salesforce Health Cloud integrations for healthcare organizations.
Top comments (0)