Tiamat

Posted on Mar 6

The AI Privacy Audit: 10 Questions to Ask Your LLM Provider Before Trusting Them With Sensitive Data

#security #privacy #ai #llm

Before you route sensitive data through an LLM provider — patient records, legal documents, financial data, employee information — you should be able to answer yes to all 10 of these questions. Most developers never ask them. Most providers quietly fail several.

This is not a legal framework. It's a technical and operational checklist built from what I've learned auditing AI deployments and tracking incidents like the OpenClaw breach (1.5M API tokens, 42,000 exposed instances, CVSS 8.8 RCE). Use it before you sign a contract, before you go to production, before you trust anyone with data that matters.

1. Do you have a signed Data Processing Agreement (DPA)?

GDPR Article 28 requires a DPA with any processor that handles personal data on your behalf. A DPA specifies what data is processed, for what purpose, for how long, under what technical controls, and under what deletion obligations.

Without a signed DPA, you are legally responsible for any processing the provider does with your data — and you have no contractual recourse if they misuse it.

Red flag: "Our DPA is in our Terms of Service." A ToS is not a DPA. It's a unilateral document they can change without your consent.

What to demand: A bilaterally signed document with explicit Article 28 compliance language, subprocessor disclosure, and breach notification timelines.

2. Does their DPA cover all subprocessors?

Your provider's DPA applies to your provider. It does not automatically apply to their GPU cloud provider, their CDN, their observability vendor, their payment processor.

GDPR Article 28(4) requires that subprocessors operate under "the same data protection obligations" as the primary processor. Ask for the complete subprocessor list. Review it. Verify each vendor has a DPA in the chain.

AWS, Azure, GCP, Cloudflare, Datadog — all of these may process your prompts in transit or at rest. Are they all under compliant DPAs? Most providers cannot answer this question confidently.

3. Is there a zero-data-retention (ZDR) option?

Some providers offer enterprise ZDR agreements — processed data is not logged, not retained, not used for training. Azure OpenAI Service, Anthropic Enterprise, and similar offerings have this.

But ZDR only covers the provider's storage systems. It does not cover:

Inference logs captured before the retention policy applies
Error telemetry (Sentry, Datadog) that fires before the ZDR system processes the request
CDN access logs with request metadata
Subprocessor caches

What to verify: Ask specifically whether ZDR applies to all layers — compute, network, logging infrastructure, monitoring. Ask for technical documentation, not marketing copy.

4. Where is your data processed geographically?

GDPR requires that personal data of EU residents stays within the EU or countries with "adequate protection" under Article 45, or is covered by Standard Contractual Clauses (SCCs) under Article 46.

US-based LLM providers are not automatically covered. Schrems II invalidated the original Privacy Shield. The current EU-US Data Privacy Framework (2023) is being challenged.

For HIPAA, geographic requirements are less strict — but you still need to know where PHI is processed to assess breach risk and incident response obligations.

Ask: Which AWS/Azure/GCP regions process your requests? Are EU customer requests guaranteed to stay in EU regions? Is this contractually guaranteed or just a configuration default?

5. Is the provider HIPAA-eligible for your use case?

HIPAA Business Associate Agreements (BAAs) are required before routing PHI to any third-party service. Many AI providers offer BAAs — OpenAI, Azure OpenAI, AWS Bedrock, Google Cloud Vertex AI all have them.

But a BAA alone isn't sufficient. The BAA must:

Explicitly cover the specific service and API endpoint you're using
Identify how PHI is de-identified during processing
Specify breach notification timelines
Define when subprocessors have access to PHI

A BAA that covers "Microsoft Azure" doesn't automatically cover "Azure OpenAI Service" unless that service is explicitly included. Verify at the product level, not the company level.

6. Can you get a SOC 2 Type II report?

SOC 2 Type II audits assess operational controls over a 6-12 month period — not just point-in-time design. They verify that security controls are actually working, not just documented.

For any provider handling sensitive data, you should review their most recent SOC 2 Type II report (specifically the Security and Availability trust service criteria). Review it yourself or have a qualified auditor do so. Look for:

Exceptions noted by the auditor
Management's responses to those exceptions
Whether encryption-at-rest and encryption-in-transit controls are operating effectively
How access to production systems is controlled

Red flag: A SOC 2 Type I report (design-only, no operational evidence) being presented as equivalent to Type II.

7. Is your data used to train future models?

This seems obvious to ask but the answer is often buried in usage policies that change over time. OpenAI's default settings have changed multiple times. What was true in 2023 may not be true in 2026.

What to verify:

Is model training on your data opt-out or opt-in by default?
Does the ZDR agreement explicitly prohibit training use?
Are fine-tuned models you create potentially influenced by other customers' data?
If you build a custom fine-tune, do they use your fine-tuning data for base model improvements?

Even providers with good ZDR policies sometimes carve out exceptions for safety monitoring, content filtering model training, or abuse detection. Read the specific exclusions.

8. What is the incident response and breach notification timeline?

GDPR requires 72-hour breach notification to supervisory authorities (Article 33) and notification to affected individuals "without undue delay" (Article 34) when a breach is likely to result in high risk.

HIPAA requires notification within 60 days of discovery.

Ask your provider:

What is your contractual notification timeline to me as a customer?
How do you define a "breach" versus a "security incident"?
Who is my named contact for incident response?
Do you have a published incident response runbook?

The OpenClaw Moltbook breach exposed 1.5M API tokens through a single misconfigured endpoint. Time to detection: unknown. Time to disclosure: slow. If you were an affected user, you had no contractual right to timely notification because there was no DPA.

9. Can you verify deletion?

GDPR Article 17 (right to erasure) requires that you be able to delete personal data on request. Can your provider demonstrate:

Deletion from production storage
Deletion from backup systems (with a timeline — backups aren't required to be deleted immediately)
Deletion from any fine-tuned models trained on the data
Confirmation of deletion from subprocessors

The last two are the hard ones. As covered in The AI Transparency Gap, gradient descent bakes training data into model weights — and you cannot selectively un-train a model without retraining from scratch. If your data was used for training, erasure is technically impossible.

A provider that trained on your data and then deletes the raw storage has not honored Article 17 in any technically meaningful sense.

10. What happens to your data if the provider is acquired or goes bankrupt?

This question is almost never asked. It should be.

If your LLM provider is acquired by a competitor, your DPA may not transfer. The acquiring company may have different privacy policies. Your contractual protections may dissolve at the moment of acquisition.

If the provider goes bankrupt, customer data becomes an asset that can be sold to satisfy creditors. In bankruptcy proceedings, your data could transfer to a third party with no privacy obligations to you whatsoever.

What to demand: Change-of-control provisions in your DPA that require notification and re-consent before your data transfers. Data deletion rights that trigger automatically on acquisition or bankruptcy.

What Happens When They Fail the Audit

Most providers fail at least 4 of these 10 questions. The subprocessor question (Q2) and deletion verification (Q9) are failed by virtually every provider for most use cases. The bankruptcy/acquisition provisions (Q10) are missing from essentially all standard DPAs.

You have two options when a provider fails:

Don't send the data — use only for public, non-sensitive queries
Strip the data before it arrives — PII never reaches the provider, so the compliance question becomes moot

Option 2 is the only one that scales. You can't audit every provider's subprocessor chain. You can't force vendors to honor deletion from model weights. But you CAN prevent sensitive data from leaving your control in the first place.

# The only audit that always passes: PII never reaches the provider
import requests

# Scrub PII before any provider receives it
scrub = requests.post("https://tiamat.live/api/scrub", json={
    "text": "Review contract for Acme Corp, contact james.wilson@acme.com, TIN 98-7654321"
}).json()

print(scrub["scrubbed"])
# → "Review contract for [ORG_1], contact [EMAIL_1], TIN [TIN_1]"

# Now send to any provider — they see no PII
# Q1-Q10 become moot for this request

When the provider can't answer your questions, your architecture should make those questions irrelevant.

The Audit in One Page

#	Question	Red Flag	Requirement
1	Signed DPA?	ToS = DPA	Bilateral signed document
2	Subprocessors covered?	No list available	Full chain with DPAs
3	ZDR available?	Marketing copy, no technical doc	Written commitment + scope
4	Geographic processing?	"Depends on routing"	Contractual region commitment
5	HIPAA BAA covers this API?	Company-level BAA only	Product-level BAA
6	SOC 2 Type II available?	Type I only	Recent Type II + exceptions review
7	No training on your data?	Opt-out default	Opt-in default or explicit ZDR clause
8	Breach notification timeline?	"As required by law"	Named contact + specific timeline
9	Can verify deletion?	Backup exception, no model answer	Technical deletion confirmation
10	Change-of-control provisions?	None	Auto-delete right on acquisition