The FTC's inquiry into consumer AI products has turned a question most teams deferred into a launch requirement: can you show what your product actually did? That answer depends on a logging decision you make before launch - and can't retroactively fix.
Content vs. Event
Teams face two extremes. Keep every conversation, or keep nothing.
Full transcripts answer any future question, which is exactly why they're expensive. Every retained message is breach surface, subpoena surface, and a trust cost you pay in your privacy policy. Teams that choose this path often discover their retention window was set by whoever wrote the database schema, not by anyone who thought about risk.
Keeping nothing is cheap and genuinely privacy-protective - until someone asks how often your assistant encountered a user in distress and what it said back. You can't answer. Not "we'd rather not" - you structurally cannot, and neither can your own safety team, which means you also can't tell whether last month's guardrail change helped.
The third option is to log the decision rather than the content. For each turn, write a structured event: which classifier fired (a classifier here is just a small model or rule that labels input, e.g. "self-harm mention" or "minor-related"), what action the system took, which policy and model version were live, a session hash. No raw user text.
How do you tell which you need? Ask what question you'll be asked. Aggregate questions - how often, what did the system do, did the rate drop after the fix - are fully answered by event logs. Forensic questions - why did this specific conversation go wrong - need content. Most teams need the first continuously and the second rarely, which argues for event logs by default plus a small consented, access-controlled transcript sample.
Real Example
A minimal safety event, emitted alongside your normal request logs:
{
"session": "sha256:9f2c...",
"ts": "2025-11-18T14:03:22Z",
"flags": ["self_harm_mention"],
"action": "crisis_resource_shown",
"policy_version": "safety-2025.11.1",
"model": "chat-v4.2",
"transcript_retained": false
}
Six fields, no user content. From a month of these you can report: how many sessions triggered the flag, what percentage received the intended response, and whether that percentage improved after safety-2025.11.1 shipped. That is the shape of a regulatory question.
The honest limitation: if a journalist surfaces one bad conversation, these logs tell you the system classified it and what it did - not what the user typed. That gap is the price. Decide consciously whether you pay it, and whether a 1% sampled transcript set with clear user disclosure closes it well enough.
Key Takeaways
- Event-level safety logging answers aggregate accountability questions at a fraction of the privacy exposure of full transcript retention - the two are not equally risky.
- Pick based on the question you expect: rates and trends need events; single-conversation forensics needs sampled content, with disclosure and tight access control.
- Logging is not backfillable. Whatever you don't capture before launch is gone from your first safety review.
If a regulator asked tomorrow how your product handled distressed users last quarter, which fields in your current logs would actually answer it?
Sources referenced: HackerNews discussion on the FTC inquiry into OpenAI, Anthropic and other AI companies
Top comments (0)