DEV Community

Ikkun
Ikkun

Posted on Originally published at agentjournal.dev

The Audit Gap Decides the Platform

TL;DR: In August 2026, Claude Cowork's activity is excluded from Compliance API and audit logs even on Anthropic Enterprise. ChatGPT Work — the equivalent non-engineer desktop surface — is covered. If your organization handles information that must not leak, this single fact outweighs every model-quality argument.

One-line summary: A surface you cannot audit cannot be your primary deployment — regardless of which vendor makes it.

This is Part 1 of a series on selecting an agent platform for an organization with no dedicated engineering staff (see the series overview for context). The client details are anonymized; the vendor facts are not, and each one below is dated and checkable.

What "audit coverage" means here

Three distinct things get called "audit," and confusing them causes bad selections:

  1. Audit logs / Compliance API — organization-level records of user activity, pullable into your SIEM via API. This is what a regulator or your own incident review needs.
  2. OpenTelemetry events — developer-oriented telemetry. Useful, but not a substitute: coverage differs, retention is yours to build, and it is not designed as an evidence trail.
  3. Local history — conversations stored on the user's machine. Admins can't query it centrally at all.

When I say "auditable" in this series, I mean #1. The other two have real value — OTel feeds dashboards and alerts, local history serves the individual user — but neither answers the question "show me everything that happened in this workspace last month."

Why staff-less organizations feel this hardest

In an engineering org, audit gaps can be compensated: someone wires up OTel pipelines, builds retention, writes detection rules. It's work, but it's work their people can do.

In an organization without dedicated engineering staff, there is no compensation path. The vendor's audit coverage is your audit coverage. Whatever the admin console shows you is the whole picture, forever. That asymmetry is why audit coverage sits above model quality in this series' evaluation order:

  • Model benchmarks change monthly; audit scope is structural and survives contract renewals.
  • You can switch models inside a platform; you cannot self-build compliance coverage for a surface where the vendor records nothing.
  • When something goes wrong — and at some point something goes wrong everywhere — the question "what exactly happened?" needs an answer from records, not from users' memories.

The facts on the table (as of August 2026)

Surface Anthropic OpenAI
Chat (web) Compliance API covered Compliance API covered
Non-engineer desktop app Claude Cowork: NOT covered (OTel only; local history is device-local) ChatGPT Work desktop: covered (workspace policies apply)
Coding CLI Claude Code: managed settings + OTel Codex: requirements.toml + Compliance API (CODEX_LOG, CODEX_SECURITY_LOG, ~30-day retention)
Cloud agents Cowork cloud (beta): not covered Codex cloud: covered

Three observations:

  • Cowork's exclusion survives the top plan tier. Anthropic Enterprise adds SCIM, custom roles, and Analytics API — and still documents Cowork activity as out of scope for Compliance API. This is a structural gap, not a pricing gap.
  • OpenAI keeps both audiences inside one auditable container. Non-engineers (Work) and developers (Codex) both land in Compliance API, with Codex events arriving as typed event streams (CODEX_LOG, security events separately).
  • Cowork does have OpenTelemetry, and it's the only telemetry route even on cheaper plans. It's real, and worth wiring into a collector if you allow Cowork at all — but treat it as monitoring, not evidence.

Retention changes the math

Coverage is only half of it; retention is the other half. An audit log that expires in 30 days supports incident triage but not annual review or a formal inquiry. When comparing platforms, ask three questions as one bundle: What is recorded? Per surface? For how long, and in what export format? A platform that answers all three consistently is worth more than one with deeper features and vaguer logging.

Verify it yourself before signing

None of the facts above require trusting this article. The verification takes under an hour with trial or demo tenants:

  1. Open the admin console for each plan tier you're considering. List every surface (web chat, desktop app, CLI, cloud agents) visible to a regular user.
  2. For each surface, generate activity — one conversation, one agent task — then search for it in the audit log / Compliance API export.
  3. Note what's missing. Anything absent from step 2's results is a surface you cannot audit. Write the list down; this becomes a policy input, not a footnote.
  4. Pull one real export via the API and check field completeness (actor, timestamp, action, object). Screenshots in a console UI don't count until the export works.
  5. Check retention terms in writing, in the contract paperwork — not just marketing pages.

Step 3's list is the deliverable that matters. In the selection process this series documents, that list contained Cowork — and that single line item reordered the whole evaluation.

What to do with excluded surfaces

Excluding a surface entirely isn't always possible; users arrive with preferences. The mitigation ladder, weakest to strongest:

  1. Exclude from regulated work by policy — written rule, enforced by review, technically unenforced. Fragile, but honest about its weakness.
  2. Read-only limitation — allowed for summarizing public material, banned for anything sensitive. Requires information-classification discipline (see Part 6 for why that's hard without staff).
  3. Telemetry-first conditional allowance — OTel wired into your collector before first use, alerting on volume anomalies, plus periodic manual review. This is the strongest available posture for Cowork today, and it still isn't evidence-grade.

Whatever rung you choose, the decision should be recorded with a date and a re-check trigger. Vendor terms move; a policy frozen in August 2026 shouldn't silently govern August 2027.

Caveats

  • Vendor terms change without notice; re-check the admin docs before signing anything.
  • This article reflects documentation review and sandbox-tier verification, not a full Enterprise deployment. Pilot results follow in later parts.
  • Nothing here claims Claude models are weaker, nor that Cowork is unsafe. It is a record that its audit story did not match this particular constraint set. If Anthropic brings Cowork into Compliance API coverage, Part 6 lists exactly how the conclusion gets re-evaluated.

Coming August 27: Part 2 — One Admin Console Beats Three

Takeaway: Check Compliance API coverage per surface, per plan tier, before comparing anything else. One excluded surface disqualifies it from hosting regulated work — and exclusions don't always disappear at higher tiers.


Originally published at agentjournal.dev.

Top comments (0)