DEV Community

Paul Wenham
Paul Wenham

Posted on Originally published at throughlineaudit.com

Collecting SOC 2 Evidence With AI: MCP Connectors, Browser Agents and a Human in the Loop

Compliance platforms boomed for a good reason. Evidence for a SOC 2 or ISO 27001 audit lives across a dozen systems: cloud consoles, repos, HR records and policy docs. Platforms solved that with hundreds of built-in integrations, which was transformative compared with folders of screenshots.

What changed is that the integration layer stopped being proprietary. AI assistants now connect to the same tools directly, through open standards like the Model Context Protocol (MCP). They can read, collect, organise and explain compliance evidence straight from the systems where your controls operate.

There are three ways to do it, and most programs will use all three. This is the auditor's view of each.

The three methods at a glance

Connectors (APIs) Browser agent Human in the loop
How it works AI reads systems through authorised API connections AI drives a signed-in browser session AI tells you exactly what's needed and where; you act
Coverage ~2/3 of a typical startup's tools, ~90% of evidence by volume Nearly anything with a login page Everything
Evidence Structured, timestamped system records (e.g. JSON exports) Screenshots and UI captures Whatever you capture, specified right the first time
Best for Recurring collection, configs, populations Tools without connectors, admin consoles, one-off checks Sign-ins and MFA, judgement calls, sensitive systems

For each piece of evidence, ask: which method produces the strongest artefact with the least ongoing effort?

Method 1: Connectors

Connectors are the most reliable method. They work like the integrations that made compliance platforms valuable, but they're now a property of your AI assistant. Grant access once, and the AI can pull records on demand or on a schedule.

They produce the strongest audit evidence: structured, timestamped records exported directly from the system of record. Think of the actual config of your production databases, the real user list from your identity provider, or the branch-protection rules on your repo. Auditors can test these at the record level, and their provenance is self-evident in a way no screenshot can match.

Coverage is further along than most teams expect. Connector directories already cover cloud infrastructure, code repos and ticketing, knowledge bases like Notion and Confluence, and HRIS tools like BambooHR and Deel. Weighted by evidence volume, that's close to 90% of a startup's audit evidence, because cloud and code are the biggest sources and the best covered.

Anything without a connector usually still has an API, which an agent can call with a read-only credential. Same mechanism, no pre-built wrapper.

Method 2: Browser agents

Browser agents take coverage close to 100%, because they can reach anything you can log into. Where no connector exists, or the evidence genuinely is a screen (an admin toggle, a vendor portal), this is the method that gets it.

The trade-offs cut both ways:

  • Downside: fragility. UI redesigns, popups and loading quirks break runs in ways an API call never does. They usually need you signed in and nearby.
  • Upside: screenshots are the format auditors have tested for decades. A well-captured, timestamped screenshot often moves through an audit with less friction than a novel data export.
  • Upside: scoping. A targeted screenshot of the one relevant setting avoids handing over sprawling records that invite questions beyond the control being tested. That's good audit hygiene, not concealment.

Use the browser for the long tail: tools without connectors, periodic spot-checks of admin settings, and anywhere the request is literally "show me this screen".

Method 3: Human in the loop, AI as navigator

The most overlooked method. The AI doesn't touch your systems. It tells you exactly what to do, and you do it.

That sounds modest until you remember what made evidence collection hard. It was never the clicking. It was the figuring out. Which of the forty AWS console screens does the auditor mean? What does "evidence of access reviews" need to show? Will this screenshot bounce back in three weeks?

Tell the AI your stack and your framework, and it can specify each piece of evidence precisely: the system, the path, what must be visible, and why.

The job is also smaller than the industry suggests. A startup with a right-sized set of 20 to 30 configured controls typically needs 30 to 50 screenshots and exports, most of them from one cloud console. With a precise shopping list, that's hours, not weeks.

This mode stays the right one permanently for sign-ins, MFA and passkeys, systems you'd rather no agent touched, and judgement calls like scoping or wording a management response.

Where your evidence actually lives

Category Typical tools What the auditor needs Best method today
Cloud infrastructure AWS, Vercel, MongoDB Atlas, Snowflake Encryption, logging, network rules, backups Connector or API (highest volume, best covered)
Identity provider Google Workspace, Microsoft Entra ID, Okta User populations, MFA status, group memberships Browser or read-only admin API (thin connector coverage, low volume)
Code and CI/CD GitHub, GitLab, Bitbucket Branch protection, peer review, tests, approval stages Connector
Organisational systems BambooHR, Deel, Rippling, Notion, Confluence, Google Drive Policies, on/offboarding records, board minutes, risk and vendor assessments Connector (search beats paging through a UI)

Your systems are already the system of record

Centralising compliance into a dedicated platform can quietly create the duplication it set out to remove. Your policies already live in Notion or Drive, onboarding in your HRIS, changes in your repo. Copying all that elsewhere means two versions of the truth, and the copy is the one that drifts.

The agentic alternative leaves every record where the control operates and lets AI connect the dots: assembling the evidence set, checking completeness against your control list and flagging what's stale. The interface where humans and AI work together can be whatever your team already lives in.

FAQ

Do you need a compliance platform to pass SOC 2 or ISO 27001? No. Auditors test controls and evidence, not tooling. Platforms can still add real value; it depends on what you're trying to achieve.

Will auditors accept AI-collected evidence? Yes, when it's a source-system artefact (an API export or a screenshot of the live system) with clear provenance and timestamps. What auditors won't accept is an AI-written summary standing in for the underlying record. The record itself is the evidence.

How much evidence does a startup audit involve? For 20 to 30 configured controls, typically 30 to 50 discrete artefacts, plus the recurring records (reviews, tickets, HR events) that build up in the systems where they happen.


Originally published on the Throughline blog. Throughline is a registered CPA firm that audits AI-native companies for SOC 2, ISO 27001 and ISO 42001.

Top comments (0)