SOC 2 Quality Review Skill: Evaluating Vendor Attestation Report Credibility

What This Skill Does

The SOC 2 Quality Review skill evaluates SOC 2 Type 1 and Type 2 vendor
attestation reports using the SOC 2 Quality Guild rubric. It assesses three
critical dimensions: Structure (S1-S3), Substance (S4-S7), and Source (S8-S11)
to help organizations make informed decisions about vendor credibility before
trusting their security claims.

Key Use Cases

• Reviewing vendor SOC 2 Type 1/Type 2 reports
• Triage report credibility assessment
• Producing risk memos for stakeholders
• Preparing diligence follow-up questions and evidence requests

Three-Dimensional Evaluation Framework

The skill scores 11 signals across three categories using a 0-2 scale where 2
= strong evidence, 1 = partial/ambiguous, and 0 = missing or weak:

Structure (S1-S3)

• S1: Required auditor report structure
• S2: Unsigned management assertion completeness
• S3: Report formatting and organization

Substance (S4-S7)

• S4: Control design testing detail
• S5: Control implementation evidence
• S6: Testing methodology clarity
• S7: Pervasive testing sufficiency

Source (S8-S11)

• S8: CPA firm licensing and verification
• S9: Auditor independence confirmation
• S10: Report signer authority
• S11: Source credibility indicators

Advanced Diligence (S12+)

After initial scoring, the skill runs additional diligence questions to
strengthen the evaluation. This includes deeper probing into control
effectiveness, testing methodologies, and evidence sufficiency for the
specific trust services categories in scope.

Hard Fail Criteria

The skill automatically flags these as high-severity findings:

• Missing required auditor report structure (S1)
• Missing/incomplete unsigned management assertion (S2)
• Unlicensed or unverified CPA firm (S8)
• Pervasive testing vagueness on critical controls (S7)

Decision Output Framework

The skill produces three standardized artifacts:

1. Executive verdict with confidence level (High/Medium/Low)
2. Signal-by-signal scorecard with evidence citations
3. Vendor follow-up request pack with deadlines

Risk Profile Customization

Users can configure:

• Primary audience (Security, Procurement, Customer Trust, or All)
• Risk posture (Conservative, Balanced, Lenient)
• Data sensitivity baseline (High/Medium/Low)
• Evidence strictness (Escalate on Unknown, Conditional acceptance, Case-by-case)

Decision Matrix

Based on the selected profile and evidence findings, the skill recommends:

• Accept: No hard fails, most signals strong
• Accept with conditions: Limited gaps with compensating evidence path
• Escalate: Mixed evidence or source credibility concerns
• Reject: Fundamental structure/source failures

Project Background

This skill was developed using SOC 2 Quality Guild resources at s2guild.org as
a baseline for quality-focused SOC 2 vendor attestation reviews. It was the
first GRC agent created with OpenClaw after extensive testing across multiple
environments including Raspberry Pi, Intel NUC, LXC containers, and Mac Studio
clusters.

Key Differentiators

Unlike basic SOC 2 report readers, this skill:

• Prioritizes evidence quality over report polish
• Penalizes boilerplate language and weak control-to-criteria logic
• Separates auditor credibility from control design concerns
• Provides actionable vendor follow-up requests

When Not to Use This Skill

This skill is not designed for:

• Legal advice or regulatory compliance conclusions
• Formal certification decisions
• Deep technical penetration testing
• Historical incident forensics
• Vendor contract drafting

Skill can be found at:
soc2-quality-review/SKILL.md>