DEV Community

Wen Ping
Wen Ping

Posted on

What scan records auditors ask for (MLPS / supply chain)

#mlps #angussecurity

Audit week asks for “security scan records,” and the folder only has green CI screenshots—no policy version, no frozen report, no who approved exceptions. Questionnaires want an evidence chain, not the sentence “we have a scanner.”

Takeaway: Keep four record types: (1) gate policy and changes; (2) completed scan jobs (with engine/rule versions); (3) frozen report snapshots (published external copies when needed); (4) suppressions/exceptions and share-access traces. AngusSecurity provides orchestration and report snapshots. It does not mean “scanning equals MLPS pass,” and it does not replace a qualified assessor’s conclusion. How to generate snapshots: see Reference (report export). Integrators, project owners, and customer security teams need to show auditors what to hand over, how to explain it, and what not to claim.

1. What auditors usually ask

Question What you should produce Common failure
What was scanned? Job targets: Git / artifact / image, IDs and completion time Pipeline name only
With what method? Engine and rule-pack versions (OpenGrep / Gitleaks / Trivy, …) No versions → not reproducible
Did policy block? Thresholds and gate PASSED/FAILED/SKIPPED “Job was green” only
What was the conclusion then? A report snapshot (incl. data-as-of style fields if shown) Live Findings list
Who approved exceptions? Suppression / risk-accept: reason, approver, TTL Permanent suppressions
Are external materials controlled? Publish state, share TTL/password, access logs if enabled Sharing drafts; secrets in PDFs

One-line answer for questionnaires: Findings for day-to-day work; frozen reports for audit and external delivery; policy explains why you blocked; exceptions explain why you did not.

2. Minimum archive set

2.1 Policy and config

  • Gate policy: severity thresholds, scope, who changed what when.
  • Capability switches: whether release gates include sast / secret / sca (and license).
  • Change ticket or config-repo commit hash (policy-as-code).
  • Written reason for SKIPPED: uncovered ≠ passed.

2.2 Scan jobs

  • Job ID, completion time, target type/ref, capabilities, gate result.
  • Separate over-threshold (fix) vs execution errors (engine/auth—exit codes in the replace-scripts article).
  • Release evidence uses completed jobs with full policy coverage only.

2.3 Report snapshots

  • Per release or per month: management / external audience → wait until READY → download PDF/HTML/JSON.
  • Keep the data-as-of timestamp from cover/metadata.
  • External: external/third-party audience, redaction, publish then share; keep evidence ZIP when used.
  • A monthly report is governance material—not an MLPS assessment report.

2.4 Exceptions and shares

  • Finding suppressions: fingerprint, reason, approver, TTL (rescan when expired).
  • Share links: TTL, password, access records.
  • Secret findings externally: counts only—never raw secret text.

3. Release audit pack layout (example)

Rename folders to match your policy, but keep a stable per-release tree:

audit/releases/x.y.z/
├── README.txt
├── policy/
│   ├── gate-policy-export…
│   └── capability-matrix.txt
├── jobs/
│   ├── job-<id>-summary.json
│   └── doctor-or-engine-versions.txt
├── reports/
│   ├── management-….pdf
│   ├── external-….pdf
│   ├── external-….json
│   └── evidence-….zip
└── exceptions/
    ├── suppressions-export…
    └── share-log-….txt
Enter fullscreen mode Exit fullscreen mode

Order of operations:

  1. Before release: PR/main gate jobs completed and explainable.
  2. Generate engineering or external snapshots from those jobs (see report-export article).
  3. Download READY artifacts into the tree above.
  4. Attach policy and exceptions; on intranet attach doctor / engine / vuln-DB age notes.
  5. External: publish → share (time-boxed) → register recipients. Revoke old links.

Retention: follow your MLPS / internal-control policy. The product does not mandate “keep N years.” Destroy expired share secrets.

4. Mapping to supply-chain questionnaires

Common ask Map to Do not write
Do you run SAST / secret scan? Completed jobs with sast/secret + report chapters “All security testing covered”
Do you manage OSS CVEs? sca jobs + component/CVE chapters or CSV “No known vulns”
Do you manage licenses? License chapter + exception list “Legal final approval” unless counsel signed
Do you produce SBOM? SBOM attach or separate flow (see SBOM article; console-dependent) “SBOM equals supply-chain security”
Can a third party verify? External report + evidence pack / verify (if offered) Shipping unpublished drafts

Do not claim host scanning, pentest, or full-site DAST as “already included”—out of product scope. Capability-narrowed exports: see Reference.

5. Talk track for audit Q&A

They say Answer Never say
“Scanned means we passed MLPS?” “Scans and reports are evidence inputs; assessors issue conclusions.” “We already passed MLPS.”
“Why don’t numbers match the console?” “Reports are point-in-time snapshots; Findings change with fixes/suppressions.” Edit old PDF numbers live
“Is a green CI enough?” “Green proves exit code then; auditors also need policy version, scope, frozen conclusion.” “Screenshot is fine.”
“Can we send the secret report out?” “Externally: counts and disposition only; paths stay on engineering audience.” Rename an engineering PDF and send

6. Failure table

Symptom Cause Fix
Only CI green screenshots No frozen report / policy version Add snapshot + policy change record
Snapshot ≠ live Findings Old snapshot or live list as “report” Explain snapshot semantics; regenerate with timestamp
External PDF has secret-like strings / repo URLs Wrong audience/redaction Void; regenerate external template and publish
Claimed “passed MLPS” Overclaim “Evidence ready; assessment separate”
Suppressions never expire No TTL Set TTL and rescan
Release used failed/skipped jobs Treated SKIPPED as pass Use completed, fully covered jobs

FAQ

Can a scan report replace an MLPS assessment report?

No. It is an evidence input for SDLC / supply-chain controls. Assessors execute the standard.

Must every job produce a report?

No. Use Findings/alerts daily; snapshot on release, monthly, or audit windows.

Should we print control-point IDs in report titles?

Avoid hard-coding unreviewed control IDs in public PDFs as if they were conclusions. Keep internal mapping in your policy annex; product reports cover scope, method, and exceptions—not compliance judgments.

Is community edition enough for the evidence chain?

Community can run the scan + report path (GPL-3.0). Whether auditors accept your private deploy shape depends on your policy and assessment scope. Commercial differences follow console/docs—no pricing here.

Summary

  • Chain: policy → completed jobs → frozen reports → exceptions and shares.
  • Snapshots for audit; Findings for daily work; CI green is not enough.
  • Never claim scanning equals MLPS pass; do not fake DAST/host inventory scope.
  • External audience only for outbound; secrets as counts; stable per-release folders.

Reference articles

Top comments (0)