I build a small external vulnerability scanner aimed at UK firms preparing for Cyber Essentials (CE), the government-backed baseline certification. The most important design decision in the product was one word I refused to ship: "pass".
The setup
Cyber Essentials has five technical controls. When I mapped them against what a scanner sitting outside your network can actually observe, I got this:
| Control | Visible from outside? |
|---|---|
| Firewalls / internet gateways | Mostly: which services are reachable at all |
| Secure configuration | Mostly: exposed admin panels, unnecessary services |
| Security update management | Mostly: known-vulnerable and end-of-life versions |
| User access control | No |
| Malware protection | No |
Three out of five, roughly. The temptation is to show a green tick on the three you can see. That would be wrong.
Why a scan can fail you but never pass you
An external scan can find a reason to fail. Under the NCSC's CE Plus test spec, a reachable service with a vulnerability scoring CVSS 7.0 or higher fails the external test outright. If my scanner sees one, that's a real, evidence-backed "this will likely fail".
But "no issues found" is not "pass". Plenty of things that matter are invisible from outside:
- default passwords that were never changed
- whether logins lock out after repeated failures
- patching on internal machines
- whether that exposed admin panel has MFA in front of it
So the readiness states in the product are:
- likely fail: evidence of a fail condition
- review: an exposure a human has to justify (e.g. a reachable admin interface needs a documented business need plus MFA or an IP allow list; the scan can see the panel, never the documentation)
- no issues found: we looked and found nothing, which is narrower than "fine"
- no scan yet: different from "no issues found", and worth keeping separate
The two controls a scan can't see at all are customer-attested: the user records a statement, versioned against the scheme version, and the UI never pretends it measured anything.
Simplified, the patching rule looks like this:
type Status = "likely_fail" | "review" | "no_issues_found" | "no_scan_yet";
function updateManagementStatus(findings: Finding[], hasScan: boolean): Status {
if (!hasScan) return "no_scan_yet";
const failing = findings.filter(
(f) => f.open && (f.checkType === "eol" || (f.cve && (f.cvss ?? 0) >= 7.0)),
);
return failing.length ? "likely_fail" : "no_issues_found";
}
Note what's missing: there is no branch that returns "pass".
A rule people mix up
"CVSS 7+ with a fix available for more than 14 days" is often quoted as the CE patching rule. In the CE Plus test spec, that specific check is sub-test 2.1, the authenticated scan of sampled devices, not the external scan. If you're building tooling against a compliance scheme, read the primary documents (NCSC publishes them as PDFs) and not vendor blog summaries. I got one claim wrong from general knowledge before checking them.
The takeaway for anyone building compliance tooling
Your product's vocabulary is a promise. A customer who sees "pass" will act on it, and then fail an audit. Listing what you couldn't check honestly is a feature, not an apology for a missing one.
Disclosure: I'm the founder of CyberSpec, the scanner described here. It's a readiness tool, not a certification body, and isn't affiliated with IASME or NCSC. Happy to answer questions in the comments.
Top comments (0)