Cloud Custodian is an engine with no rules in it.
You install it, write a policy, it works, you like it. You write another one. Months later there are thirty YAML files that different people wrote on different days, someone asks what your PCI coverage is, and you have no answer. Not a bad one. None.
The official docs have 109 example policies and not one of them says which control it covers. Prowler ships that mapping for free, so does Steampipe. Cloud Custodian never did.
So here is mine: 325 policies, one YAML file per AWS service, each naming the control it covers.
github.com/gustavoortega/cloud-custodian-compliance-policies
Using them
Take the file for a service and run it. Nothing to adopt, no wrapper.
$ custodian run -s out policies/aws/rds.yml
That is 34 RDS policies against whatever your credentials point at. Or copy five into the YAML you already have. They are ordinary c7n policies.
For a whole organisation the layout already matches what c7n-org expects:
$ c7n-org run -c accounts.yml -s out -u policies/aws/
One thing before you do. Some policies carry an allow-list of account IDs that ships as 111111111111 with a REPLACE comment. Those filters treat anything off the list as external, so until you put your own accounts in they will flag every grant between your own accounts. Loud, never quiet.
What the mapping buys you
Each policy carries this:
metadata:
severity: high
frameworks: ['FSBP RDS.3', 'PCI 3.5.1']
Plain c7n, no fork, and it travels into the run's output. Every finding arrives knowing its control, so your dashboard groups by framework without a lookup table you maintain by hand.
And you can finally answer the question:
$ c7n-kit coverage catalogs/fsbp.txt policies/aws
FSBP 229/369 controls (62%)
orphans: none
229 of the 369 controls in AWS Foundational Security Best Practices. That one gets a percentage because I rebuilt the catalogue from the AWS docs page by page and know it is complete. PCI, CIS and SOX are mapped too, but their catalogues are partial, so those stay as counts. A number whose denominator you cannot defend is worse than no number.
Orphans are the other half: a policy claiming FSBP RDS.99, which does not exist, raises your coverage and errors on nothing. That check caught two in my own files.
The repo has a page per framework with every control listed, covered or not, so "what is missing" is a link rather than an exercise.
What it found
I wrote the harness, then pointed it at the 325 policies I had just published.
74 of them were not doing what their YAML said.
The common one, forty-six: a value: false filter on a field AWS did not return. In c7n a missing key is None, and None == False is False, so the resource does not match and gets reported as compliant. If the field never came back you checked nothing, and the output looks identical to a pass.
Sixteen were worse. An absent key reaches a JMESPath function that cannot take one, the exception kills the run for that whole account, and the account writes no results. Which on a dashboard looks exactly like an account with nothing wrong.
Nine were missing a presence guard before the real filter. Five of those hid findings. The other four did the opposite and are worth knowing about, because they break the pattern: op: not-equal on an absent key returns True, not False. So those four reported healthy resources as findings. Same root cause, opposite symptom.
Three had never worked at all. One filtered on Encrypted when DescribeDBClusterSnapshots returns StorageEncrypted, so it matched zero and called every Aurora snapshot encrypted. One filtered on a key nothing populates and matched every table. One read a load balancer attribute that DescribeLoadBalancers does not return, so it flagged every classic load balancer on every run.
None of it was visible from the output. They had been producing reasonable-looking findings for months.
All 74 are fixed, and every fix ships with the mutation that proves it: remove the guard and the test goes red. Eight more were deliberately left alone and say so in the file, marked DELIBERATE: no absent branch with the reason, because absence there means the safe state and widening the filter would have invented findings.
Stricter than an internal catalogue, on purpose
Somebody running this against their own accounts pointed out that in their fleet most of those fields come back 100% of the time, so the bug never fires and the fix adds nothing. They measured it, and they were right about their fleet.
A published catalogue cannot reason that way. Whoever clones this has a fleet nobody has measured, with services and tiers and regions I have never seen. A policy that reports an unverified resource as compliant is a defect for them even when it would never fire here. So the rule in this repo is the strict one: if the field did not come back, say so.
The same logic sets the limit. Two of the absent branches are scoped rather than open, because an open one would have been worse than the bug it fixed: MSK only for provisioned clusters, since serverless has no such block and mandatory TLS, and ACM renewal only for issued certificates, since the expiry date is absent by design before validation.
What this is not
It only detects. No policy here has an actions: block, because adding remediation to a rule you have not watched for a few weeks is how you find out what it really matches.
It does not replace Security Hub. AWS evaluates all 369 controls natively. I run these because a Security Hub control is a black box you can only suppress while a c7n policy is a filter you can read and fix, and because Config recording has a bill that not every account agreed to pay.
And it will age. Every AWS field rename rots a policy, and a rotted policy returns zero resources, which renders as compliant. Same failure as everything above, which is why the tests exist.
One rule
An absence is not a zero. A field that never came back is not a false, an account nobody could scan is not a clean account, a control ID that does not exist is not covered. Say unknown and leave it on the screen, because all of those errors point the same way, toward everything looking fine.
The policies and the toolkit are Apache 2.0. Issues and PRs welcome, especially for controls I have not covered.
Top comments (0)