A contractor leaves. Nobody knows which .env files still hold production database passwords. Continuous integration (CI) uses the same AWS access key in staging and production. A secret scanner fires on Friday night, and the on-call engineer rotates nothing because "we will deal with it Monday."
That is not a tooling problem alone. It is a maturity problem. Teams buy HashiCorp Vault or AWS Secrets Manager and still behave like secrets live in chat logs and shared spreadsheets.
In this article, you will learn a five-level secrets management maturity model for DevSecOps teams. You will assess where your organization sits today, recognize the signals at each level, and follow concrete steps to move up without boiling the ocean.
Who this is for: Platform engineers, DevSecOps engineers, security champions, and engineering leads responsible for credentials in cloud and CI/CD environments.
What you will learn: How to map people, process, and technology practices to maturity levels, and what to fix first at each stage.
Prerequisites:
- Basic familiarity with environment variables, API keys, and cloud identity
- Awareness of your team's current secret storage (even if the answer is uncomfortable)
Related work: For the detection layer that catches secrets before merge, see Secret Scanning in CI: Three Layers with Gitleaks. For identity instead of keys on AWS compute, see Terraform EC2 and S3 Upload Access.
TL;DR
- Level 1: Ad hoc: secrets in code, chat, and local files; no owner, no rotation.
- Level 2: Centralized: A vault or cloud secret store exists, but access is manual and broad.
- Level 3: Guarded: secrets stay out of git, scanners block merges, and rotation has a calendar.
- Level 4: Short-lived: workloads and CI use roles, OIDC, or dynamic credentials, not standing keys.
- Level 5: Governed: automated rotation, audit evidence, break-glass, and blast-radius controls are standard.
- You do not skip levels. You stabilize each before adding complexity.
- Measure maturity with observable signals: git history, CloudTrail, scanner results, and time-to-revoke after offboarding.
Table of Contents
- Why "We Will Use a Vault Later" Fails
- What the Maturity Model Is (and Is Not)
- The Five Levels at a Glance
- Level 1: Ad Hoc Secrets
- Level 2: Centralized Storage
- Level 3: Guarded Delivery
- Level 4: Short-Lived Credentials
- Level 5: Governed Lifecycle
- How to Assess Your Current Level
- How to Move Up One Level at a Time
- How to Map Tools to Maturity Levels
- How to Verify Progress
- When This Breaks Down
- Frequently Asked Questions
- Conclusion
- References
Why "We Will Use a Vault Later" Fails
| Common habit | What breaks |
|---|---|
Shared .env files in Slack or email |
No audit trail; offboarding does not revoke access |
| One production API key reused in CI, scripts, and laptops | Single leak compromises every path |
Vault installed but apps still read process.env from flat files |
Tooling theater, maturity does not move |
| Rotation "scheduled" with no owner | Keys age past policy; audits fail quietly |
| Scanner alerts ignored as false positives | Real leaks hide in noise |
Key idea: Maturity is the combination of where secrets live, who can access them, how long they last, and what you can prove after an incident.
What the Maturity Model Is (and Is Not)
This model is a practical assessment framework for engineering teams. It is not a compliance certification, and it is not tied to one vendor.
It answers four questions at each level:
- Storage: Where do secrets live?
- Delivery: How do apps and pipelines receive them?
- Lifecycle: How are secrets created, rotated, and revoked?
- Evidence: What logs and metrics prove control?
You can use it in a one-hour workshop with platform and application leads. Score each domain honestly. Your lowest domain sets your effective level.
The Five Levels at a Glance
| Level | Name | Storage | Delivery | Lifecycle | Evidence |
|---|---|---|---|---|---|
| 1 | Ad hoc | Code, chat, local files | Copy-paste | None | None |
| 2 | Centralized | Vault, Secrets Manager, Parameter Store | Manual fetch, broad IAM | Rare manual rotation | Partial access logs |
| 3 | Guarded | Central store + no secrets in git | CI injects at runtime; least privilege emerging | Scheduled rotation for critical secrets | Scanner + access logs |
| 4 | Short-lived | Central store + workload identity | OIDC, IAM roles, dynamic secrets | Automatic expiry by design | CloudTrail, session attribution |
| 5 | Governed | Tiered stores by sensitivity | Zero standing privilege where possible | Automated rotation + break-glass | Audit-ready reports, MTTR metrics |
Level 1: Ad Hoc Secrets
You are here if: passwords and API keys live in source code, wiki pages, ticket comments, or shared drives, and nobody can list every production credential.
Typical signals
-
.envfiles committed to git (current or historical) - "Ask Sarah for the prod key" is a documented onboarding step
- No named owner for database or cloud credentials
- The offboarding checklist does not mention secret revocation
Risk profile
- Time to revoke after departure: unknown or days
- Blast radius of one leak: entire environment
- Audit answer to "who accessed this secret?": shrug
Anti-patterns you will recognize
- The onboarding packet: a Google Doc titled "Prod Credentials" shared with every new hire.
- The demo that became production: a Stripe test key hardcoded in a tutorial branch that merged years ago.
- The hero key: one senior engineer's personal AWS access key wired into cron jobs because "it was faster than fixing IAM."
None of these fails because the team is careless. They fail because there is no model for what "better" looks like.
What to fix first (exit criteria for Level 1)
- Inventory every known production secret (spreadsheet is fine for week one).
- Assign an owner per secret class (database, cloud, third-party API).
- Remove secrets from the latest code branch, not git history yet; stop the bleeding first.
- Run a secret scanner on the default branch and open issues for findings.
You are ready for Level 2 when no new secrets are added to git and every production credential has an owner.
Level 2: Centralized Storage
You are here if: secrets moved into AWS Secrets Manager, Systems Manager (SSM) Parameter Store, HashiCorp Vault, or similar, but humans still copy values manually, and IAM policies are wider than necessary.
Typical signals
- Central store exists; developers still export secrets to local
.envfor convenience - CI reads long-lived keys from the store, but stores them as pipeline variables duplicated everywhere
- Rotation is possible in the product, but not enabled
- Staging and production share the same credentials "to save time"
What good looks like at Level 2
- One approved store per environment class (non-prod vs prod)
- Read access scoped by role, not by individual IAM users with access keys
- Documentation: which app reads which secret path
Example: store a database URL in AWS Secrets Manager
aws secretsmanager create-secret \
--name "/prod/myapp/database-url" \
--description "Production database connection string" \
--secret-string "postgresql://user:PASSWORD@host:5432/dbname"
Restrict read with an IAM policy on the application role, not on a shared developer user:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ReadAppDatabaseSecret",
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:/prod/myapp/database-url-*"
}
]
}
Exit criteria for Level 2
- Zero net-new secrets in repositories (enforced by habit, not yet by CI gate)
- Production secrets exist only in the approved store
- Each secret path has an owner and a documented consumer list
Parameter Store vs Secrets Manager (quick choice)
| Need | Start here |
|---|---|
| Simple config values, non-rotating strings | SSM Parameter Store (SecureString) |
| Database passwords with automatic rotation | AWS Secrets Manager |
| Cross-account secret sharing with audit | Secrets Manager + resource policy |
| On-prem or multi-cloud with dynamic DB creds | HashiCorp Vault |
For many Level 2 teams on AWS, SSM for config and Secrets Manager for credentials that rotate is enough. Do not delay Level 2 while debating Vault versus cloud-native stores.
Level 3: Guarded Delivery
You are here if: secrets rarely enter git because controls enforce it, rotation is scheduled for high-risk credentials, and CI/CD injects secrets at runtime instead of baking them into images.
Typical signals
- Pre-commit or pull request secret scanning blocks merges
- CI pulls secrets from the store during deploy, not from static variables checked into the pipeline repo
- Critical secrets (database admin, cloud root-adjacent keys) have a rotation date
- Separate credentials for staging and production
How to add a merge gate with Gitleaks in GitHub Actions
name: secret-scan
on:
pull_request:
jobs:
gitleaks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Pair scanning with a shared allowlist (.gitleaks.toml) so test fixtures do not train people to ignore alerts.
Rotation calendar (minimum viable)
| Secret type | Maximum age | Owner action |
|---|---|---|
| Database admin | 90 days | Automated or ticketed rotation |
| Third-party API (billing, email) | 90 days | Owner rotates + updates store |
| CI deploy keys (if still required) | 30 days | Prefer elimination over rotation |
| TLS private keys | Before cert expiry | Automated renewal where possible |
Exit criteria for Level 3
- Pull requests with new secrets fail CI
- No standing production keys on developer laptops for routine work
- Rotation tickets exist and close on schedule for tier-1 secrets
- Incident runbook includes "revoke and rotate" as step one, not "delete git history" alone
How Level 3 maps to compliance language
Auditors rarely ask "What maturity level are you?" They ask for controls. Level 3 typically satisfies early evidence for:
- Access control: secrets not in source code; store access IAM-scoped
- Change management: PR gates show who approved code without embedded credentials
- Logging: central store access logs retained
You still need policy documents and owners. The model tells you which technical artifacts to produce.
Level 4: Short-Lived Credentials
You are here if: applications and pipelines assume roles or exchange identity tokens instead of reading long-lived API keys. Secrets expire by design. Session attribution appears in logs.
Typical signals
- GitHub Actions uses OpenID Connect (OIDC) to assume AWS IAM roles, no
AWS_ACCESS_KEY_IDin GitHub secrets for deploy - EC2, ECS, Lambda, and Kubernetes workloads use instance/task/pod identity, not embedded keys
- Database access uses IAM authentication or credentials with a TTL under 24 hours, where supported
- Dynamic secrets (Vault database engine, AWS STS) appear in architecture diagrams
Example: GitHub Actions OIDC to AWS (pattern)
Trust policy on the deployment role (simplified):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/token.actions.githubusercontent.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
},
"StringLike": {
"token.actions.githubusercontent.com:sub": "repo:ORG/REPO:ref:refs/heads/main"
}
}
}
]
}
Workflow step:
permissions:
id-token: write
contents: read
steps:
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::ACCOUNT_ID:role/github-deploy-prod
aws-region: eu-west-1
The pipeline never stores static AWS keys in GitHub.
What changes in threat model
- Leaked CI logs no longer contain permanent cloud credentials
- A compromised laptop does not equal compromised production API access
- Revocation means "end the session" plus policy change, not hunt for copied keys
Exit criteria for Level 4
- Tier-1 systems have no long-lived cloud access keys in CI
- Workloads use identity-based access for cloud APIs where available
- CloudTrail (or equivalent) shows who assumed which role for deploy paths
Level 5: Governed Lifecycle
You are here if: secret management is measurable, audited, and resilient. Rotation is automated where possible. Break-glass exists but is rare, logged, and reviewed.
Typical signals
- Automated rotation enabled in Secrets Manager or Vault for supported secret types
- Break-glass credentials are sealed, monitored, and used only in emergencies
- Mean time to revoke (MTTR) after offboarding or leak is tracked and under team SLO
- Compliance evidence (SOC 2, ISO 27001) pulls from logs, not screenshots
- Secret sprawl reviews happen quarterly; unused secrets are deleted
Break-glass principles
- Separate break-glass from daily admin roles.
- Require MFA and ticket reference for use.
- Alert on every break-glass authentication.
- Review usage in post-incident or weekly security sync.
Evidence pipeline (what auditors ask for)
| Control | Evidence source |
|---|---|
| Secrets not in source code | Gitleaks CI history + clean default-branch scan |
| Access limited by role | IAM policy export + role assumption logs |
| Rotation | Secrets Manager rotation Lambda logs or Vault audit log |
| Offboarding | HR date vs IAM disable date vs secret version deprecation |
Exit criteria for Level 5
- Documented SLO: for example, "revoke all access within 4 hours of termination"
- Automated rotation for ≥80% of tier-1 secrets that support it
- Quarterly access review with deletion of unused secrets
- One successful tabletop exercise: leaked key → rotate → verify blast radius contained
Metrics worth tracking at Level 5
| Metric | Why it matters |
|---|---|
| Mean time to revoke (MTTR) after offboarding | Proves HR and IAM stay in sync |
| Mean time to rotate after scanner alert | Proves alerts are not cosmetic |
| Count of active IAM access keys older than 90 days | Surfaces Level 4 debt |
| Secrets with no read in 90 days | Candidates for deletion |
| Break-glass uses per quarter | Should be near zero; spikes warrant review |
How to Assess Your Current Level
Run this workshop in 60 minutes. Score 1 (no), 2 (partial), 3 (yes) for each row. Average per level. Your floor level is the lowest level where any critical row scores 1.
Level 1 indicators
| Question | 1 | 2 | 3 |
|---|---|---|---|
| Are production secrets committed to git today? | |||
| Can you list all production API keys in one document? | |||
| Does offboarding include credential revocation? |
Level 2 indicators
| Question | 1 | 2 | 3 |
|---|---|---|---|
| Is there one approved secret store per environment? | |||
| Do apps fetch secrets at runtime (not build time in images)? | |||
Are IAM policies scoped to secret paths, not *? |
Level 3 indicators
| Question | 1 | 2 | 3 |
|---|---|---|---|
| Do PRs fail when scanners find new secrets? | |||
| Are staging and prod credentials separate? | |||
| Do tier-1 secrets rotate on schedule? |
Level 4 indicators
| Question | 1 | 2 | 3 |
|---|---|---|---|
| Does CI use OIDC/roles instead of static cloud keys? | |||
| Do workloads use instance/task identity for cloud APIs? | |||
| Can you attribute deploy actions to a role session in logs? |
Level 5 indicators
| Question | 1 | 2 | 3 |
|---|---|---|---|
| Is rotation automated for supported secret types? | |||
| Is break-glass defined, monitored, and reviewed? | |||
| Do you track time-to-revoke after incidents? |
Verification signal: If two teams score themselves Level 4 but only one uses OIDC for CI, trust logs, not self-assessment slides.
One-hour workshop agenda (copy for your team)
| Minutes | Activity |
|---|---|
| 0–10 | Define scope: which systems are in-scope for "production secrets" |
| 10–25 | Silent scoring: each lead fills indicator tables independently |
| 25–40 | Compare scores; argue with evidence (logs, scans), not opinions |
| 40–50 | Agree on effective level and pick one exit criterion for next sprint |
| 50–60 | Assign owner, due date, and verification command (for example, gitleaks detect) |
Publish notes internally. Re-run quarterly. Maturity without remeasurement becomes a poster.
How to Move Up One Level at a Time
Do not jump from Level 1 to Level 5 in one quarter. Sequence work that sticks.
Level 1 → 2
- Secret inventory workshop.
- Pick one store: AWS Secrets Manager, SSM Parameter Store (SecureString), or Vault.
- Migrate the top five highest-risk secrets.
- Delete local copies from laptops after migration.
Level 2 → 3
- Deploy three-layer secret scanning (local, PR, default branch), see related Gitleaks article.
- Remove secrets from CI YAML; inject from store at deploy time.
- Split staging and production credentials.
- Put rotation on the calendar for database and payment API keys.
Level 3 → 4
- Replace GitHub static AWS keys with OIDC federation.
- Move EC2/ECS workloads to IAM roles (no access keys on instances).
- Enable CloudTrail log file validation and alert on
CreateAccessKey. - Deprecate one long-lived key per sprint until none remain for tier-1 paths.
Level 4 → 5 (ongoing)
- Enable automatic rotation where the datastore supports it.
- Define break-glass; run one drill.
- Build evidence exports for your compliance framework.
- Set MTTR SLO and review monthly.
How to Map Tools to Maturity Levels
Tools do not equal maturity. They enable it when the process matches.
| Tool | Typical starting level | What it unlocks |
|---|---|---|
.env + gitignore only |
Level 1 (fragile) | Local dev convenience not production control |
| AWS SSM Parameter Store (SecureString) | Level 2 | Central storage, IAM-scoped access |
| AWS Secrets Manager | Level 2–5 | Storage, rotation Lambda, audit integration |
| HashiCorp Vault | Level 2–5 | Dynamic secrets, PKI, advanced policies |
| Gitleaks / GitHub secret scanning | Level 3 | Merge gates, hygiene scans |
| GitHub OIDC + cloud IAM | Level 4 | Short-lived CI credentials |
| AWS IAM Roles Anywhere / workload identity | Level 4 | Non-AWS workload federation |
| SIEM / CloudTrail alerts | Level 5 | Evidence and anomaly detection |
Common mistake: buying Vault (Level 2–5 tool) while teams still email secrets (Level 1 behavior). Fix behavior and inventory first.
Kubernetes and container notes
Container platforms often stall at Level 2–3 because secrets become Kubernetes Secret objects mounted as files, better than git, but still long-lived and often over-shared across namespaces.
| Practice | Maturity lift |
|---|---|
| External Secrets Operator pulling from Vault or Secrets Manager | Level 3 delivery |
| Separate namespaces and RBAC per environment | Level 3 isolation |
| Workload identity (IRSA on EKS, GKE workload identity) instead of static cloud keys in secrets | Level 4 |
| Sealed Secrets or SOPS for git-stored encrypted manifests | Level 3 bridge not a substitute for rotation |
If your roadmap includes Kubernetes, treat cluster secrets as a delivery mechanism, not a maturity destination. The level is defined by lifetime, scope, and evidence, not by whether the YAML type is Secret.
How to Verify Progress
Use observable checks, not maturity posters on the wall.
Quick technical checks
# Historical secrets in git (example with gitleaks)
gitleaks detect --source . --verbose
# List IAM users with access keys (AWS CLI)
aws iam generate-credential-report
aws iam get-credential-report --output text | cut -f1,9,11,14
Interpretation:
- Gitleaks findings on the default branch → below Level 3 until resolved and prevented
- IAM users with active access keys used by automation → below Level 4 for those paths
- No rotation events in Secrets Manager for 12+ months → below Level 5 for those secrets
Process checks
- Offboarding ticket includes secret revocation, closed before the access removal ticket closes.
- New service checklist includes "no secrets in image" and "identity-based cloud access."
- Incident runbook step one: rotate and revoke, not only "remove from git."
When This Breaks Down
-
Maturity cosplay: central store exists, but every team still keeps a backup
.env"just in case." - Scanner fatigue: allowlists grow until real leaks merge. Tune rules; do not disable gates.
- Rotation without revocation: new secret version exists; old keys still work everywhere.
- Zero standing privilege everywhere day one: teams bypass controls with shared admin "temporarily" for years. Sequence Level 4 by workload tier.
- Multi-cloud sprawl: Each cloud has a different model. Pick a logical maturity target per environment, not one global tool mandate.
Objections you will hear (and how to answer them)
"Rotation will cause an outage."
Start with non-production secrets and one low-risk production credential. Automate rollback in the store. Outages come from surprise dependencies, not rotation itself.
"OIDC is too complex for our small team."
One GitHub Actions deploy path to one AWS role is a weekend project. That single path often removes the highest-risk static key.
"Developers need local prod access to debug."
Grant break-glass with logging, not shared prod keys on laptops. Level 5 is not "no access"; it is attributed, time-bound access.
"We passed the penetration test; we are fine."
Point-in-time tests do not prove daily hygiene. Maturity is measured in continuous signals: scanners, logs, and rotation events.
Frequently Asked Questions
Q: Is Level 5 realistic for a ten-person startup?
You can reach Level 4 patterns (OIDC, roles, scanning) on a small team. Level 5 evidence automation often grows with compliance pressure. Aim for Level 4 with one or two Level 5 practices (break-glass, MTTR tracking).
Q: Vault or AWS Secrets Manager?
If you are AWS-primary, start with Secrets Manager and SSM. Add Vault when you need dynamic database credentials, multi-cloud abstraction, or complex policy workflows. Maturity level matters more than brand.
Q: Does deleting secrets from git history count as maturity?
It is a cleanup, not control. Rotation and revocation contain compromise. History rewrite without rotation leaves active credentials in the attacker's hands.
Q: How does this relate to the software development lifecycle (SDLC)?
Level 3 maps to shift-left detection in CI. Level 4 maps to secure deployment and runtime identity. Level 5 maps to operations, audit, and incident response. The model spans the full SDLC—not just production.
Q: Can we stay at Level 3 forever?
Many regulated teams have been doing so for years. Level 4 becomes urgent when CI keys leak, auditors ask for session attribution, or you scale past one cloud account.
Conclusion
In this article, you learned a five-level secrets management maturity model for DevSecOps teams: from ad hoc credentials in chat and code, through centralized storage and guarded CI delivery, to short-lived identity and governed lifecycle with audit evidence.
You can assess your team with the indicator tables, move up one level at a time, and verify progress with scanners, IAM reports, and rotation logs, not slide decks.
Your next step: run the 60-minute assessment workshop, pick the lowest-scoring level, and close one exit criterion this sprint. Maturity compounds when storage, delivery, lifecycle, and evidence move together.
Top comments (0)