“What gets measured gets managed.” — But what if you’re measuring the wrong thing?
You can have a team achieving ASPICE Level 2 on paper, with beautiful plans 📊 and tracked deadlines 📅. But if no one assessed the risk of a faulty braking algorithm 🚨, you’re measuring motion, not progress. You’re managing the schedule, but not the safety.
In Episode 2, we learned the what — the maturity levels. Now, let's talk about the how — the two fundamental lenses for an ASPICE assessment: Capability-based 🔧 and Risk-based ⚠️.
Choosing between them isn’t a technicality; it’s a strategic decision that determines whether you get a true picture of your engineering health 🩺 or just a glossy brochure ✨.
Driving a sports car fast is fun — until you find out the brakes were never tested. 🏎️💥 (Gemini generated image)
The Core Idea: Two Lenses, One Goal 🎯
Think of it like a medical check-up:
A Capability-Based Assessment is your full annual physical 🩺. It checks all your vitals, runs standard tests, and gives a broad overview of your health. It’s systematic, comprehensive, and reveals systemic issues.
A Risk-Based Assessment is a targeted MRI 🧲 for a specific pain. Your knee hurts? 🦵 Let’s scan the joint, ligaments, and cartilage in detail. It’s focused, deep, and designed to investigate a known or suspected problem.
Both are essential. One tells you your overall health (Capability). The other investigates a critical weakness before it becomes a catastrophe (Risk).
Capability-Based Assessment: The Full Physical 🩺
The Reality:
This assessment measures the maturity of your processes across the board 📋. It follows the V-model thoroughly, assessing a set of processes to determine their capability level (0–5). The question it answers is: "Is our entire engineering system mature, standardized, and predictable?"
When it's used:
- New supplier qualification 🤝
- Major program kick-offs 🚀
- Internal benchmarking to understand the complete baseline 🧭
The Value:
Builds a foundation of excellence 🏗️. It ensures consistency and predictability across all projects, not just the "important" ones.
🚨 Snake Oil Alert:
“We did a ‘Light’ Capability Assessment!”
Consultants or internal teams under pressure may sell a “light” version 💨. This often means skipping processes, sampling superficially, or ignoring attributes. The result is an inflated rating ⭐ that looks good on a vendor slide deck 📑 but crumbles under the slightest pressure. It’s a full physical where the doctor doesn’t bother to take your blood pressure.
The Organizational Challenge:
These assessments are resource-intensive ⏳. They require access to multiple projects, countless artifacts, and significant time from key engineers 👨💻👩💻. Leaders often balk at the cost 💸 without seeing the ROI in prevented recalls and reduced firefighting 🔥. The outcome can feel like a “report card” 📝 that triggers blame rather than improvement.
Risk-Based Assessment: The Targeted MRI 🧲
The Reality:
This assessment focuses on the areas of highest potential failure ⚠️. It asks: “Given this specific feature (e.g., autonomous emergency braking 🚘💥), this technology (e.g., new AI chip 🤖), or this team’s history 📚, where are we most likely to fail, and are our processes strong enough there to prevent it?”
It’s not about a level; it’s about confidence ✅.
When it’s used:
- For a safety-critical component 🛑
- When integrating a new technology 🔬
- After a major project failure 💣
- As a follow-up to a broader assessment 🔄
The Value:
Directly enhances product safety 🛡️ and reliability. It’s efficient, focusing precious resources on what matters most 🎯.
🚨 Snake Oil Alert:
“We don’t need capability, we’ll just manage risks.”
This is the mantra of the perpetually chaotic organization 🌪️. They use risk as an excuse to avoid building fundamental engineering discipline 🧱. You can’t effectively mitigate a process risk (e.g., “requirements are unclear” ❓) if you don’t have a managed process to improve (SWE.1). Risk-based must build upon capability, not replace it.
The Organizational Challenge:
It requires deep honesty and vulnerability. Teams must be willing to say, “Our braking algorithm is a high-risk area.” 🚨 This opens them up to scrutiny 👀. Without a blameless culture 🕊️, people will hide risks rather than surface them. It also requires real expertise 🎓 to correctly identify and prioritize the true risks.
The choice of assessment method isn't binary. The most mature organizations blend both.
The Sweet Spot: Intelligence-Driven Assessment 🧠
The choice isn’t binary. The most mature organizations blend both 🔄.
- Start with a Baseline: Use a capability-based assessment to understand your systemic strengths and weaknesses.
- Prioritize by Risk: Analyze the results. Where were the lowest scores? What processes are linked to your highest product risks? Example: A low score in software testing (SWE.5) 🧪 for a team building braking software 🚘 = 🚨 red flag.
- Zoom In with a Risk-Lens: Conduct a focused, risk-based deep dive 🔍 into those critical areas. This is where you move from a score to a meaningful action plan.
This approach tells you that you have a problem (capability) 🔧 and how bad that problem truly is (risk) ⚠️.
The Takeaway: It’s About Asking the Right Question ❓
Don’t ask “What level do we need to achieve?”
Instead, ask: “What do we need to be confident in?” 💡
- Confidence in a new supplier? 🤝 → Capability-Based assessment.
- Confidence in your steering system? 🛞 → Risk-Based assessment.
- Confidence in your entire vehicle? 🚗 → A Baseline Capability assessment, with Risk-Based deep dives on critical components.
Ultimately, both lenses exist for the same reason: to replace guesswork with evidence 📊, and fear with confidence ✅.
Pull-Quote for Leaders:
“Weak leaders settle for PowerPoint scores 💻✨. Strong leaders invest in truth 💡 — even when it’s uncomfortable.”
What’s Next?
You’ve chosen your lens 🔍. Now, who’s holding the camera? 📷
In the next episode, we’ll pull back the curtain on the people in the room: The Assessors. Who are they, what makes a good one (and a bad one), and how do you navigate an assessment without losing your sanity 🧘 — or your team 🤝?
🔖 If you found this perspective helpful, follow me for more insights on software quality, testing strategies, and ASPICE in practice.
© 2025 Abdul Osman. All rights reserved. You are welcome to share the link to this article on social media or other platforms. However, reproducing the full text or republishing it elsewhere without permission is prohibited.
Top comments (0)