Every few months, we see companies like Facebook and Google go in front of the Congress to answer questions like, "Have you ever had a data breach?"
If your stakeholders ask you this question today, what would you say?
As developers and engineers (I still don't get the difference), there are a lot of benefits on being data-driven. Perhaps we can use metrics to answer questions like, ”How secure are we today vs yesterday, last month or even last year?”
Here are just a few metrics I've seen people use to measure security:
- Number of resolved vs unresolved security risks
- Number of hosts running unpatched or outdated kernels
- Number of recalled or deprecated dependencies
- Number of detected vulnerabilities through manual or automated code review (ex: static code analysis)
- etc.
I think having security metrics is one way we can raise the security bar for our team, company, and industry. It can help us drive actions to make our applications more secure in order to protect sensitive data and maintain our customers’ trust.
I understand that the security metrics you choose depends on the type of data or application you are trying to protect. For example, there is not a lot of value in measuring security for trying to protect access to data that is purposely made available to the public.
So given an application that handles sensitive data, what are some ways you would measure its security? What security metrics would you implement?
Top comments (5)
Honestly, it's so hard to say. You can't even measure your system's security by the number of vulnerabilities pentesters find (the most obvious metric!) because all this comments on is the skill of your pentest team. A terrible, insecure site could find 0 vulns with a bad red team, while large companies find numerous issues every week because they have an amazing security team.
If you hire competent pentesters, and they find no issues, perhaps you can say your site is secure. But how do you know the penstesters are competent?
This issue gets worse when you consider that hackers are constantly learning, just as we are. So a site that's secure today might be a disaster tomorrow.
You can try to benchmark your system vs known good processes. For example, are you doing x,y,z things in a standard checklist? But find comparisons to "secure" systems might be a good place to start. Why are the considered secure?
For more explicitly "testing the tester" check out this page, and the Youden index, as well. Maybe you could run the benchmark app through your security process to see how things pan out: owasp.org/index.php/Benchmark
You are absolutely right! The point of measuring these types of metrics is to guide your improvement and make sure that you are going in the right direction. Perhaps a better metric would be the time to resolution.
I really like the idea of time to resolution. If your teams can't make maintainable well tested code, they likely aren't making secure code. If they can manage tech debt (and then you manage security flaws as heavy teach debt) then you are in a better position to improve security practices.
You can help them out with tools like SAST, DAST, IAST and code quality tools, but they still rely on your team's having competency to fix issues when they are detected and to get those fixes into production.
I agree. I remember reading a paper about a study showing a corellation between the number of bugs and security vulnerabilities found in software (I'll link it here is I find it). Insisting in higher standards when it comes to code quality is definitely a step towards a more secure system.
Thanks for the tools! I'll use these as a reference.