Security Metrics That Matter in a DevOps World 

The Emergence of DevSecOps DevSecOps has recently gained traction as a fusion of DevOps and security practices. At its core, it’s not just about automating security but creating a culture that yields tangible outcomes for businesses. If DevOps emphasizes software delivery performance, what then should security focus on to bolster this?

Redefining Security in a DevOps Environment

Typically, DevSecOps has been seen as automating the build/delivery pipeline. However, a more comprehensive view places it as a mindset that transcends mere security automation, reflecting a culture aiming at business value.

The challenge now is to determine which security metrics truly assist, rather than hinder, an organization’s business processes.

Four Core Principles for Security Measurement

Drawing from the influential book “Accelerate” by Nicole Forsgren and colleagues, there are four essential principles to remember when measuring security:

1. Global over Team-Level Measurement: Always prioritize business needs over team needs. By measuring at an organization-wide level, you prevent siloed thinking.
2. Outcomes over Outputs: It’s not about how much work you do, but the results it produces. Hours worked or tests conducted don’t necessarily reflect progress; it’s about reducing vulnerabilities and potential attacks.
3. Build Resilience, Not Just Maturity: Rather than trying to meet some abstract “maturity threshold”, the focus should be on strengthening the organization’s ability to handle and recover from vulnerabilities.
4. Focus on the Big Picture: Avoid becoming too engrossed in individual security components; always consider their cumulative impact.

Crucial Security Metrics

Many security metrics are vendor-driven, especially those linked to DevSecOps. Businesses should be wary of this and choose metrics aligning with their security goals.

Three Major Security Metrics:

1. Deployment Metrics: These gauge the deployment process’s health. Examples include time-to-deploy and deployment frequency. The best in this domain can deploy whenever required.
2. Lead Time Metrics: These evaluate the organization’s capability to adapt and provide business value. Examples encompass individual productivity and rework time. The best performers typically have lead times of less than an hour.
3. MTTR (Mean Time to Repair) Metrics: These assess how swiftly threats can be addressed and services restored. Key metrics include time to investigate and remediate. Top-tier performers have an MTTR under an hour.

What Not to Measure: MTTF (Mean Time to Failure) Avoid the trap of measuring MTTF. Failures are bound to happen; the focus should be on metrics that enhance resilience and tackle threats.

Balancing Security with Practicality Enhanced security usually entails sacrifices, such as increased costs or friction. It’s vital to have metrics assessing the repercussions of new security implementations. These might be more time spent on tools or additional burden transferred to other departments.

Questions to guide metric development include:

• What proportion of the team’s time goes to product maintenance vs. problem-solving?
• Are there more support tickets hinting at confusion regarding new security rules?
• Do teams approach you proactively, or do they seem to be avoiding you?

Considering System Complexity The intricacies of your entire system introduce risks surpassing any single component. Such risks should factor into your security plan. This involves evaluating both short-term stresses like active incidents and long-term ones like employee turnover.

Concluding Thoughts While there’s enthusiasm to test early in the Software Development Life Cycle (SDLC), it’s pivotal to anticipate failures even during runtime. Only by comprehending incidents in production can there be a feedback loop to reinforce security. Adopting the right security metrics can pave the way for a business to thrive without security constraints.