As a team that’s knee-deep in development work, we’ve come across various metrics, buzzwords, and frameworks touted as the next big thing to measure our performance. DORA metrics, SPACE framework – they’ve all been in the limelight. But how effective are they truly? Let’s dive in.
Unpacking the Hype: DORA and SPACE
A recent article by Lawrence E Hecht made waves by revealing that a significant portion of engineers weren’t particularly sold on DORA metrics. Now, as developers, we can understand the skepticism.
How Effective Are They Really?
According to Lawrence E Hecht, when engineers were asked about the efficacy of DORA and SPACE frameworks in measuring team performance, the results were intriguing:
For the SPACE framework:
- 5% believed it to be very effective.
- 18% found it effective.
- 4% deemed it somewhat effective.
- 18% felt it wasn’t effective.
- A whopping 55% of respondents were unsure or didn’t know about its efficacy.
For DORA metrics:
- 8% found it very effective.
- 26% felt it was effective.
- 31% believed it was somewhat effective.
- 6% deemed it not effective.
- 29% of the respondents were unsure or didn’t know about it.
This indicates a significant gap in awareness and understanding, with many engineers uncertain about these tools’ effectiveness.
But the question remains: How do these metrics compare to other commonly used measures in gauging team productivity? According to data gathered, the most commonly used approaches include “Cycle time,” “Deploy frequency,” and “Lead time,” with “Meeting deadlines” being ranked as the most useful productivity measure.
On our team, some swear by DORA, while others still question its actual utility.
DORA metrics (DevOps Research and Assessment) by Google have always been in the spotlight. However, while they can offer insights, they aren’t always tailored to the intricacies of every development team. For instance, while DORA might offer robust general insights, it doesn’t always cater to the unique challenges and requirements of niche projects.
SPACE, albeit lesser-known, isn’t faring much better. Developed by GitHub, Microsoft Research, and the University of Victoria, SPACE might have academic backing, but its real-world application remains in question. Over half of our team wasn’t even aware of it until recently.
Beyond Metrics: What Truly Matters?
The New Stack brought up an excellent point: focusing solely on individual developer metrics might divert our attention from more critical team-based or business achievements. And we couldn’t agree more.
In our team discussions, we’ve often found that metrics like user growth and satisfaction are pivotal. They give a holistic view of how well our software or app is being received, rather than just how fast or efficiently it was developed.
Moreover, meeting service-level objectives (SLOs), though ranked last in options, is crucial for us. It’s an indicator of the quality we deliver. And in a world where quality trumps quantity, it’s a metric we can’t afford to overlook.
The Quantification Conundrum
Surveys might be touted as the least effective approach to measuring developer productivity, but they do hold some water in specific scenarios. For instance, DX’s DEVEX 360, co-founded by Abi Noda, promises an easier way for software managers to gather insights and compare them with industry benchmarks. Yet, the efficacy of surveys largely hinges on their design and length.
Google’s “Accelerate State of DevOps Report” took a smart approach by trimming their survey size, ensuring better participation. And, while collecting non-burdensome data is essential, the challenges arise when we aim to quantify “soft” aspects such as culture.
Google’s attempt to build a proxy for productivity by assessing work alignment with skills, value creation, and efficiency is commendable. Still, it doesn’t provide a concrete, one-size-fits-all number for developer productivity.
Wrapping Up: Our Take on Metrics
While frameworks and metrics like DORA and SPACE have their merits, a one-size-fits-all approach doesn’t cut it in the diverse world of software development. The best metric will always be tailored to a team’s unique needs, objectives, and challenges.
In the end, it’s not just about how quickly or efficiently we develop but about the tangible value and quality we bring to the table. And as the DevSquad team, we believe in a balanced approach, combining the best of metrics with real-world feedback, ensuring we deliver nothing short of excellence.
Want to discuss this further? Let’s connect and explore the vast realm of developer metrics together.
Experts in development, customization, release and production support of mobile and desktop applications and games. Offering a well-balanced blend of technology skills, domain knowledge, hands-on experience, effective methodology, and passion for IT.
