How to measure code quality

You can not control what you can’t measure, you can’t measure what you can’t define. You can’t define what you don't know.

Now that code quality and its importance has been established the significance of measuring and monitoring it is pretty self explanatory. In a data driven approach you will need a set of metrics to give clear goals and gather every stakeholder around a project to agree about the standard of acceptance when it comes to quality. eMetrics are a tool to incentive, to communicate, to draw a perimeter.

So when should you measure code quality?

We will answer this question addressing 3 dimension of measuring code quality

  1. Where in the software development lifecycle

The answer is everywhere. Code quality happens at the beginning, the middle and the end of your software development lifecycle and it needs to be monitored everywhere it can be. With continuous monitoring you can accurately identify where your code quality drops and act adequately. Be reasonable though, metrics need to be synthetic but true to the reality of quality. Generating information overload will also deteriorate the effectiveness of your code quality strategy.

If you are looking for best practices to reduce the cost of code quality you should definitely investigate the shift left approach.

This code-quality-enabling technique advocates for continuous and intensive monitoring of your code quality and the setup of of systematic code quality targets as early on in the lifecycle as possible in order to catch flaws before the cost of repairing them skyrockets. You also have quality gates in the CI/CD which consists of creating self sufficient checkpoints before shipping code to make sure the minimum standards of acceptance for a given set of quality metrics is reached. Having quality gates enables developers and managers to see clearly what level of quality needs to be obtained and creates a standard that cannot be manipulated depending on the project owner’s personal bias or the difficulty of meeting the project constraints. They are stable tools, an efficient north star for quality while presenting the flexibility of being open to review and to changes as best practices and know-how regarding code quality evolves within your organization.

Measuring at every step of the life cycle enables you to identify where defects are created, adapt your processes to fix loopholes and gaps in your code quality strategy and bring automation and tooling where manual code review and manual testing generates flaws.

  1. Where in the process

Before and after your changes is the adequate way. In a snapshot approach to code quality control you want to take a picture before and after changes in order to identify when and where you generate most defects and consequently design new approaches to build code quality organically with as few iterations as possible. This snapshot approach can also enable you to have an experiment-driven approach to code quality and make your development team the source of code quality innovation. Code quality as you might have guessed will not be achieved by imposing rules and tools to developers but by making them decision makers to code quality improvements. Last but not least this approach enables one to have a more direct view of the work done and create more satisfaction across the team.

If we were to take an example we could mention that collecting metrics right before and right after an operation of refactoring will immediately show the impact of refactoring (speed, performance) and its short term impact (usage, resource consumption, business benefit). Same with legacy code quality improvement projects. The improvement of key code quality metrics should always be clear and simple enough to show non technical members the effect of your tech team work. It should also act as a control center and generate warnings in the case where flaws have been introduced.

  1. What aspects of the code

Even though we defined in our first code quality article that the 5 factors coming into the equation when it comes to code quality were efficiency, refactorization, test, documentation and security there are not necessarily metrics which strictly fit each of these boxes. That is why we’ll present you with a practical approach based on metrics available to developers that tries to cover all these characteristics.

  • Monitoring defects: the number and the nature of defects found in software can help you identify which effort to reduce them is the most effective
  • Monitoring efficiency
  • Monitoring maintainability
  • Monitoring readability and formatting: comprehensible code is key to quick collaboration between different teams or different team members
  • Monitoring documentation: All the information surrounding the code are precious assets that need just as much care as your code does
  • Quantitative metrics: such measures usually state the complexity of the code in numeric figures. The most famous matrices are Halstead Complexity Measures, Weighted Micro Function Points, and Cyclomatic Complexity.

(skip to How to measure and Metrics to get a deeper dive into these concepts)

With systematic code quality control and ambitious targets you can eradicate major flaws and significantly filter minor bugs and regressions.
As you build your code quality processes and your monitoring dashboard you should keep in mind that consistency is key to success. Measuring code quality all the time and everywhere you can with an industrial software manufacturing mindset is the most efficient approach to this.

How to measure quality ?

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

Bill Gates

When it comes to measuring code quality, we often misunderstand the quantity of quality processes at the expense of the quality of these. For example how well tested will be controlled by the quantity of tests, and the documentation by the quantity of documentation written. As if an efficient crash test for car manufacturers depended on how many times they crashed the car against the wall. Measuring how much we do without monitoring how well we do means that we reach high scores on the metrics we have set but still launch defective markets on the product. This is exactly what is happening in the software industry. Companies who miss the code quality monitoring target create a false sense of satisfaction which only generates more frustration for all and more risk for the business.

Firms are avoiding the responsibility of code quality and creating a deceiving relationship with metrics through these quantity based quality metrics thus missing the point of monitoring code quality in the first place.

According to a Forrester & Tricentis survey from 2018 only 18% of the non Agile Devops specific decision makers surveyed thought that their business risk was covered completely in all phases of testing, 8% thought their test suite always gave them an accurate indication of their business risk. We can legitimately wonder how come, in the era of all analytics, we are seeing such discrepancies between code quality data and the observed amount of bugs and regressions in production.We have not been able to connect the dots efficiently yet.

ISO 25010 formerly known as ISO 9126 and ISE/IEC 15939

One of the most advanced matrices to perform software quality analysis was set up by the standard ISO 9126 which later evolved into the current ISO 25010.

It’s originally a Factor-Criteria-Metric model composed of high level factors of quality drilled down into criterias. This standard defines software quality as "the totality of features and characteristics of a software product that bear on its ability to satisfy stated or implied needs."

The model has 8 factors subdivided into sub characteristics. 

• Functional 

• Suitability 

• Maintainability 

• Portability 

• Compatibility 

• Reliability 

• Security 

• Performance 

• Efficiency 

• Usability  

McCall software quality model (1977)

The ISO model offers an exhaustive list of code quality characteristics to monitor but, and this says a lot regarding industrial grade code quality standards, does not give any metrics nor acceptable level for metrics to guide its users. It is left to each company to make its own decisions regarding how to apply the model to their software production.

Moreover software quality is a bigger scale than code quality. Beware not to mix both. Your code quality and product quality strategies must be coherent but they don’t rely on the same things. Code quality is about code best practices, it serves the product qualities.

So let’s see before reviewing some metrics and their limitations how you should challenge your development team regarding each dimension of code quality and state what code quality metrics should inform you on.

Maintainability - How well your code reacts to change

Bug free software is a debated target. Most developers believe that the intresinquent nature of software development necessarily involves bugs. In order to develop sustainable software you need to make sure that if your code has defects, they are easily removed and the code can be manipulated and improved by any capable developer in your team at any point in time. The average speed to fix these bugs should give you a clue of how maintainable your code is.

Efficiency - How well your code processes

Any software program runs using computing hardware resources. Efficient code uses the minimum possible hardware resources that it needs to get the intended task done. This will reduce completion time and resources consumption Programs using more hardware resources that it requires will deteriorate your profitability.

Readability - How understandable your code is

Good quality code should be easy to read even for beginner programmers. The formatting and spacing used while writing programs should follow the standards of the programming language used. You can control that those aspects are manageable with the right KPIs. You should understand what the code does by taking a look at it. This comes particularly handy in collaborations so developers can understand code they have not written.

Extensibility - How future proof is your code

Extensibility in coding refers to the level of flexibility of adding more features in your program. Good quality code should make adding new features to the program easy and shouldn’t require making a lot of modifications for the code to run properly.   

We established the significance of measuring and monitoring code quality. Now let’s continue to the next step: discovering what makes a good metric and the tools to start your own code quality metric journey


The Developer Coefficient-Stripe Harris

Métriques et critères d'évaluation de la qualité du code source d’un logiciel-Pierre Mengal

Comparison of the SIG Maintainability Model and the Maintainability Index - Frank R. Oppedijk

Sustainable software development

2018 Google Adwords Survey France

Also on the hub

Ready to write beautiful code?

Smart and simple unit testing assistant. Now available for free.

Try it now

Solutions for JS, TS, Java and Python

Lines Footer
Flexing Unicorn by Ponicode