What do we mean by Testing Maturity?

Organisations aim to continuously improve in order to maintain competitive advantages and drive profitability. The challenge they face is knowing how successful they are at improving.

To address this challenge, companies look externally to benchmark against their competition and to identify what has been missed. There are many approaches to this: some look at standards such as ISO9001; others will look at market analysis such as Gartner Magic Quadrants; and some will focus on internal processes such as Capability Maturity Model Integration (CMMI).

What does ‘good’ testing look like?

The challenge with these approaches is that they don’t focus on testing processes. To identify key areas of improvement it is necessary to get into the detail of how each step of testing is executed. Test process improvement efforts should focus on the needs of the organisation within the context of the business environment. Some organisations will prioritise lower costs, whilst others will want to mitigate risk or reduce time to market through testing.

This means that what good testing looks like can differ greatly between two organisations, even within the same industry.

To accommodate this variability, we use frameworks known as maturity models. Organisations are audited against these models to gauge the level at which they are currently working and to identify what behaviour they need to change in order to progress to the higher level that will deliver the desired business benefits.

Levels of Maturity

Even a very young company can have mature testing processes. The companies that are most able to improve are those that are capable of learning quickly.

Maturity is scored in numbered levels that are baselined against what testing behaviours are being demonstrated. The scale of scoring can vary between models; here, we will consider five levels of behaviours and what they look like in testing teams.

Lowest: Ad hoc test processes

• At the first maturity level, testing activity either does not happen or is purely exploratory. Defects are found almost by accident and delivery teams may not have specialist or defined roles
• There is no understanding of testing risk, quality and costs. Any documentation is limited and not effective
• Maturity Marker: no testing responsibilities in the organisation

Starting: Basic test processes

• The organisation has awareness of testing risk, quality and costs. The building blocks of basic testing are in place with test plans, scripting, test execution and reporting
• A testing policy or strategy shows the intent to have effective testing but the organisation has not developed or executed the approach
• Maturity Marker: a Testing Strategy that nobody has read

Building: Some processes defined

• Testing activities are more organised, with test team members having defined roles and accountabilities. There are repeatable and documented processes, though they are not always effective and are rarely efficient
• The organisation is measuring and reporting testing risk, quality and costs. This is the level at which many testing teams find themselves — and then struggle to improve further in the absence of a deeper assessment of their testing processes and behaviours
• Maturity Marker: defects leaking between delivery phases

Improving: Efficient test processes

• A testing leadership structure is place with a measured plan for improvement. There are controlled testing risks, quality and costs with regular reviews and lessons learned
• The team has the capacity to make process improvements and be responsive to the needs of the business. Team training is prevalent and the organisation has an established testing methodology
• Maturity Marker: risk-based test planning approach

Learning: Continuously adapting testing

• The highest level of testing maturity sees reduced testing risk and optimised quality and costs. Improvement has become preventative rather than reactive and quality is being actively controlled throughout the delivery lifecycle
• The team has a culture of continuous improvement and a knowledge-sharing community to remove any repetition of poor practice and to innovate better ways of working
• Maturity Marker: lower costs of delivery