What is Quality?
We all know quality. Something that doesn’t break 5 minutes after you start using it right? You call it ‘a quality piece of kit’. You paid a lot of money to get that particular one. That’s why you call it quality.
We’ll that’s your opinion. My £3,000 laptop is much better quality that your £200 one. OK, so I only use it to check Gmail, but the quality is there.
“Quality” is hard to quantify, in most cases it is subjective, it is a perception. In the delivery of IT solutions we make it even harder, we try to assess the the quality not only of the outputs, but also the delivery process.
The most effective measure of quality is Total Cost of Ownership (TCO). This considers factors beyond delivery costs to include opportunity and support costs. This makes our two laptops described above much more comparable. The £3,000 laptop could be considered better quality if it lasts 20 times as long for example.
Or is that better Value?
One popular definition of Quality comes from a discussion with Jerry Weinberg that states:
“QUALITY IS VALUE TO SOME PERSON AT SOME TIME”*
Put simply this means it is the effectiveness of an output that is relevant to a stakeholder. i.e. it is what you want when you need it. It has value to you.
What then is Value?
Value is again about deriving a monetary amount for something, much like TCO. Dictionary definitions centre around money when defining value.
a thing that is of great worth
worth a great deal of money
Everything, it is said, has a price. But we can take a wider view of value to mean:
Value is the delivery of Efficiency (e.g. R.O.I), Agility (e.g. Operations) and Predictability (e.g. Estimates).
These can each be be defined many ways but from an IT delivery point of view it can be helpful to consider them in terms of how they are used for Agile Delivery. In the process of Agile backlog grooming reviews user stories and prioritizes them for their value. An effective Agile project must optimise the delivery of value.
Given this, we should consider the definitions of the above as:
Efficiency: How easy is it to realise the benefits of the change. This is not necessarily the cheapest to implement, rather what has the best and quickest return on investment relative to the requirements of the user.
Agility: How robust is the change relative to the solution, is it easily maintainable? Can it be easily adapted in the future? Is removal low cost?
Predictability: Are the delivery and outputs transparent, can the effort for delivery be easily and accurately estimated? Are the benefits to be realised relatively certain, or is there a large element of risk?
What is significant here is that Price / Cost is not the main measure for any of these. If the return on investment justifies the expenditure then cost should not be a barrier. This is counter-intuitive to many business cultures and programme plans, where quick wins are often prioritized over optimized user benefit. It is also where many agile projects fail; For example by the underestimation of technical debt.
But what has this got to do with Testing?
In waterfall delivery testing activity is often simplified to the verification of the outputs. Does the solution do what it should, both functionally and non-functionally? This has been the assumed logic for many years across all delivery methodologies. The testing team give a pass or fail, or better a recommendation, for the changes to be deployed to a production environment.
That’s not what testing is. Testing is about giving feedback. it is about telling the delivery team about issues, defects and risks as soon as possible so it is easier and cheaper to fix. Testing is about enabling the users to maximise the realisation of the benefits of change. It is about optimising the delivery of value.
Feedback, including testing results, can be given at any stage in the project from story refinement, through to UAT and Beta Testing. You may have noticed the emergence of the QA Analyst role in recent years. In some areas of IT and recruitment this is used interchangeably with Test Analyst positions. Really these both distinct roles.
A Test Analyst, in the traditional sense, is a specialist software tester. An expert in interrogating systems and requirements to identify risks. Most often deployed during a focused testing stage.
A QA Analyst has a much wider focus. They are providing assurance of the delivery structure, with more focus on the delivery of inputs, process improvements and the quality of reporting. Both roles play in important part in identifying risks, but are very different skill sets to recruit.
What Does this Tell us about Testing (or Assurance) of Quality?
You can have tested software, but this does not mean that is it of high quality or value. We have all seen fantastic, defect free products delivered too late to take advantage of user needs.
Whilst there is some moral satisfaction of delivering defect free, and that should always be the ambition, there should be more benefit from delivering higher value solutions.
It is not enough to test the outputs, testing should be present at every stage of delivery. This is what testing professionals mean when they recommend ‘shift left on testing’. Testing teams should be held accountable for verifying the value delivery. They have a moral responsibility to go beyond validation of requirements and be a proxy for users. Testers must assure quality, checking for Efficiency, Agility and Predictability. You could go as far as to describe this as governance.
So, What was Quality again?
Quality is the delivery of user value and that is assured throughout delivery and the lifetime of the outputs.
It is not just the result of testing and it is not the outcome from delivering the cheapest or most expensive solution.
It is delivering what the user wants.
Best wishes and happy collaborating
Andrew
Andrew Palmer CITP is Senior Test Manager at Mastek
He has been coaching agile teams since 2008 and is an advocate for continuous improvement in the workplace.
References:
*http://www.shino.de/2010/07/22/quality-is-value-to-some-person-at-some-time/
