Guidelines: Quality in the
Test Plan
A Dynamic View of Good Enough 
Let's look more closely at the view of Good Enough Quality (GEQ) that sees quality as a dynamic tradeoff. The concept goes like this: The quality of any product lies somewhere between terrible and ideal. Terrible quality is much less expensive to produce than ideal quality (how expensive is a blank disk?), and much less valuable. A product is good enough when it has enough value without costing too much. Exactly how good that is changes throughout the project, depending on business conditions and other factors. "Cost" in this case might mean actual money or something that substitutes for money, such as time, materials, or staff. "Value" means value in the opinion of someone who matters. What one person considers sufficiently valuable, another person may not value at all, so the analysis of quality always begins with identifying the people who get to decide how good is good.
In a general sense, Tradeoff GEQ is an ancient idea. It's part of basic economics and engineering and, therefore, it's embedded in the very DNA of any business that involves engineering. What's new about the tradeoff idea is the public admission that, practically speaking, we don't have a choice about releasing a product that has problems. The product will have problems, whether or not we know about them. However, if we work at it, we might be able to choose the kind of problems that we ship with. Let's ship with the right bugs, instead of the wrong ones. In 1996, James Bach created a heuristic model of good enough tradeoffs, based on his experiences at Borland International and Apple Computer. Bach developed the model to provide a set of talking points for successfully arguing that a product was not good enough. Originally, the model was used to persuade management to allow incremental improvements of products prior to release. However, the model also proved useful to motivate process improvement, and has been used in court cases to attack or defend the quality of software.
Let's go back to how a manager or CEO might argue, under deadline pressure. "Perfection would be nice, but we have to be practical. We're running a business. Quality is good, but not quality at any cost. As you know, all software has bugs." A Tradeoff GEQ advocate will agree that practicality is vital to the business and agree that the product will have problems no matter when it's shipped. But he will also go on to argue (if the situation merits it) that the manager should not yet think that the product has reached a point where it would be practical to release. A GEQ argument is based on specific risks faced or specific benefits missing. It may also be based on a concern that there is not enough information on which to base a responsible decision about quality. GEQ directs attention to satisfiable (though not necessarily quantifiable) concerns that encompass economic factors as well as quality factors.
Low quality might be good enough. High quality might not be good enough. In the dynamic view, good enough is whatever satisfies these four criteria:
- It has sufficient benefits.
- It has no critical problems.
- The benefits sufficiently outweigh the problems.
- In the present situation, and all things considered, further improvement would be more harmful than helpful.
Each point is critical. If any one of them is not satisfied, then the product, although perhaps good, cannot be good enough. The first two seem fairly obvious, but notice that they are not exact opposites of each other. The complete absence of problems cannot guarantee infinite benefits, nor can infinite benefits guarantee the absence of problems. Benefits and problems do offset each other, but it's important to consider the product from both perspectives.
The third criterion reminds us that benefits must not merely outweigh problems, they must do so to a sufficient degree. In a medical device, for instance, we may want a large margin of safety. This criterion also reminds us that even in the absence of any individual critical problem, there may be patterns of non-critical problems that more than negate the benefits of the product.
The fourth criterion introduces the important matter of logistics and side effects. If high quality is too expensive to achieve, or if achieving it would cause other unacceptable problems, then we either have to accept lower quality as being good enough or we have to accept that a good enough product is impossible.
These criteria form the basis of a case that a product is or is not good enough, but we can do better.
