Best Practice: Continuously Verify Quality

Software problems are 100 to 1000 times more costly to find and repair after deployment. Verifying and managing quality throughout the project's lifecycle is essential to achieving the right objectives at the right time.

Topics

• What Do We Mean by Quality Verification Throughout the Lifecycle?
• What Is Quality?
  • Introduction
  • Definition of Quality
  • Who Owns Quality?
  • Common Misconceptions about Quality
  • Management of Quality in the RUP

What Do We Mean by Quality Verification Throughout the Lifecycle?

It's important that the quality of all artifacts are assessed at several points in the project's lifecycle as they mature. Artifacts should be evaluated as the activities that produce them complete and at the conclusion of each iteration. In particular, as executable software is produced, it should be subjected to demonstration and test of important scenarios in each iteration, which provides a more tangible understanding of design trade-offs and earlier elimination of architectural defects. This is in contrast to a more traditional approach that leaves the testing of integrated software until late in the project's lifecycle.

What Is Quality?

Introduction

Quality is something we all strive for in our products, processes, and services. Yet when asked, "What is Quality?", everyone has a different opinion. Common responses include one or the other of these:

• "Quality ... I'm not sure how to describe it, but I'll know it when I see it."
• "... meeting requirements."

Perhaps the most frequent reference to quality, specifically related to software, is this remark regarding its absence:

"How could they release something like this with such low quality!?"

These commonplace responses are telling, but they offer little room to rigorously examine quality and improve upon its execution. These comments all illustrate the need to define quality in a manner in which it can be measured and achieved.

Quality, however, is not a singular characteristic or attribute. It's multi-dimensional and can be possessed by a product or a process. Product quality is concentrated on building the right product, whereas process quality is focused on building the product correctly. See Concepts: Product Quality and Concepts: Process Quality for additional information.

Definition of Quality

The definition of quality, taken from The American Heritage Dictionary of the English Language, 3rd Edition, Houghton Mifflin Co.,© 1992, 1996, is:

Quality (kwol'i-te) n., pl. -ties. Abbr. qlty. 1.a. An inherent or distinguishing characteristic; a property. b. A personal trait, especially a character trait. 2. Essential character; nature. 3.a. Superiority of kind. b. Degree or grade of excellence.

As demonstrated by this definition, quality is not a single dimension, but many. To use the definition and apply it to software development, the definition must be refined. Therefore, for the purposes of the Rational Unified Process (RUP), quality is defined as:

"...the characteristic of having demonstrated the achievement of producing a product that meets or exceeds agreed-on requirements-as measured by agreed-on measures and criteria-and that is produced by an agreed-on process."

Achieving quality is not simply "meeting requirements", or producing a product that meets user needs and expectations. Rather, quality also includes identifying the measures and criteria to demonstrate the achievement of quality, and the implementation of a process to ensure that the product created by the process has achieved the desired degree of quality, and can be repeated and managed.

See also the following pages for additional information on how the RUP defines the idea of quality:

• Concept: Product Quality
• Concept: Process Quality
• Concept: Measuring Quality
• Concept: Evaluating Quality

Who Owns Quality?

A common misconception is that quality is owned by, or is the responsibility of, one group. This myth is often perpetuated by creating a group, sometimes called Quality Assurance-other names include Test, Quality Control, and Quality Engineering-and giving them the charter and the responsibility for quality.

Quality is, and should be, the responsibility of everyone. Achieving quality must be integral to almost all process activities, instead of a separate discipline, thereby making everyone responsible for the quality of the products (or artifacts) they produce and for the implementation of the process in which they are involved.

Each role contributes to the achievement of quality in the following ways:

• Product quality-the contribution to the overall achievement of quality in each artifact being produced.
• Process quality-the achievement of quality in the process activities for which they are involved.

Everyone shares in the responsibility and glory for achieving a high-quality product, or in the shame of a low-quality product. But only those directly involved in a specific process component are responsible for the glory, or shame, for the quality of those process components (and the artifacts). Someone, however, must take the responsibility for managing quality; that is, providing the supervision to ensure that quality is being managed, measured, and achieved. The role responsible for managing quality is the Project Manager.

Common Misconceptions about Quality

There are many misconceptions regarding quality and the most common include:

• Quality can be added to or "tested" into a product
• Quality is a single dimension, attribute, or characteristic and means the same thing to everyone
• Quality happens on its own

Quality can be added to or "tested" into a product

Just as a product cannot be produced if there is no description of what it is, what it needs to do, who uses it and how it's used, and so on, quality and its achievement cannot be attained if it's not described, measured, and part of the process of creating the product.

See Concepts: Measuring Quality and the section of this document titled Quality happens on its own.

Quality is a single dimension, attribute, or characteristic and means the same thing to everyone

Quality is not a single dimension, attribute, or characteristic. Quality is measured in many ways-quality metrics and criteria are established to meet the needs of project, organization, and customer.

Quality can be measured along several dimensions-some apply to process quality; some to product quality; some to both. Quality can be measured for:

• Progress-such as use cases demonstrated or milestones completed
• Variance-differences between planned and actual schedules, budgets, staffing requirements, and so forth
• Reliability-resistance to failure (crashing, hanging, memory leaks, and so on) during execution
• Function-the artifact implements and executes the required use cases as intended
• Performance-the artifact executes and responds in a timely and acceptable manner, and continues to perform acceptably when subjected to real-world operational characteristics such as load, stress, and lengthy periods of operation

See Concepts: Quality Dimensions, Concepts: Product Quality, and Concepts: Process Quality for additional information.

Quality happens on its own

Quality cannot happen by itself. For quality to be achieved, a process is must be implemented, adhered to, and measured. The purpose of the RUP is to provide a disciplined approach to assigning tasks and responsibilities within a development organization. Our goal is to ensure the production of high-quality software that meets the needs of our end users, within a predictable schedule and budget. The RUP captures many of the best practices in modern software development in a form that can be tailored for a wide range of projects and organizations. The Environment discipline gives you guidance about how to best configure the process to your needs.

Processes can be configured and quality-criteria for acceptability-can be negotiated, based upon several factors. The most common factors are:

• Risk (including liability)
• Market opportunities
• Revenue requirements
• Staffing or scheduling issues
• Budgets

Changes in the process and criteria for acceptability should be identified and agreed upon at the outset of the project.

Management of Quality in the RUP

Managing quality is done for these purposes:

• To identify appropriate indicators (metrics) of acceptable quality
• To identify appropriate measures to be used in evaluating and assessing quality
• To identify and appropriately address issues affecting quality as early and effectively as possible

Managing quality is implemented throughout all disciplines, workflows, phases, and iterations in the RUP. In general, managing quality throughout the lifecycle means you implement, measure, and assess both process quality and product quality. Some of the efforts expended to manage quality in each discipline are highlighted in the following list:

• Managing quality in the Requirements discipline includes analyzing the requirements artifact set for consistency (between artifact standards and other artifacts), clarity (clearly communicates information to all shareholders, stakeholders, and other roles), and precision (the appropriate level of detail and accuracy).
• In the Analysis & Design discipline, managing quality includes assessing the design artifact set, including the consistency of the design model, its translation from the requirements artifacts, and its translation into the implementation artifacts.
• In the Implementation discipline, managing quality includes assessing the implementation artifacts and evaluating the source code or executable artifacts against the appropriate requirements, design, and test artifacts.
• The Test discipline is highly focused toward managing quality, as most of the efforts expended in this discipline address the three purposes of managing quality, identified previously.
• The Environment discipline, like the Test discipline, includes many efforts addressing the purposes of managing quality. Here you can find guidance on how to best configure your process to meet your needs.
• Managing quality in the Deployment discipline includes assessing the implementation and deployment artifacts, and evaluating the executable and deployment artifacts against the appropriate requirements, design, and test artifacts needed to deliver the product to your customer.
• The Project Management discipline includes an overview of many efforts for managing quality, including the reviews and audits required to assess the implementation, adherence, and progress of the development process.

Rational Unified Process