Workflow Detail: Verify Test Approach

Description

The objective is to gain an understanding of the constraints and limitations of each technique as it will be applied in the given project context, and to either:

• find an appropriate implementation solution for each technique or
• find alternative techniques that can be used.

This helps to mitigate the risk of discovering too late in the project lifecycle that the test approach is unworkable.
For each iteration, this work is focused mainly on:

• Early verification that the intended test strategy will work and produces results of value
• Establishing the basic infrastructure to enable and support the test strategy
• Obtaining commitment from the development team to develop the software to meet testability requirements necessary to achieve the test strategy, and to provide continued support for those testability requirements.
• Identifying the scope, boundaries, limitations and constraints of each technique

Related Information

This section provides links to additional information related to this workflow detail.

• Concept: Levels of Test
• Concept: Test Automation and Tools
• Concept: Test Strategy
• Concept: Types of Test

Timing

Starts early on each in each iteration, as soon as sufficient agreement is reached on the mission for the iteration, and continues as needed throughout the iteration. More frequently addressed in the earlier phases of Inception, Elaboration and early Construction, typically tapering off in late Construction and Transition.

Optionality

Considered optional when the test approach is well known, and its applicability in the current context is well established.

How to Staff

Although most of the roles involved in the Test discipline play a part in performing this work, the effort is primarily centered around the Test Designer and Tester roles. The most important skill areas required for this work include software architecture, software design and problem solving.

It is typical for this work to require more resource in iterations from the late Inception to early Construction phases, often requiring minimal resource late in the Construction and in the Transition phases. However, be aware that as the project progresses, new objectives or deliverables may be identified that require new test strategies to be defined and verified.

As a heuristic for relative resource allocation by phase, typical percentages of test resource use for this workflow detail are: Inception - 30%, Elaboration - 20%, Construction - 10% and Transition - 05%.

Work Guidelines

This work is somewhat independent of the test cycles, often involving the verification of techniques that will not be used until subsequent Iterations. This work normally begins after the evaluation mission has been defined for the current Iteration, although it can begin earlier. In some cases, finding the best implementation approach to a technique may take multiple Iterations.

The test implementation and execution activities that form a part of this work are performed for the purpose of obtaining demonstrable proof that the techniques being verified can actually work. As such, you should limit your selection of tests to a small, representative subset; typically focusing on areas with substantial quality risk. You should try to include a selection of tests that you expect to fail to confirm that the technique will successfully detect these failures.

While failures with the target test items will be identified and these incidents logged accordingly, this focus of this work is not directly on attempting to identify failures in the target test items as it's main objective. Again, the objective is to verify that the approach is appropriate (it produces results that complement the Iteration objectives), is achievable (it can be implemented with given resource constraints), and that it will work.

For this work to produce timely results, it is often necessary to make use of incomplete, "unofficial" Builds, or to perform this work outside of a recognized Test Environment Configuration. Although these are appropriate compromises, be aware of the constraints, assumptions and risks involved in verifying your approach in under these conditions.

As the lifecycle progresses through its Phases, the focus of the test effort typically changes. Potentially this requires new or additional approaches, often requiring the introduction of new types of tests or new techniques to support the test effort.

In situations where the combination of domain, the test environment and other critical aspects of the strategy are unprecedented, you should allow more time and effort to complete this work. In some cases-especially where automation is a requirement-it may be more economic to obtain the use of resources with specialized skills that have proven experience in the unprecedented aspects of the strategy for a limited period of time (such as on contract) to define and verify the key technical needs of the test strategy.

See the Related Information section for additional guidance that will help you in performing this work.

Rational Unified Process