Concepts: Levels
of Test
Testing is applied to different types of targets, in different stages or levels
of work effort. These levels are distinguished typically by those roles that
are best skilled to design and conduct the tests, and where techniques are most
appropriate for testing at each level. It's important to ensure a balance of
focus is retained across these different work efforts.
Developer Testing
Developer testing denotes the aspects of test design and implementation most
appropriate for the team of developers to undertake. This is in contrast to
Independent Testing. In most cases, test execution initially occurs with the
developer testing group who designed and implemented the test, but it is a good
practice for the developers to create their tests in such a way so as to make
them available to independent testing groups for execution.
Traditionally, developer testing has been considered mainly with respect to
unit testing. While some developers also perform varying levels integration
testing, this is largely dependent on culture and other context issues. We recommend
that developer testing should cover more than just testing independent units
in isolation.
Independent Testing
Independent testing denotes the test design and implementation most appropriately
performed by someone who is independent from the team of developers. You can
consider this distinction a superset, which includes Independent Verification
& Validation. In most cases, test execution initially occurs with the independent
testing group that designed and implemented the test, but the independent testers
should create their tests to make them available to the developer testing groups
for execution. Boris Beizer gives the following explanation of the different
objective that independent testing has over developer testing:
"The purpose of independent testing is to provide a different perspective
and, therefore, different tests; furthermore to conduct those tests in a richer
[...] environment than is possible for the developer." [BEI95]
Independent Stakeholder Testing
An alternate view of independent testing is that it represents testing that
is based on the needs and concerns of various stakeholders. Therefore it's referred
to as Stakeholder Testing. This is an important distinction-it helps to
include a broader set of stakeholder concerns than might traditionally be considered,
extending the somewhat generic "customer" with stakeholders such as
technical support staff, technical trainers, sales staff in additional to customers,
and end users.
As a final comment, XP''s
notion of customer tests relates to this categorization of independent
testing in the RUP.
Unit Testing
Unit testing focuses on verifying the smallest testable elements of the software.
Typically unit testing is applied to components represented in the implementation
model to verify that control flows and data flows are covered, and that they
function as expected. The Implementer performs unit testing as the unit is developed.
The details of unit testing are described in the Implementation discipline.
Integration Testing
Integration testing is performed to ensure that the components in the implementation
model operate properly when combined to execute a use case. The target-of-test
is a package or a set of packages in the implementation model. Often the packages
being combined come from different development organizations. Integration testing
exposes incompleteness or mistakes in the package's interface specifications.
In some cases, the assumption by developers is that other groups such as independent
testers will perform integration tests. This situation presents risks to the
software project and ultimately the software quality because:
- integration areas are a common point of software failure.
- integration tests performed by independent testers typically use black-box
techniques and are typically dealing with larger software components.
A better approach is to consider integration testing the repsonsibility of
both developer and independent testers, but make the strategy of each teams
testing effort does not overlap significantly. The exact nature of that overlap
is based on the needs of the individual project. We recommend you foster an
environment where developers and independent system testers share a single vision
of quality. See Concepts: Developer Testing for
additional information.
System Testing
Traditionally system testing is done when the software is functioning as a
whole. An iterative lifecycle allows system testing to occur much earlier-as
soon as well-formed subsets of the use-case behavior are implemented. Usually
the target is the system's end-to-end functioning elements.
Acceptance Testing
User acceptance testing is the final test action taken before deploying
the software. The goal of acceptance testing is to verify that the software
is ready, and that it can be used by end users to perform those functions and
tasks for which the software was built. See Concepts:
Acceptance Testing for additional information.
There are other notions of acceptance testing, which are generally characterized
by a hand-off from one group or one team to another. For example, a build
acceptance test is the testing done to accept the hand-over of a new software
build from development into independent testing.
A comment about sequence and timing of test levels
Traditionally, unit testing is thought of as being implemented early in the
iteration as the first stage of testing: all units required to be passed before
subsequent stages are conducted. However, in an iterative development process,
this approach is as a general rule, inappropriate. A better approach is to identify
the unit, integration and system tests that offer most potential for finding
errors, then implement and execute them based on a combination of greatest risk
and supporting environement.
|