Concepts: Acceptance Testing
Acceptance testing is the final test action before deploying the software.
The goal of acceptance testing is to verify that the software is ready and can
be used by your end users to perform those functions and tasks for which the
software was built. There are three common strategies for implementing an acceptance
test. They are:
The strategy you select is often based on the contractual requirements, organizational
and corporate standards, and application domain.
Formal acceptance testing is a highly managed process and is often an extension
of the system test. The tests are planned and designed as carefully as, and
in the same detail as, system testing. The test cases chosen should be a subset
of those performed in system test. It's important not to deviate in any way
from the chosen test cases. In many organizations, formal acceptance testing
is fully automated.
The activities and artifacts are the same as for system testing. In some organizations,
the development organization (or its independent test group), with the representatives
of the end-user organization, performs the acceptance test. In other organizations,
acceptance testing is performed completely by the end-user organization or an
objective group of people chosen by the end-user organization.
The benefits of this form of testing are:
- The functions and features to be tested are known.
- The details of the tests are known and can be measured.
- The tests can be automated, which permits regression testing.
- The progress of the tests can be measured and monitored.
- The acceptability criteria are known.
The disadvantages include:
- Requires significant resources and planning.
- The tests may be a re-implementation of system tests.
- The testing may not uncover subjective defects in the software, since you're
only looking for defects you expect to find.
In informal acceptance testing, the test procedures for performing the test
are not as rigorously defined as for formal acceptance testing. The functions
and business tasks to be explored are identified and documented, but there are
no particular test cases to follow. The individual tester determines what to do.
This approach to acceptance testing is not as controlled as formal testing and
is more subjective than the formal one.
Informal acceptance testing is most frequently performed by the end-user
organization.
The benefits of this form of testing are:
- The functions and features to be tested are known.
- The progress of the tests can be measured and monitored.
- The acceptability criteria are known.
- You will uncover more subjective defects than with formal acceptance
testing.
The disadvantages include:
- Resources, planning, and management resources are required.
- You have no control over what test cases are used.
- End users may conform to the way the system works and not see the defects.
- End users might focus on comparing the new system to a legacy system, rather
than looking for defects.
- Resources for acceptance testing are not under the control of the project
and could be constricted.
Beta testing is the least controlled of the three acceptance test strategies.
In beta testing, the amount of detail, the data, and the approach taken is entirely
up to the individual tester. Each tester is responsible for creating his or
her own environment, selecting his or her data, and determining what functions,
features, or tasks to explore. Each tester is responsible for identifying his
or her own criteria for whether to accept the system in its current state or
not.
Beta testing is implemented by end users, often with little or no management
from the development (or other non end-user) organization. Beta testing is the
most subjective of all acceptance test strategies.
The benefits of this form of testing are:
- Testing is implemented by end users.
- There are large volumes of potential test resources.
- There is increased customer satisfaction for those who participate.
- You uncover more subjective defects than with formal or informal acceptance
testing.
The disadvantages include:
- You might not test all functions or features.
- Test progress is difficult to measure.
- End users might conform to the way the system works and not see or report
the defects.
- End users may focus on comparing the new system to a legacy system, rather
than looking for defects.
- Resources for acceptance testing are not under the control of the project
and could be constricted.
- Acceptability criteria are not known.
- You need increased support resources to manage the beta testers.
|