Guidelines: Equivalence Class Analysis

Topics

Introduction
Strategies
Further Reading and References

Introduction

Except for the most trivial of software applications, it is generally considered impossible to test all the input combinations logically feasible for a software system. Therefore, selecting a good subset that has the highest probability of finding the most errors, is a worthwhile and important task for testers to undertake.

Testing based on equivalence class analysis (synonyms: equivalence partitioning, domain analysis) is a form of black-box test analysis that attempts to reduce the total number of potential tests to a minimal set of tests that will uncover as many errors as possible [MYE79]. It is a method that partitions the set of inputs and outputs into a finite number of equivalence classes that enable the selection of a representative test value for each class. The test that results from the representative value for a class is said to be "equivalent" to the other values in the same class. If no errors were found in the test of the representative value, it is reasoned that all the other "equivalent" values wouldn't identify any errors either.

The power of Equivalence Classes lies in their ability to guide the tester using a sampling strategy to reduce the combinatorial explosion of potentially necessary tests. The technique provides a logical bases by which a subset of the total conceivable number of tests can be selected. Here are some categories of problem areas for large numbers of tests that can be benefit from the consideration of equivalence classes:

Combinations of independent variables
Dependent variables based on hierarchical relationship
Dependent variables based on temporal relationship
Clustered relationships based on market exemplars
Complex relationships that can be modeled

Strategies

There are different strategies and techniques that can be used in equivalence partition testing. Here are some examples:

Equivalence Class Partition

Equivalence partition theory as proposed by Glenford Myers [MYE79]. attempts to reduce the total number of test cases necessary by partitioning the input conditions into a finite number of equivalence classes. Two types of equivalence classes are classified: the set of valid inputs to the program is regarded as the valid equivalence class, and all other inputs are included in the invalid equivalence class.

Here are a set of guidelines to identify equivalence classes:

If an input condition specifies a range of values (such as, program "accepts values from 10 to 100"), then one valid equivalence class (from 10 to 100) and two invalid equivalence classes are identified (less than 10 and greater than 100).
If an input condition specifies a set of values (such as, "cloth can be many colors: RED, WHITE, BLACK, GREEN, BROWN "), then one valid equivalence class (the valid values) and one invalid equivalence class (all the other invalid values) are identified. Each value of the valid equivalence class should be handled distinctly.
If the input condition is specified as a "must be" situation (such as, "the input string must be upper case"), then one valid equivalence class (uppercase characters) and one invalid equivalence (all the other input except uppercase characters) class are identified.
Everything finished "long" before the task is done is an equivalence class. Everything done within some short time interval before the program is finished is another class. Everything done just before program starts another operation is another class.
If a program is specified to work with memory size from 64M to 256M. Then this size range is an equivalence class. Any other memory size, which is greater than 256M or less than 64M, can be accepted.
The partition of output event lies in the inputs of the program. Even though different input equivalence classes could have same type of output event, you should still treat the input equivalence classes distinctly.

Boundary Value Analysis

In each of the equivalence classes, the boundary conditions are considered to have a higher rate of success identifying resulting failures than non-boundary conditions. Boundary conditions are the values at, immediately above or below the boundary or "edges" of each equivalence classes.

Tests that result from boundary conditions make use of values at the minimum (min), just above minimum (min+), just below the maximum (max-), and the maximum (max) of the range that needs be tested. When testing boundary values, testers choose a few test cases for each equivalence class. For the relatively small sample of tests the likelihood of failure discovery is high. The Tester is given some relief from the burden of testing a huge population of cases in an equivalent class of values that are unlikely to produce large differences in testing results.

Some recommendations when choosing boundary values:

For a floating variable, if the valid condition of it is from -1.0 to 1.0, test -1.0, 1.0, -1.001 and 1.001.
For an integer, if the valid range of input is 10 to 100, test 9, 10, 100, 101.
If a program expects an uppercase letter, test the boundary A and Z. Test @ and [ too, because in ASCII code, @ is just below A and [ is just beyond the Z.
If the input or output of a program is an ordered set, pay attention on the first and the last element of the set.
If the sum of the inputs must be a specific number (n), test the program where the sum is n-1, n, or n+1.
If the program accepts a list, test values in the list. All the other values are invalid.
When reading from or writing to a file, check the first and last characters in the file.
The smallest denomination of money is one cent or equivalent. If the program accepts a specific range, from a to b, test a -0.01 and b+0.01.
For a variable with multiple ranges, each range is an equivalence class. If the sub-ranges are not overlapped, test the values on the boundaries, beyond the upper boundary, and below the lower boundary.

Special Values

After attempting the two previous boundary analysis strategies, an experienced tester will observe the program inputs to discovery any "special value" cases, which are again potentially rich sources for uncovering software failures. Here are some examples:

For an integer type, zero should always be tested if it is in the valid equivalence class.
When testing time (hour, minute and second), 59 and 0 should always be tested as the upper and lower bound for each field, no matter what constraint the input variable has. Thus, except the boundary values of the input, -1, 0, 59 and 60 should always be test cases.
When testing date (year, month and day), several test cases, such as number of days in a specific month, the number of days in February in leap year, the number of days in the non-leap year, should be involved.

"Category-Partition" Method

Ostrand and Balcer [16] developed a partition method that helps testers to analyze the system specification, write test scripts, and manage them. Different from common strategies that mostly focuses on the code, their method is based on the specification and design information too.

The main benefit of this method is its ability to expose errors before the code has been written because the input source is the specification and the tests result from the analysis of that specification. Faults in the specifications will be discovered early, often well before they are implemented in code.

The strategy for the "category-partition" method follows:

Analyze the specification: decompose the system functionality into functional units, which can be tested independently both by specification and implementation.
From there;
1. Identify the parameters and the environment conditions that will influence the function's execution. Parameters are the inputs of the function unit. Environment conditions are the system states, which will effect the execution of the function unit.
2. Identify the characteristics of the parameters and the environment conditions.
3. Classify the characteristics into categories, which effect the behavior of the system.
Ambiguous, contradictory, and missing descriptions of behavior will be discovered in this stage.
Partition the categories into choices: Choices are the different possible situations that might occur and not be expected. They represent the same type of information in a category.
Determine the relations and the constraints among choices. The choices in different categories influence with each other, which also have an influence of building the test suite. Constraints are added to eliminate the contradiction of between choices of different parameters and environments.
Design test cases according to the categories, choices and constraint information. If a choice causes an error, don't combine it with other choices to create the test case. If a choice can be "adequately" tested by one single test, it is either the representative of the choice or a special value.