Guidelines: Programming Automated Test Scripts
Topics
To increase the maintainability and reusability of your Test Scripts, they
should have been structured before they are implemented. You will probably find
that there are actions that will appear in several Test Scripts. A goal should
be to identify these actions so that you can reuse their implementation.
For example, you may have Test Scripts that are combinations of different actions
you can perform to a record. These Test Scripts may be combinations of the addition,
modification, and the deletion of a record:
- Add, Modify, Delete (the obvious one)
- Add, Delete, Modify
- Add, Delete, Add, Delete, ...
- Add, Add, Add, ...
If you identify and implement these actions as separate Test Scripts and reuse
them in other Test Scripts you will achieve a higher level of reuse.
Another goal would be to structure your Test Scripts in such a way that a change
in the target software causes a localized and controllable change in your Test
Scripts. This will make your Test Scripts more resilient to changes in the target
software. For example, say the log-in portion of the software has changed. For
all test cases that traverses the log-in portion, only the Test Script pertaining
to log-in will have to change.
To achieve higher maintainability of your test scripts, you should record them
in a way that is least vulnerable to changes in the target-of-test. For example,
for a test script that fills in dialog box fields, there are choices for how
to proceed from one field to the next:
- Use the TAB key
- Use the mouse
- Use the keyboard accelerator keys
Of these choices, some are more vulnerable to design changes than others. If
a new field is inserted on the screen the TAB key approach will not be reliable.
If accelerator keys are reassigned, they will not provide a good recording.
If the method that the mouse uses to identify a field is subject to change,
that may not be a reliable method either. However, some test automation tools
have test script recorders that can be instructed to identify the field by a
more reliable method, such as its Object Name assigned by the development tool
(PowerBuilder, SQLWindows, or Visual Basic). In this way, a recorded test script
is not effected by minor changes to the user interface (e.g., layout changes,
field label changes, formatting changes, etc.)
Many Test Scripts involve entering several sets of field data in a given data
entry screen to check field validation functions, error handling, and so on.
The procedural steps are the same; only the data is different. Rather than recording
a Test Script for every set of input data, a single recording should be made
and then modified to handle multiple data sets. For example, all the data sets
that produce the same error because of invalid data can share the same recorded
Test Script. The Test Script is modified to address the data as variable information,
to read the data sets from a file or other external source, and to loop through
all of the relevant data sets.
If Test Scripts or test code have been developed to loop through sets of input
and output data the data sets must be established. The usual format to use for
these data sets is records of comma-separated fields in a text file. This format
is easy to read from Test Scripts and test code, and is easy to create and maintain.
Most database and spreadsheet packages can produce comma-separated textual
output. Using these packages to organize or capture data sets has two important
benefits. First, they provide a more structured environment for entering and
editing the data than simply using a text editor or word processor. Second,
most have the ability to query existing databases and capture the returned data,
allowing an easy way to extract data sets from existing sources.
The recorded Test Script is sequential in its execution. There are no branch
points. Robust error handling in the Test Scripts requires additional logic
to respond to error conditions. Decision logic that can be employed when errors
occur includes:
- Branching to a different Test Script.
- Calling a script that attempts to clean up the error condition.
- Exiting the script and starting the next one.
- Exiting the script and the software, restarting, and resuming testing at
the next Test Script after the one that failed.
Each error-handling technique requires program logic added to the Test Script.
As much as possible, this logic should be confined to the high-level Test Scripts
that control the sequencing of lower-level Test Scripts. This allows the lower-level
Test Scripts to be created completely from recording.
When doing stress testing, it is often desirable to synchronize Test Scripts
so that they start at predefined times. Test Scripts can be modified to start
at a particular time by comparing the desired start time with the system time.
In networked systems each test station will share, via the network, the same
clock. In the following example (from a script written in BASIC) statements
have been inserted at the start of a script to suspend the execution of the
script until the required time is reached.
InputFile$ = "\TIME.DAT"
Open InputFile$ For Input As 1
Input #1, StartTime$
Close #1
Do While TimeValue(StartTime$) > Time
DoEvents
Loop
[Start script]
In this example, the required start time is stored in a file. This allows the
start time to be changed without changing the Test Script. The time is read
and stored in a variable called StartTime$ . The Do While loop continues until
the starting time is reached. The DoEvents statement is important: it
allows background tasks to execute while the Test Script is suspended and waiting
to start. Without the DoEvents statement, the system would be unresponsive until
the start time had been reached.
When the newly recorded Test Scripts are executed on the same software on which
they were recorded, there should be no errors. The environment and the software
are identical to when it was recorded. There may be instances where the Test
Script does not run successfully. Testing the Test Scripts uncovers these cases,
and allows the scripts to be corrected before being used in a real test. Two
typical kinds of problems are discussed here:
- Ambiguity in the methods used for selecting items in a user interface can
make Test Scripts operate differently upon playback. For example, two items
recognized by their text (or caption) may have identical text. There will
be ambiguity when the script is executed.
- Test run/session specific data is recorded (i.e., a pointer, date/timestamp
or some other system generated data value), but is different upon playback.
Timing differences in recording and playback can lead to problems. Recording
a Test Script is inherently a slower process than executing it. Sometimes this
time difference results in the Test Script running ahead of the software. In
these cases, Wait States can be inserted to throttle the Test Script to the
speed of the software.
| |
|