Tool Mentor: Executing Test Suites Using the Rational PurifyPlus Tools (Windows and UNIX)

Purpose

This tool mentor describes the use of the Rational PurifyPlus tools (Rational Purify, Rational PureCoverage, and Rational Quantify) in test suites to detect potentially harmful memory errors and leaks, to ensure that your tests are achieving an acceptable level of code coverage, and to bring performance problems to your attention. This tool mentor is applicable for use both with Microsoft Windows systems and with UNIX systems.

The PurifyPlus tools include Rational Purify, Rational PureCoverage, and Rational Quantify.

To learn more about PurifyPlus tools, read the Getting Started manual for PurifyPlus (Windows version or UNIX version).

For step-by-step information about using PurifyPlus tools, see the online Help for the tool.

This section provides links to additional information related to this tool mentor.

• Execute Developer Tests
• Execute Test Suite

Overview

You can use PurifyPlus tools to do the following:

• Detect memory errors and leaks (C/C++ programs on Windows and UNIX). Use Purify to pinpoint errors and leaks in the application under test.
• Monitor code coverage (Windows and UNIX). Use PureCoverage to show you gaps in your testing and alert you to regressions.
• Profile performance (Windows and UNIX). Use Quantify to warn you if new check-ins degrade performance.

Executing test suites with PurifyPlus tools results in reliable, error-free code that runs at maximum efficiency.

Tool Steps

To implement tests using the PurifyPlus tools:

1. Run the program under Purify to collect error and leak data (C/C++ programs only)
2. Run the program under PureCoverage to monitor code coverage
3. Run the program under Quantify to profile performance

1. Run the program under Purify to collect error and leak data (C/C++ programs only) 

Purify detects runtime errors, including memory leaks, in the application under test and in the components it uses. It reports memory errors such as array bounds errors, access through dangling pointers, uninitialized memory reads, memory allocation errors, and memory leaks, so that they can be resolved before they do any damage.  

Begin by incorporating Purify into your existing Perl scripts, batch files, and makefiles.

• On Windows , modify the command lines that run your program <ProgramName>.exe to include Purify:
   
  Purify /SaveTextData <ProgramName>.exe
   

 
• On UNIX , modify the compile and link line:
   
  % purify -logfile=<filename> cc -g <program_name>.c

Using the /SaveTextData on Windows, or the -logfile option on UNIX, causes Purify to run without the user interface, and, when you run the program from your script, sends the error and leak data to a text output file when your program terminates.

Examine this output file yourself, or write scripts to analyze it. You can use the error and warning messages in the file as additional criteria for your test results.

You can collect coverage data for the same runs, if you have PureCoverage installed, by adding the option /Coverage (Windows) or by adding purecov to your link line after purify (UNIX). Using the coverage data, you can also find the parts of your code that you have not checked for errors. For more information about using coverage data, see Step 2 of this Tool Mentor.

Notes :

• Use the Purify API to control data collection from within your program.
• Purify also has command-line options that allow you to save test data as binary files that you can examine using the Purify graphical user interface; this is discussed in the Tool Mentor "Analyzing Runtime Performance Using the Rational PurifyPlus Tools."

For more information, look up the following topics in the Purify online Help index:

• running programs
• error messages
• options
• API functions
• saving data

2. Run the program under PureCoverage to monitor code coverage. 

PureCoverage provides accurate line-level and function-level code coverage information. Use PureCoverage with your nightly tests to ensure that the tests are keeping pace with your code development.

Begin by incorporating PureCoverage into your existing Perl scripts, batch files, and makefiles.

• On Windows , modify the command lines that run your program, or Java class file, .jar file, or applet, to include PureCoverage:
   
  For C/C++ programs: Coverage /SaveTextData <ProgramName>.exe
  For Java class files: Coverage /SaveTextData /Java Java.exe <ClassName>.class
  For .NET managed code programs: Coverage /SaveTextData /Net <ProgramName>.exe
   
  Using the /SaveTextData option causes PureCoverage to run without the user interface, and, when you run the program from your script, sends the data to a text output file when your program terminates.

 
• On UNIX , modify the compile and link line:
   
  For C/C++ programs: % purecov cc -g <program_name>.c
  For Java class files: % purecov -java java <class_name>
   
  Use the PureCoverage -export option after the program exits to write the coverage data to a text output file.

As you exercise your code, PureCoverage records data about the lines and functions that are used. You can call PureCoverage API functions from your program to save data at specific points in your code, or to collect data only for certain routines. When you exit the program, you have an accurate set of data indicating which lines and which functions have, and have not, been covered by your test.

You can merge coverage data from multiple runs, or keep it in separate files to analyze changes in coverage.

Use scripts to compare the current data with data you collected from previous runs. If coverage drops, Your tests may not be exercising new code, or the new code may have introduced a defect that is causing a large section of code not to be tested. Use a testing tool such as Rational Robot or Rational Visual Test to write test cases that exercise the new code.

Note: You can examine binary coverage data files using the PureCoverage graphical user interface.

For more information, look up the following topics in the PureCoverage online Help index:

• running programs
• options
• API functions
• saving data
• comparing data

3. Run the program under Quantify to profile performance  

Begin by incorporating Quantify into your existing Perl scripts, batch files, and makefiles.

• On Windows , modify the command lines that run your program, or class file, .jar file, or applet, to include Quantify:
   
  For C/C++ programs: Quantify /SaveTextData <ProgramName>.exe
  For Java class files: Quantify /SaveTextData /Java Java.exe <ClassName>.class
  For .NET managed code programs: Quantify /SaveTextData /Net <ProgramName>.exe
   
  Using the /SaveTextData option causes Quantify to run without the user interface, and, when you run the program from your script, sends the data to a text output file when your program terminates.

 
• On UNIX , modify the compile and link line:
   
  For C/C++ programs: % quantify cc -g <program_name>.c
  For Java class files: % quantify -java java <class_name>
   
  Use the Quantify -export option after the program exits to write the performance data to a text output file.

As you exercise your code, Quantify records data about your program's performance. You can use API functions to pause and resume data recording at any time, and so limit profiling to specific portions of code. You can also save data at specific points in your code's execution, or collect performance data only for specific routines. When you exit your program, Quantify has an accurate profile of its performance.

You can write scripts that compare datasets and report changes in performance:

• Degradations in performance may indicate that the most recently checked-in code has slowed down the program. Analyze the data you have collected to find the sections of the program that have unacceptable performance.
• Marked improvements may indicate that the developers have improved their code, or that your tests for some reason have stopped exercising large sections of the code. Check your coverage data to see whether you are still achieving your previous levels of coverage.

Note:Quantify can also save test data as binary files that you can examine using the Quantify graphical user interface; this is discussed in the Tool Mentor "Analyzing Runtime Performance Using the Rational PurifyPlus Tools."

For more information, look up the following topics in the Quantify online Help index:

• running programs
• options
• API functions
• saving data
• comparing data

Rational Unified Process