Lesson 1 - Performing White-Box Testing

White-box testing checks that the class is structurally sound. It doesn't test that the class behaves according to the specification, but instead ensures that the class doesn't crash and that it behaves correctly when passed unexpected input.

White-box testing involves looking at the class code and trying to find out if there are any possible class usages that will make the class crash (in Java this is equivalent to throwing an uncaught runtime exception).

Jtest completely automates white-box testing by automatically generating and executing test cases designed to fully test the class. During the test, Jtest executes the class using a symbolic virtual machine, then searches for uncaught runtime exceptions. For each uncaught runtime exception that is detected, Jtest reports an error and provides the stack trace and the calling sequence that led to the problem.

This lesson contains the following sections:

Performing White-Box Testing: Overview

To perform white-box testing, just tell Jtest what class or set of classes to test then click the Start button.

If you test a project, results will be displayed in the Class Name> Errors Found> Uncaught Runtime Exceptions branch of the Project Testing UI's Results panel.

If you test a single class, results will be displayed in the Uncaught Runtime Exceptions branch of the Class Testing UI's Errors Found panel.

Performing White-Box Testing: Example

Go to Jtest's Class Testing UI. (This UI opens by default when you launch Jtest).
If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.
Browse to Simple.class (in <jtest_install_dir>/examples/eval) using the Browse button in the Class Name panel.
Click the Start button in the tool bar.

Jtest will perform static and dynamic analysis on the class. A dialog box will open to notify you when testing is complete. Information on test progress will be displayed in the Test Progress panel. Problems found will be reported in the Errors Found panel.

You will see two types of errors reported in the Errors Found panel. For this example, we're going to focus on the uncaught runtime exception found.

The Errors Found panel will list the following uncaught runtime exception:

This error message reveals that there is some input for which the class will throw an uncaught runtime exception at runtime. This could cause the application running this class to crash.

To see a stack trace like the one the Java virtual machine would give if this uncaught runtime exception were thrown, expand this branch.

To see an example usage of this class that would lead to the reported uncaught runtime exception, expand the Test Case Input branch.

The error shows us that the "startsWith" method is implemented incorrectly. The method should return false for the argument "" and "0" instead of throwing a runtime exception.

If the error is not fixed, any application using this class will eventually crash or give incorrect results.

To view the source code of the class (with the problematic line of the stack trace highlighted), double-click the node containing the exception's file/line information.

Viewing Automatically Generated Test Cases

To see a selection of the test cases that Jtest automatically generated for this test, click the View button in the tool bar. The View Test Cases window will open.

In the View Test Cases window, expand the Automatic Test Cases branch. This will display a list of this class's methods. Click any method with a number greater than zero by it, and open the Test Cases branch, or press Control and click your right mouse button, then choose Expand Children from the shortcut menu that opens. The inputs that Jtest generated for each method are now displayed.

Suppressing Exceptions

You can tell Jtest to suppress uncaught runtime exceptions that you do not expect to occur or that you are not concerned with by adding Design by Contract tags to your code. To have Jtest suppress errors for inputs that you do not expect to occur, use the @pre tag to specify what inputs are permissible. To have Jtest suppress expected exceptions, use the @exception tag to specify what exceptions you want Jtest to ignore.

For example, if you wanted to suppress reports of an expected NegativeArraySizeException that occurs when a negative index is used as an index to an array, you might enter the following comment above the appropriate method:

/** @exception java.lang.NegativeArraySizeException */

This not only tells Jtest to ignore the exception, but it also makes code easier to understand and maintain.

If you use the @pre tag to indicate valid method inputs, and then use ParaSoft's Jcontract to check Design by Contract contracts at runtime, you will automatically be alerted to instances where the system passes this method any unexpected inputs.

Viewing a Text or HTML Format Report

Jtest automatically creates a report for each test. You can view this report by clicking the Report button. By default, this report is formatted in text (ASCII) format. If you would like Jtest to generate HTML reports (e.g., if you want to post the report on your development intranet), choose Preferences> Configuration Options> Report Format> HTML before you click the Report button..

By default, the report file will be saved in <jtest_install_dir>/u/username/results. To prompt Jtest to save reports to a different location, modify the Common Parameters> Directories> Results parameter at the Global, Class, or Project level.

Setting an Object to a Certain State

In some cases, you may want to set up an initial state prior to testing a class. For example, suppose that a class is used as a global object containing static member variables accessible by any other project within the application. When Jtest tests an object that uses this static member variable, a NullPointerException will result because the variable has not been set. This problem can be solved by giving Jtest initialization code.

For example, the file ExecGlobal.java (in <tutorial_install_dir>/lesson1/ExecGlobal.java) contains a static variable that is assigned by its ancestor Exec, which is defined in the Exec.java file (also in <tutorial_install_dir>/lesson1/Exec.java). In this example, the instantiation on the Exec object is performed by the Exec object's "main" method, which is defined in the Exec.java file. When Jtest tests the ExecTest object's doTest method, it does not know that a certain state is assumed by the doTest method. Thus, initialization needs to be performed.

To see why initialization is necessary:

Go to Jtest's Class Testing UI. (This UI opens by default when you launch Jtest).
If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.
Use the Browse button in the Class Name panel to choose ExecTest.class (in <tutorial_install_dir>/lesson1/ExecTest.class)
Test the class by clicking the Start button in the Class Testing UI tool bar.

A NullPointerException is reported because the assumed state is not available. You will need to perform static initialization on the ExecTest class to ensure the objects referenced by this class are in the assumed state.

To perform the necessary initialization:

Click the Class button to open the Class Test Parameters window.
In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> Common.
Double-click the Static Class Initialization node.

The Static Class Initialization window will open.
Enter the following initialization code in the Static Class Initialization window.

new Exec ();This will create an instance of the "Exec" object before the class "ExecTest" is tested.

(Optional) If you want to check the method, choose Save & Check from the Options menu.
Choose Options> Save.
Choose Options> Quit.
Click Start in the Class Testing UI tool bar to retest the class. You no longer receive a NullPointerException because an instance of "Exec" is available when testing the "ExecTest" object.