TSIPP Workbench - Qualification

Qualification

The author believes that release 1.6a, while not introducing many new features compared to 1.5a, should prove to be of far higher quality. This has been achieved through three types of testing. The test infrastructure used to achieve this is included within the released program, and documented below, so that the end-user can repeat the verification within the end-user's environment.

Regression Testing

This is performed using the capture/playback features, which were inspired by the program "TkReplay" by Charles Crowley, which is documented in [ref 2]. Examples of capture files for each release starting from 1.6a onwards may be found in the "regression" directory. The following filename conventions are used:

t<id>.tcl is a cature file;
e<id>.tcl is the source file saved immediately after capturing t<id>.tcl;
it is recommended that when you play back t<id>.tcl, you should then save the source for the image generated as f<id>.tcl;
The files e<id>.tcl and f<id>.tcl may match exactly. If the playback is performed on a different machine architecture however, differences in real number precision may occur. In this case, use the command "::testProcs::compareFiles e<id>.tcl f<id>.tcl [tolerance]" to allow for this. The optional parameter tolerance defaults to 1e-6.
The file t<id>.jpg contains a rendered image from the test, which may be used to compare the result of the regression test visually.

To run the regression script, execute TSIPPwb and immediately go to the "File Control" view; hit "Playback"; enter the filename t<n>.tcl and hit "ok".

Directed Testing

The command testProcs::loadSaveTest may be run which verifies that one or more source files may be loaded, rendered and saved so that they match the original file. The command is declared using the "opt" package, and a summary listing of the optional parameters can be obtained with the command "randomTest::randomTest -help". The following table gives more detail.

**Parameters of the Utility "`testProcs::loadSaveTest`"**
Flag	Type	Default Value	Description
-tolerance	float	(0.0000001)	Tolerance in the variation of real numbers due to rounding errors. On completion of the test, the maximum deviation in any numeric parameter will be printed. This can be used as the tolerance provided with this flag to a subsequent run of the test so that the test can be run with maximum checking while still giving a clean result.
-keepSavedFiles	boolflag	(false)	By default, the saved files will then deleted and the directory will be left as it was before the command was entered, with the modification dates on the files unchanged. This option instructs the test to keep the saved files with extension "`.tcl-test`".
-keepMultiViews	boolflag	(false)	By default, all MultiViews will be deleted on termination of the test, so that the routine can check that no Component objects remain, to demonstrate that the TSIPPwb has no memory leakage. This option instructs the test to keep MultiViews instead of destroying them.
-file	string	([glob *.tcl])	Files to be loaded and saved, defaulting to all files in the current directory with extension "`.tcl`"
-checkSelected	boolflag	(false)	After each file is loaded, check the consistency of the canvas selection agains the tracking variable selectedObjects.
-checkShaders	boolflag	(false)	After each file is loaded, check the correctness of the AutoDestruct usage counters of all Shaders.
-checkVertexSet	boolflag	(false)	After each file is loaded, check that all Polygon vertex linking is bidirectional.
-checkAggregate	boolflag	(false)	After each file is loaded, check that all objects of class Aggregation are properly owned by either Freehand or MultiView objects.

Constrained Random Testing

The command randomTest::randomTest provides an extensive facility for this.

Known Problems

There are still bugs! This version of the TSIPPwb must be considered an alpha release. Problems and requests for future work are tracked on SourceForge. Visit http://sourceforge.net/tracker/?group_id=38565 to see the current status.