Reliability in the face of complexity is the real challenge of software development, as discussed in this paper.

Download and explode the test cases

The input files for the current version of the test suite is in this directory and the resulting output is in this directory.  There are three classes of tests.  The "auto", which is run every night in Lexington, is a large number of simulations that test some aspect of the physics.  These are contained in the files "tsuite_input_auto*".  The second "slow" part is contained in the files "tsuite_input_slow*" , which takes about three days and are continuously running on machines here.  The file "programs*" contains programs that test code when called as a subroutine of a larger program.  Each download contains several perl scripts and html files. The readme_tests.htm file gives an overview and  doc_tsuite.htm documents the purpose of each of the tests. These should be studied to see how to set up the code. 

Explode the files into a directory.  They should not be placed in a directory which contained an older version until the old version is first deleted.  The names of the test cases change from time to time and placing the current version of the suite on top of an old one will not overwrite all the old ones.  This will cause problems when the test suite is run.  Either start with a new directory or first delete the contents of the directory.

Running the test cases 

The test suite includes Perl scripts to run all the test cases and then check for problems.  These are files that have names  *.pl.  

The runall.pl script will run all the input files.  You will need to edit it so that the string "$exe" points to your executable for Cloudy.  This test suite takes about half a day on with a fairly modern CPU, so you might want to do this overnight.

The "assert" commands within the test suite

A large code must be completely checked every time anything is changed.  The code uses extensive self-checking during a calculation to insure that the results are valid.  The assert commands in the input files tell the code what answer to expect, based either on analytical or previous results. This allows the code to confirm that it has found the correct answer.  A distinctive string is printed if the right answer is not obtained.  The Perl script checkall.pl will examine all output files to search for failed asserts, other problems, or crashes.  These would indicate a problem.

The assert commands have nothing to do with the simulation or the astrophysics, but instead provide a way for the code to implement these automatic self-checks.  The Perl script tests_remove_asserts.pl will remove all the asserts if you wish to reuse these scripts for other purposes.

Check the results

Problems will be announced at the end of the calculation by a line that includes the string PROBLEM or "botched asserts".  If the code crashes then the normal end-of-calculation string will not be printed.

The test distribution includes the Perl script checkall.pl that will check for problems once the test cases have been computed.  Run this script and notice what it says.

Some of the tests cases "stars_*.in" require the compiled stellar atmosphere files.  These will not suceed if you do not set up the stellar atmospheres.

Test cases output

The files "tsuite*output" in the test suite directory here have output for several platforms.  Note that the code that generated this output did not have any hot fixes included. 

Next step, apply any hot fixes.

Last changed 04/08/06.
Return to the Cloudy Home Page.
Copyright 1978-2006 Gary J. Ferland