Regression testing of simulation programs A. Markus At WL|Delft Hydraulics we develop and maintain large computational programs to simulate (model) the hydrodynamics of such diverse natural systems as rivers, estuaries, lakes and coastal seas. Other aspects of these water bodies that are being modelled in our programs are: water quality in general, primary production (growth of algae), toxicants and morphology (bottom changes due to sediment transport). Because these programs are used by ourselves as well as by our clients, we need to be able to guarantee that new developments and bug fixes do not have unexpected consequences. In other words: we need to verify that results remain the same, unless we know they should be different. These programs are extensive and complex: partly because they have a long history of development and partly because they are used for many different situations and hence model many physical and biochemical processes. Manual testing new versions became an ever growing burden and several years ago we decided to automate the procedures. There were several requirements to be met: - the test suite should run on different platforms (currently: Windows and Linux) - it should be extensible (adding new test cases should be easy) - it should be possible for non-programmers to contribute - it should be flexible enough to cover all our programs - results from a new run should be compared _numerically_ to those from the reference run - it should be easy to use: - switch between different version of the computational programs - run only a part of all the tests - allow different levels of details for reporting differences, so that the user can get an overview as well as a detailed report for specific cases. - both running the test cases and reporting the results should be completely automatic: some of the test cases take several hours to finish. It is then desirable that the test suite runs overnight and the results can be inspected the next morning. After some research into the various possibilities Tcl was chosen as the language to implement the test suite in. The main reasons for this choice were the ease by which external programs can be controlled and the short learning curve and platform-independency. While over the years the actual software has become quite complex, the original structure and philosophy are still recognisable.