At Edinburgh we are hosting the full range of VO Services from datasets such as the WFCAM Science Archive to applications such as the Anomaly Detector and STILTS to infrastructure such as MySpace. We have to make sure that we know that systems have fallen over before our customers do.
We are going to need a multilayered solution, from simple-minded pinging of webservices such as that provided by MARS (currently down, awaiting a java upgrade) to the heartBeat service in the VOSupportInterfaces currently being developed by the IVOA GWS group. Going to greater depth, I think we need automated system tests of all of our user-facing components, testing them in the same way as a user would. These system tests would be run (say) once a day, rather than providing the real-time feedback offered by the other techniques. For example, MySpace should be tested by creating folders and files, storing and retrieving data. A dataset should be tested by running a standard conesearch etc.
A convenient way to run these tests is via the Astro Server Runtime developed by Noel Winstanley. This offers an easy API to all the VO services we deploy here at Edinburgh and allows me to write the tests in my language-of-choice. From my previous experience of running the Integration&Release process, I've decided to go with JUnit tests wrapped in a Maven project run by a cron job. This is because JUnit tests are easy to write, and Maven provides easy facilities to publish reports to the web on the success rate of the tests.
Status of the System Tests project:
- Tests for Community and MySpace written
- Tests for conesearches fleshed out
- Code has been put into cvs under: http://www.astrogrid.org/viewcvs/test/AgAtRoe/
- Bugs in the Workbench and Astro Server Runtime have been identified that block further development
- Need to restructure build to use the full, un-Proguarded ASR.
