The first three points of the joel test, if you’re not familiar with it, basically ensures that your software is always in a known state. You always have a build you can throw to testers, internal users or sales and it means fixing a critical bug is about the bug, and not the build. If you’ve ever been on a project where you have a weekly or monthly integration task where builds and releases ‘come together’ you’ll understand the value of release early and often.
One of the tools I introduced where I work was cruisecontrol. As a continuous integration tool, it means our software is always ready to go. Basically the way we have it setup, clicking build can build all our java and c# source, run unit tests, version and package into all the various installers and publish to the releases directory for testing. The problem at the moment is the last mile. Our UI tests for the winforms components are run in Mercury, sorry now, HP Quick Test Pro which lives out of the continuous integration tool and are run manually. This is a pretty obvious sore point so I’ve been working with the testing team to close the circle.
The last step in our build automation plan is to automate the UI testing into our nightly and release builds. We basically have build machines for each product, and each stream for each product and then another beefy test box which runs all the Mercury tests. This distributed environment means that we needed the test cruisecontrol to be able to watch the releases share to pick up new builds. Because we have multiple products publishing into this directory the filesystem modificationset listener isn’t specific enough as it won’t pick up a single product suite. So to resolve this, today I wrote a custom modification set listener. All up the process was pretty straight forward, and in the end including unit tests and the automated install script for building and configuring build machine, it took about an hour and a half.
So basically it’s the filesystem listener but now takes a base bath and then a regex to watch for new directories being created in that base directory. Whenever we get a new root directory we know that we’ve had a published release and we should run the UI tests. The most confusing thing when writing the new plugin is that technically it appears that the all the modification system classes implement a sourcecontrol interface, which at least semantically, seemed misleading.
All up I think it’s an extremely solid product and the cost of getting the source, figuring it out and getting a new plugin running was simple enough. The only downside I think is that plugins, despite being relatively simple are a bit of a pain to install and run. My current thought was that if we could supply a new plugin or customise via a scripting language it would be considerably simpler rather than deploying a new jar into cruisecontrol and the associated configuration in the config.