This old site is no longer updated. Visit the new site.

Unit testing hints that you might have missed the first time

After spending all too many hours toiling away on Drupal 5 sites without the benefit of unit testing, I’m finally getting to spend some quality time with Drupal 6’s SimpleTest module. Things have certainly gotten a lot better in the Drupal testing world over the last year or two.

As I work to develop my testing chops it’s hard to avoid pining for Ruby, where the community has been obsessed with testing for years, where lots of experiments have been tried, and where the language syntax actually works with you to make test code more elegant. Even something basic, like asserting the presence of an exception in PHP:

    $threw = FALSE;
    try {
    catch (MyKindOfException $e) {
       $threw = TRUE;

…becomes so much nicer in Ruby test/unit (which is hardly the last word in Ruby testing utilities):

    assert_raise MyKindOfError do

But, though we can’t hope to mimic the elegance of Ruby’s block syntax in PHP, we can at least try to take advantage of the Ruby community’s hard-won experience with testing methodologies and design patterns. One of my favorite Ruby bloggers was Jay Fields – he’s been working more in Java lately, but his old Rails links live on:

  • First, the disclaimer: Even the “industry experts” on testing are still trying to figure it out, and there is no silver bullet: the right kind of testing to do depends on what you’re trying to write.

  • Having said that: Setup and teardown methods make your tests less readable and maintainable, and the guy who designed the NUnit library wishes he hadn’t supported them at all.

  • The Rails community has experimented with many ways of representing the data that your tests will operate on. Rails 1.0 provided fixtures; after much trial and error, the consensus seems to be that they suck and that you will eventually figure that out. The folks I follow have dabbled with Object Mothers before settling (for the moment) on the Test Data Builders pattern; the fairly popular Factory Girl library appears to be a Rails implementation of the Test Data Builder concept.

  • In production software, duplicated code is generally considered harmful. In test code, duplicated code is often your friend. It is more important for a test routine to be easily understood in isolation than for it to avoid duplicating other test routines. Test code is not production code: The tests run in isolation, are generally read in isolation, and benefit from being written in isolation.

  • While we’re on the subject of isolation: write one assertion per test. I like this idea, but – alas! – it is an even more radical idea in Drupal than it is in Rails or Java, because SimpleTest is (a) wicked slow at the moment and (b) linearly dependent on the number of test methods. (Or such is my impression – insert a caveat about the importance of formal benchmarking here.) I believe SimpleTest does a lot of tedious DB setup and teardown for every test method, and it doesn’t have transactional tests. One of us needs to be chained to a desk next to boombatower and forced to implement some speed boosts.

But, even if we are compelled to compromise in practice, we can still hold the ideal in our hearts: one assertion per test is great, multiple assertions per test is a compromise, and tests which interdepend on each other in a complicated way are… not good.

  • Here’s one I need to think about: writing tests without names. Not that we will be doing this in PHP soon; even in Ruby it looks a little weird. But if zero names might be right, then two names is almost certainly one too many. I’ve been thinking about the Drupal SimpleTest convention of equipping every assertion with a gloriously descriptive, HTML-formatted comment:

    $this->assertTrue($bool, t('The bool should be true'));

    and noticing that the presence of the comment makes the test method’s name entirely redundant. Maybe I’ll experiment with using tiny, inconspicuous method names.