Testing randomness

The problem you run into when trying to write unit tests for methods using random data is that you can’t predict what random data it’s going to use. The idea behind a unit test is that you control the input and test whether the output is as you expected, so that’s a problem.

Let’s assume you have a token generator like this:

module Token
  DEFAULT_LENGTH = 8
  CHARACTERS = ('a'..'f').to_a + ('0'..'9').to_a
  
  def self.generate(options={})
    srand
    options[:length] = DEFAULT_LENGTH if options[:length].nil?
    (1..options[:length]).map { CHARACTERS.rand }.join
  end
end

Now you can write tests to see if it generates tokens of the correct length, doesn’t generate homogeneous tokens and doesn’t generate the same token a number of times in a row. Unfortunately these tests could succeed a number of times and suddenly fail for no apparent reason. Usually this happens when you’re on vacation and one of your collegues is running the test on some weird architecture leaving them baffled.

The solution is to make Array#rand predictable by stubbing out the normal implementation:

class Array
  def round_robin_rand_list=(list)
    @round_robin_rand_list = list
  end
  
  def rand
    current = @round_robin_rand_list.shift
    @round_robin_rand_list.push(current)
    current
  end
end

Now you can write a test that checks the output, because you control what the random method returns:

class TokenTest < Test::Unit::TestCase
  def test_token_generate
    Token::CHARACTERS.round_robin_rand_list = [0, 1, 14]
    assert_equal 'ab8ab8ab', Token.generate
    Token::CHARACTERS.round_robin_rand_list = [7, 12, 15, 3]
    assert_equal '169d169d', Token.generate
  end
end

Obviously you need to clean up after yourself in tests so you can’t just redefine methods. It would be really cool if we could do something like this:

class TokenTest < Test::Unit::TestCase
  def test_token_generate
    Token::CHARACTERS.define_round_robin_method(:rand, ['a', 'b', '8'])
    assert_equal 'ab8ab8ab', Token.generate
    Token::CHARACTERS.undefine_round_robin_method(:rand)
  end
end

We’ve implemented this as round_robin_method.rb, complete with its own testsuite. You could go even further and hook it into TestUnit to automatically undefine the round robin methods after a test has run. We’ll leave that as an exercise for another day.