The power of one little automated test
by Andy Lester
It all started easily enough. There was a little note sent to the Perl
perlop man page mentions: Binary "x" is the repetition operator
... repeated the number of times specified by the right operand.
It should mention what about if the right operand is negative, e.g.,
print '-' x -80
I figured I could make a quick documentation fix, and maybe even add
some automated tests to the Perl test suite.
x operator in Perl does repetition on a scalar or list, as
appropriate. For example:
$a = "abc" x 2; # $a = "abcabc";
@a = ("abc") x 2; # @a = ("abc","abc");
If the right-hand operator is 0, then you get an empty scalar or list,
as appropriate. If the right-hand operator was negative, it was the
same effect as having it be zero. As the bug said, the documentation didn't say anything about negatives, so I decided to investigate, and document appropriately.
I added a little sentence to the paragraph describing the operator, and
then I added some tests. If it's worth documenting, it's worth testing.
Documentation and tests are as much a part of the code as the code itself.
The t/op/repeat.t file in the Perl distribution already had a lot of tests in it, like:
is('-' x 5, '-----', 'compile time x');
is('-' x 1, '-', ' x 1');
is('-' x 0, '', ' x 0');
So I added the obvious add-ons:
is('-' x -1, '', ' x -1');
is('-' x undef,'', ' x undef');
And then went to add them to the list-related sections:
@x = qw( a b c );
is(join('', (@x) x -14), '', '(@x) x -14');
Before I sent the patch in, I ran a full
make test and found that
the last test didn't pass. In fact, it caused a Panic in Perl, and the
program died. I boiled it down to a simple:
@x = (1,2,3);
@y = (@x) x -1;
Turns out that that case of a negative or zero operand wasn't handling the stack correctly. Fortunately, this was only in the latest development version of Perl, but at least we found the problem.
A quick patch made it all better.
Some morals to this story:
- Never underestimate the power of one little test.
- There is no such thing as a dumb test.
- Your tests can often find problems where you're not expecting them.
- Test that everything you say happens actually does happen.
- If it's worth documenting, it's worth testing.
What bugs have you accidentally stumbled across, and how did you find them?