OSCON 3.6: Practical Perl Testing
by Geoff Broadwell
(Auggghhh! The original version of this entry was lost due to system crash. I'm recreating it here, but it may have lost something. Or it may not have had it to start out with, we'll never know.)
Practical Perl testing was an odd session, with four instructors taking two session timeslots to deal somewhat randomly with various topics relating to Perl testing. Meanwhile MJD (in barefoot mode) lay on the floor chatting with Schwern and interrupted at random.
During the course of the rambling talk, they recommended several helpful modules:
- Use eq_or_diff instead of is_deeply to get better display of data structure differences
- Show shorter and hopefully more helpful failure messages when very long strings don't compare equal; several people pointed out that this module fell short of what it should have done
- Very flexible deep comparison of data structures
- Allow you to create your own testing harness; heavily work in progress right now
- Test::Exception and Test::Warnings
- Test for thrown exceptions and warnings (or lack thereof)
- Great gobs of sugary user agent goodness; greatly aids in testing and automating web site interactions
- Check for HTML errors in strings or files; also includes Test::More style wrapper and weblint CLI tool
- Flexibly compare HTML source against desired structure
- Make sure tests are actually exercising all existing code paths
- Specification-based automated boundary condition tests
Along the way, they recommended a couple good books: Sean M. Burke's Perl & LWP and a Java testing book whose title I have clearly written down incorrectly, as searching for it is bringing up no obvious matches. Sigh.
The Java book recommended a testing mnemonic called "Right BICEP", which roughly stands for:
Right: Correct output with normal inputs
B: Boundary conditions, random parameters, etc.
I: Inverse relationship (didn't quite catch the details on this one)
C: Cross-check using a different algorithm
E: Error handling tests
P: Performance within reasonable bounds, and not suddenly worse after a change
Finally, MJD asked about doing test-first development when you're unclear on the form that your app will eventually take. The most common suggestion was to write an exploratory mockup, write a full set of tests for it, comment out the entire mockup, and start uncommenting and fixing the mockup until the tests are happy again.
(Now that I have written this entry twice and fought with a broken machine in between, I'm literally falling asleep at the keyboard during page refreshes. I'll save today's sessions for tomorrow -- sorry, folks.)
What favorite Perl module or book would you recommend for test writers?
More on the MJD question
Finally, MJD asked about doing test-first development when you're unclear on the form that your app will eventually take.
There's a perlmonks thread (http://www.perlmonks.org/index.pl?node_id=480680) that Ovid started on the points that MJD raised.
Geoff, the name of that Java book is "Pragmatic Unit Testing in Java and JUnit".
Tell me about it
Practical Perl testing was an odd session
Sorry, left off the closing italics tag on the Test::LongString paragraph. Everything from "Rafael was actually in the audience" and onward is from me, not Geoff.
Ah, thank you brad! With that fix, it came up as the first entire page of Google results. :-)
Tell me about it
Trust me, it felt odd being one of the presenters, too. In my opinion (and I think it's shared by my cohorts), the presentation format was an experiment that went awry. :-)
Tell me about it
What was the original plan? I'm curious if it was an experiment that bears repeating with a few tweaks / lessons learned.