OSCON 3.6: Practical Perl Testing

by Geoff Broadwell

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7002




(Auggghhh! The original version of this entry was lost due to system crash. I'm recreating it here, but it may have lost something. Or it may not have had it to start out with, we'll never know.)




Practical Perl testing was an odd session, with four instructors taking two session timeslots to deal somewhat randomly with various topics relating to Perl testing. Meanwhile MJD (in barefoot mode) lay on the floor chatting with Schwern and interrupted at random.




During the course of the rambling talk, they recommended several helpful modules:




Test::Differences

Use eq_or_diff instead of is_deeply to get better display of data structure differences


Test::LongString

Show shorter and hopefully more helpful failure messages when very long strings don't compare equal; several people pointed out that this module fell short of what it should have done


Test::Deep

Very flexible deep comparison of data structures


Test::Harness::Straps

Allow you to create your own testing harness; heavily work in progress right now


Test::Exception and Test::Warnings

Test for thrown exceptions and warnings (or lack thereof)


WWW::Mechanize

Great gobs of sugary user agent goodness; greatly aids in testing and automating web site interactions


HTML::Lint

Check for HTML errors in strings or files; also includes Test::More style wrapper and weblint CLI tool


Test::HTML::Content

Flexibly compare HTML source against desired structure


Devel::Cover

Make sure tests are actually exercising all existing code paths


Test::Lectrotest

Specification-based automated boundary condition tests




Along the way, they recommended a couple good books: Sean M. Burke's Perl & LWP and a Java testing book whose title I have clearly written down incorrectly, as searching for it is bringing up no obvious matches. Sigh.




The Java book recommended a testing mnemonic called "Right BICEP", which roughly stands for:




Right: Correct output with normal inputs

B: Boundary conditions, random parameters, etc.

I: Inverse relationship (didn't quite catch the details on this one)

C: Cross-check using a different algorithm

E: Error handling tests

P: Performance within reasonable bounds, and not suddenly worse after a change




Finally, MJD asked about doing test-first development when you're unclear on the form that your app will eventually take. The most common suggestion was to write an exploratory mockup, write a full set of tests for it, comment out the entire mockup, and start uncommenting and fixing the mockup until the tests are happy again.



(Now that I have written this entry twice and fought with a broken machine in between, I'm literally falling asleep at the keyboard during page refreshes. I'll save today's sessions for tomorrow -- sorry, folks.)


What favorite Perl module or book would you recommend for test writers?


7 Comments

adrianh
2005-08-05 04:28:19
More on the MJD question
Finally, MJD asked about doing test-first development when you're unclear on the form that your app will eventually take.


There's a perlmonks thread (http://www.perlmonks.org/index.pl?node_id=480680) that Ovid started on the points that MJD raised.


brad.cavanagh
2005-08-05 08:40:14
Java book
Geoff, the name of that Java book is "Pragmatic Unit Testing in Java and JUnit".
wnodom
2005-08-06 21:11:49
Tell me about it
Practical Perl testing was an odd session


Trust me, it felt odd being one of the presenters, too. In my opinion (and I think it's shared by my cohorts), the presentation format was an experiment that went awry. :-)


Fortunately, quite a few people told me afterwards that they got a lot out of the talk, and had added new tools to their toolboxes as a result of it. The session may have been choppy and a little surreal, but I'm thankful that a lot of folks found it useful nonetheless.


Meanwhile MJD (in barefoot mode) lay on the floor chatting with Schwern and interrupted at random.


Well, sort of. You may have seen me duck down early in the session to talk with both of them. I wasn't asking them to be quiet; rather, I was actually asking them to speak up if they had a question or comment, instead of just talking to each other. They're both really smart, with interesting (albeit very different) perspectives, so I asked them to jump in when appropriate. I was glad they did.


Test::LongString - Show shorter and hopefully more helpful failure messages when very long strings don't compare equal; several people pointed out that this module fell short of what it should have done.


Rafael was actually in the audience, and updated this module to address some of those shortcomings while the session was still underway. Sometimes the system works. :-)


Thanks for the synopsis, and the comments (even the ones that sting a little).


Bill Odom

wnodom
2005-08-06 21:15:29
Stupid fingers
Sorry, left off the closing italics tag on the Test::LongString paragraph. Everything from "Rafael was actually in the audience" and onward is from me, not Geoff.
Geoff_Broadwell
2005-08-07 11:29:49
Java book
Ah, thank you brad! With that fix, it came up as the first entire page of Google results. :-)
Geoff_Broadwell
2005-08-07 11:40:36
Tell me about it
Trust me, it felt odd being one of the presenters, too. In my opinion (and I think it's shared by my cohorts), the presentation format was an experiment that went awry. :-)


What was the original plan? I'm curious if it was an experiment that bears repeating with a few tweaks / lessons learned.


Fortunately, quite a few people told me afterwards that they got a lot out of the talk, and had added new tools to their toolboxes as a result of it. The session may have been choppy and a little surreal, but I'm thankful that a lot of folks found it useful nonetheless.


And I was certainly one of those folks. Sorry that the bitterness associated with computer issues made me miss saying so the second time around!


<MJD and Schwern interruptions>


Actually, I think this might have worked well as a planned thing -- "We're the presenters, but we've asked a couple other people to jump in if we miss something."


Rafael was actually in the audience, and updated this module to address some of those shortcomings while the session was still underway. Sometimes the system works. :-)


Very cool. One of those things I really love about real-time geek get-togethers of any sort.


Thanks for the synopsis, and the comments (even the ones that sting a little).


You're welcome, and sorry if they came out too harsh. "Computer troubles and sleep deprivation make Geoff something something."

wnodom
2005-08-08 18:30:52
Tell me about it
What was the original plan? I'm curious if it was an experiment that bears repeating with a few tweaks / lessons learned.


We wanted to make things fast-paced, switch among topics relatively quickly, and to have all four of us participating more rather than less. Think one main presenter at a time, with others jumping in with additional information, questions, or even alternative viewpoints. Last-minute changes (that seemed like a good idea at the time) -- coupled with an unexpected shortage of wireless mics -- made that a lot harder to do. I'm oversimplifying, of course, but you get the idea.


All four of us have done "pair presentations" before, so we know that can work well, but the jury's still out on how to make it scale. :-)


Actually, I think this might have worked well as a planned thing -- "We're the presenters, but we've asked a couple other people to jump in if we miss something."


So you can see how we thought that what I described above could work. I did ask Schwern to show up, and Mark was a pleasant surprise. (Really, I mean it.)


sorry if they came out too harsh


Oh, no worries. Believe me, I expected much worse... and it's probably out there, waiting for me to find it. :-)