Testing, Logging, and the Art of Monkey Patching

by Jeremy Jones

Most of the code that I've been writing for the book has been getting its own unit tests. I've been working on a chapter on networking for the past week and a half and have written a little code for the chapter. One of the challenges of writing tests for something like networking code is that there are so many variables which may influence a suite of unit tests. For example, if my unit tests rely on hitting some Google webserver and I encounter problems, trouble shooting questions may include: is my router acting up, is my ISP acting up, am I failing to get DNS resolution, is that particular server down, have they changed the URL for this resource, etc.

So, for the purpose of testing, I decided to bypass the socket module in this case and handle everything locally. I created a faux socket class fleshed out with the methods that I needed. I then monkey patched my module under test with the new faux socket class. All attempts to connect to a real socket actually "connected to" a fake socket from which I could totally control the behavior.

7 Comments

Lawrence Oluyede
2007-08-22 02:53:34
I too use a lot faux (stubs) objects and sometimes mocks. It depends on which kind of testing I'm doing. I find Fowler's article about stubs and mocks very enlighting: http://www.martinfowler.com/articles/mocksArentStubs.html



Jeremy M. Jones
2007-08-22 04:56:21
@Lawrence,


I was actually reading that article last night. Good stuff. This definitely falls more on the stub side of things than the mock side. I need to do a little work with mocks to see if they fit my way of doing things. Thanks for the post and the reference to Fowler's article.

Doug Hellmann
2007-08-22 06:10:23
Fake objects are great for testing, since you don't need the test fixtures in place, as you point out. I set up a real "fake" web server in the tests for feedcache, which turned out to be fairly easy. I didn't consider monkey patching urllib as you did with socket, mostly because I've found monkey patching introduces difficult-to-debug errors.


In your example, you could also have passed a "socket factory" to your function. The default would be the normal socket class, and the tests could explicitly pass in a FauxSocket. By monkey patching socket.socket, all users of the socket module are using the FauxSocket instead, which might cause issues if some of the tests weren't expecting it.


Oh, and I had the same reaction you did to the mock library -- what's the point? Why not just write fake objects, like this? I need to read the article Lawrence pointed out.

Jeremy M. Jones
2007-08-22 06:29:53
@Doug,


I had thought about changing the function so that it would take a socket class (or module or something), but didn't really think of using a factory per se. The only problem I have with that is that going that route just for the sake of testing results in this over-configurable feeling piece of code. I'm all for allowing options where they make sense, but it seems like a lot of the pattern-heavy folks want to make everything so optional because "what if someone 20 years from now wants to allow passing in a string-like object which isn't really a string to your method? We need a stringish factory!". What I'm saying is that my reluctance to do that is a psychological result of overpatterning and overengineering out there. That little rant was against super pattern heavy people out there (you know who you are!) and not you, Doug. Sorry.


However, great point about affecting other tests. That would be a strong reason to not monkey patch, but to use your factory suggestion. On the other hand, I should be able to fix things by monkey patching in setUp and un-monkey patching in tearDown.


As always, Doug, thanks for the comment!

Doug Hellmann
2007-08-23 04:18:48
Is un-monkey patching a verb? :-)


I agree that gratuitous pattern application leads to over-engineered designs. I've found YAGNI to be a helpful mantra.


Another way to hide the "testish" nature of the factory pattern would be to manage it at the module level. Rather than having your real function take the factory as an argument, provide module level functions like "setSocketFactory()" (used by the test to change the value) and "getSocketFactory()" (used internally by your real function to discover how to create sockets). The functions could operate on a module-level global variable to hold the factory.


This approach gives you the best of both worlds: all of the adventure of monkey patching, without the risk! :-)

Jeremy M. Jones
2007-08-23 05:04:22
@Doug,


I like the module level approach. But if it's module level, how would that prevent other tests from getting potentially mixed up by this test setting it? Or were you talking about my monkey patching affecting the `socket` module. If the latter, then I can see that. That'll work nicely. And I can set and re-set the real socket with setUp and tearDown just to insure that nothing else is going to be confused by this module under test being modified.


BTW - "un-monkeypatching" is a gerund. "Un-monkeypatch" is a verb :-)

Doug Hellmann
2007-08-23 16:25:01
Thanks for the primer in parts-of-speech. :-)


You have the idea: I'm worried that modifying the contents of `socket` will break other modules in unpredictable ways. If you limit yourself to making runtime changes in your own modules, at least you'll only break code you know depends on those modules.