Django, MySQL, and data type oddness

by Jeremy Jones

My wife's store has been running in production for a few weeks now without much event. As with any website/webapp, enhancements and bug fixes are inevitable. I've already installed one new release which fixed a Javascript bug. I'm currently working on another release which will add some additional content and a couple of areas of feature enhancements. I've also been refining my release management procedures which includes a more formal test environment.

Which brings me to the topic at hand. I recently created a test environment at my hosting service. The test environment consists of a different domain name (subdomain, actually), a different directory within my home directory for placing application files, and a new database instance. When I installed the latest version of my code from Subversion, I encountered an error that I could not get past and which took me several hours to track down. (As a side note, it's amazing how "obvious" a solution is once you've solved it.)

It turns out that in my test environment, all "decimal" types from the database were being pulled into my application as strings. When I performed a math calculation on them such as adding what should be a decimal to an int, I would get a type error. If only it were that obvious at first. The first error that I saw was a pickling error. And it wasn't the pretty traceback that Django typically gives. This looked more like a documentation page generated from Python source, only nastier. I couldn't figure out why I was getting a pickling error. I thought at first that it didn't like me putting an instance of a custom class in session memory. (By the way, the pickling error appeared to be happening when Django was trying to stuff something into session memory.) That turned out to not be it. Somehow, through changing code, I was able to get a better error message and diagnose the problem.

This odd behavior is only happening in my test environment, and in neither of my primary development environment nor my production environment. Here is a query in my production environment showing the proper behavior:

>>> from django.models import store
>>> p = store.products.get_list()[0]
>>> p
The Gosling Design
>>> p.price
Decimal("25.00")

And here is the test environment showing bad behavior:

>>> from django.models import store
>>> p = store.products.get_list()[0]
>>> p
The Gosling Design
>>> p.price
'25.00'


Just for the sake of getting everything working properly, I had to go through the code and explicitly wrap decimal database lookups with a call to Python's decimal.Decimal(). The only thing I can conclude is that my hosting service created the new database instance on a different version of MySQL. So, it really could be either a Django or a MySQL thing.

2 Comments

Adrian Holovaty
2006-02-27 09:54:01
Somebody just reported this in the Django bug tracker a couple of days ago -- it appears the newest version of MySQL typecasts decimal objects into a different type than before. We'll put this one to bed soon.
Jeremy Jones
2006-02-27 10:03:33
Adrian - I figured something like that was the case. Good to hear you are on it. My confidence in Django wasn't disturbed for a moment.