Embeddable DBMSes for Python

by Uche Ogbuji

A proposal for bundling the Gadfly RDBMS with Python on the Python developer's list sparked a discussion of embeddable DBMS packages for Python.

The main distinction of Gadfly is that it is implemented entirely in Python and thus strongly cross-platform (the data files are also cross-platform portable).

Kevin Altis mentioned other embeddable DBMSes for Python which had cought his interest:

Interesting. I'm in the process of trying out Gadfly, PySQLite, and MetaKit as embedded databases. For reference, the links are:

Gadfly http://gadfly.sourceforge.net/

SQLite and PySQLite http://www.hwaci.com/sw/sqlite/ http://pysqlite.sourceforge.net/

MetaKit, Mk4py, MkSQL http://www.equi4.com/metakit/ http://www.equi4.com/metakit/python.html http://www.mcmillan-inc.com/mksqlintro.html

All are embeddable databases, but they each have their pros and cons. I can see how Gadfly would have a lot of appeal since it can be used as a pure Python solution. The licensing for MetaKit probably makes it inappropriate for the Python standard libs, but I'm sure that could be brought up with the author. PySQLite seems to be the most mature (MetaKit users may disagree), certainly SQLite is better documented, has a richer feature set, and as a bonus the source code is in the public domain! PySQLite appears to be quite fast.


Since it doesn't use a memory map like MetaKit, it should work equally well with small and large data sets.

Anyway, I'm probably a month away from being able to present an adequate comparison of using each for different relational datasets. One data set I'm looking at is roughly 800MB of data, the other is only about 256KB and I'm looking at the smaller one first since it also has a simpler table structure.

I would be interested in seeing both Gadfly and PySQLite supported in the standard libs. I'm guessing that Gadfly needs a lot of testing and probably bug fixes to justify including it in the 2.3 standard libs.

Richard Jones, Gadfly Author, responded:

Gadfly has the advantage that any marshallable Python object may be stored with no mess, no fuss. Sqlite is restricted to only storing strings. Metakit supports a variety of data types, but no explicit NULL. Actually, the three support wildly different types of "unset" values:

gadfly: python's None sqlite: sql NULL (and all its quirks ;) metakit: no support

Gadfly has the additional benefit that any Python object may support its View interface, and thus participate in SQL queries. Pretty powerful stuff.

What are your thoughts on these DBMSes, and any others like them? Do you prefer using commercial DBMSes with Python?


2003-01-23 18:31:01
Where does Zope's Object Database fit into the picture?
2003-01-23 18:41:20
ZODB is certainly another entrant.


Thanks for mentioning it. Do folks have others to mention?