I'm Hung Up on Main Memory Databases
by Brian Jepson
e-commerce types had just begun sniffing around in that area, hoping to make their web sites faster and faster.
Ever since then, I've been hoping to find time to play around with a main-memory database. But, there are only so many hours in a day, and there is so much other fun stuff competing for my attention.
Main memory databases are very cool. Once you throw away the assumption that everything is stored on a disk, you get a lot of advantages. For example, you can use hashtables for indexes instead of B-plus trees. That makes everything a lot faster, but you're not allowed to perform partial matches in queries (perhaps you are in some implementations, but the queries would be very expensive).
Of course, there is a lot of weirdness, too. Since all your data would disappear if the power went out, you need a good replication architecture. Or, you could use the main memory DBMS as a super-cache for something like Oracle. In that configuration, you could send UPDATE and INSERT statements directly to Oracle, but perform all your SELECTs against the main memory database.
If you want to play around with an open source main memory database, you might already have one installed. Check out MySQL's HEAP tables. I haven't evaluated how they compare with Polyhedra and TimesTen, but MySQL's HEAP table implementation seems comprehensive.
- Examining Main Memory Databases (although the author of this article sells a main memory database, he makes some interesting points).
- FastDB, an open source main memory database. (mirror)
- Main Memory Database Systems: An Overview by Hector Garcia-Molina and Kenneth Salem (IEEE subscription required for full text).
Speaking of Google, this article suggests that Google uses main memory databases to make their searches so darn fast. The Slashdot discussion of this piece makes for interesting reading, too.
Have you worked with a main-memory database? How did it compare to traditional relational database offerings?