VoltDB’s New Command Logging Feature

written by VoltDB Team on July 19, 2011 with no comments

We, at VoltDB, are excited to tell you about the command logging feature that we’ve been working on this summer. We’ve built this feature because our customers asked for it, and they’ve given us some great feedback on how it should work. Here’s a heads up to get you thinking about how you can make use of this new functionality.

So what’s command logging and why are we so excited about it? First, let me remind you of VoltDB’s snapshot functionality. A database snapshot is exactly what it sounds like — a point-in-time copy of the database contents written to disk. You can later use the snapshot to restore the data. To save and restore data across sessions, you can perform a snapshot before shutting down the database and then restore the snapshot after the database restarts. That’s pretty useful if you know that you want to stop the database. But what about the times when the database stops due to some unplanned event, such as a power loss? That’s where command logging can help.

When you enable command logging, the database not only performs periodic snapshots, it also keeps a log of all “commands” (i.e. procedure invocations) that are initiated at each partition. If the database stops for any reason — either intentionally or due to system failure — when the server process restarts, the database can “replay” the command log to recover all of the data committed prior to the cluster shutting down.

The command log in VoltDB consists of stored procedure invocations and their parameters. A log is created at each node, and each log is replicated because all work is replicated to multiple nodes. This results in a replicated command log that can be de-duped at replay time. Because VoltDB transactions are strongly ordered, the command log contains ordering information as well. Thus the replay can occur in the exact order the original transactions ran in, with the full transaction isolation VoltDB offers. Since the invocations themselves are often smaller than the modified data, and can be logged before they are committed, this approach has a very modest effect on performance. This means VoltDB users can achieve the same kind of stratospheric performance numbers, with additional durability assurances.

We’ll be doing some additional blog posts on this feature over the next weeks.  We’ll also be making a pre-release version of VoltDB command logging available to a limited number of users in a week or so, with the objective of getting feedback.  If you are interesting in testing the pre-release version of VoltDB that includes command logging, please send an email to support@voltdb.com and put “Command Logging” in the subject line.

- Bobbi Heath
VP of Engineering