Understanding VoltDB Data Partitioning

written by Ryan Betts on June 27, 2011 with 5 comments

I’ve talked to hundreds of people about VoltDB over the past year. Data partitioning and its consequences are almost always the hardest parts to explain.

VoltDB is a clustered database. Each node of the cluster contains a subset of the data in the database. People seem to understand this quickly. However, explaining how data is distributed, how and when it can be accessed efficiently, and what operations are supported within and across data partitions is harder to describe succinctly. So I thought it might be helpful to explain partitioning in a blog post.

Here’s my current approach. Feedback welcome!

Start with a single partitioned table

To start, assume there’s just a single, partitioned table in your schema. One column of this table is the user designated partitioning column. VoltDB distributes this table very much like a distributed hash table system (key value store) where each row’s partitioning column value is a key and the row itself is the value. Each row is an object that is assigned to single partition by hashing its partitioning column attribute. As a result, each data partition is assigned a subset of the total rows of the table.

More than one partitioned table

All partitioned tables are distributed using the same approach and the same hash function.

Example: Consider a database with a CUSTOMERS table partitioned on customer_id. Each row of the CUSTOMERS table is a definition of a customer. The database additionally has an ORDERS table also partitioned on customer_id. Each row of the ORDERS table describes a single order for a specific customer. Both tables will be distributed by hashing customer_id.  Consequently, all the orders for a given customer will be mapped to the same data partition as the customer’s CUSTOMERS record. All of the data relevant to a single customer will end up in a single partition.

Another example: Consider a database that has a flights table partitioned on flight_id and a reservations table partitioned on reservation_id. Reservation ids are not flight ids. There may be a flight with id 42 and a reservation with id 42. They will co-reside in the same data partition but they are not logically related to each other.

Replicated tables

A replicated table is simply duplicated in full at each partition. Modifying a replicated table is a global transaction. Reading a replicated table can be done by accessing just one data partition (because each partition has a full, consistent copy). Replicated tables are generally used for infrequently-changing reference data (e.g., product/service codes and descriptions).  By replicating reference data on each partition, you’ll reduce the need for multi-partition operations in your VoltDB application, thereby optimizing throughput and performance.

What does it mean!?

Once you understand how VoltDB distributes data, you can think about which transactions are single partition (need only the data from one partition) and which are global (need data from two or more partitions). VoltDB can access discrete objects (where an object is relationally associated rows residing in a single partition) very rapidly while still supporting less frequent multi-partition queries. Single partition transactions scale – VoltDB can do 100′s of thousands of single partition procedures per second with just a few nodes.

Using the examples above, single partition transactions are things like: total the value of the orders of a single customer, or change a single flight to indicate it has landed.

Multi-partition transactions can be done hundreds to thousands of times per second.  They’re still very fast, but they come at the cost of additional network overhead. Using the examples above, multi-partition transactions are queries like: find the customer with the most open orders, or mark all the reservations for a specific flight as cancelled, or find the customer that should receive order_id 100.

In summary …

Although partitioning a VoltDB database is important to achieving the best performance and scale, it can also be somewhat difficult to grasp initially.  Once you get the hang of it, though, it’s not as difficult as it may initially seem.  I hope this post has helped to shed some light on the subject, and inspires you to jump in and try Volt sometime soon.