Planning Guide

contents

4.2. Planning for Server Distribution

The preceding sections explain how to calculate the total memory required for storing the database content and indexes. However, at least four other aspects of your configuration contribute to how you calculate memory usage:

  • The server process memory requirements

  • The number of servers in the cluster

  • K-Safety

  • Special buffers for rejoin and overflow

4.2.1. Server Process Memory Usage

On a single server, the memory for a VoltDB database is the memory required for the content plus the memory required for the server process itself. The majority of the memory required for the Java server process is defined by the Java max heap that is set using the -Xmx argument when starting the server process (or defined in the Database Configuration dialog in the VoltDB Enterprise Manager).

 Memory per server = java max heap + database content

Using the recommended max heap of one gigabyte, and our estimated database volume of 500 megabytes, our example Flight database would require a minimum of 1.5 gigabytes of memory. Keeping in mind that memory usage should be kept under 75% of total memory, this means that at least 2 gigabytes should be allowed for running the VoltDB database (beyond what the operating system and other processes may require).

4.2.2. Memory Usage in Clusters

Both capacity and throughput can be increased by running a VoltDB database on a cluster. Although it is tempting to simply divide the total memory consumption by the number of servers, this is not an accurate formula for two reasons:

  • Not all data is partitioned. Replicated tables (and their indexes) appear on all servers.

  • Few if any partitioning schemes provide perfectly even distribution. It is important to account for some variation in the distribution.

To accurately calculate the memory usage per server in a cluster, you must account for the server process, all replicated tables and indexes, and the server's portion of the partitioned tables and indexes.

 Memory per server = java max heap + replicated tables + (partitioned tables/number of servers)

Using the sample sizing for the Flight database described in Section 4.1.3, “An Example of Database Sizing”, the total memory required for the replicated tables and indexes (for the Flight and Airport tables) is only approximately 12 megabytes. The memory required for the remaining partitioned tables and indexes is approximately 490 megabytes. Assuming the database is run on two servers, the total memory required for each server is approximately 1.25 gigabytes:

       1,000   Java max heap (in MB)
          12   Replicated data
    2          Number of servers
  490    245   Paritioned data total / per server
       1,257   Total

4.2.3. K-Safety

The features you plan to use with a VoltDB database can also impact capacity planning, most notably K-Safety. K-Safety improves availability by replicating data across the cluster, allowing the database to survive individual node failures.

Because K-Safety involves replication, it also increases the memory requirements for storing the replicated data. Perhaps the easiest way to size a K-Safe cluster is to size a non-replicated cluster, then multiply by the K-Safety value plus one.

For example, let's assume you plan to run a database with a K-Safety value of 2 (in other words, three copies) on a 6-node cluster. The easiest way to determine the required memory capacity per server is to calculate the memory requirements for a 2-node (non K-Safe) cluster, then create three copies of that hardware and memory configuration.

4.2.4. Special Buffers for Rejoin and Overflow

The previous sections describe memory usage under normal operating conditions. In special cases where nodes fail and rejoin or network problems cause other processes to buffer their output, VoltDB must accommodate this additional memory usage. Special conditions that can affect memory usage in the Java heap include the following:

  • Rejoin — When a node fails and rejoins a high availability cluster, the rejoining node uses additional memory to buffer the incoming data for the partitions it must recreate. The rejoin buffer can use up to 128MB per partition during the rejoin process.

  • Export — If network issues or slow response from the export target result in the export process not keeping up with the queued export data, VoltDB first buffers the output in memory before overflowing to disk. Export buffering can use up to 4MB per partition.

  • Data Replication — If the database is being replicated (that is, it is a master that a DR agent is connected to) and either the replica fails, the DR agent stops, of network issues cause the replication process to not keep up, the cluster will buffer DR data in memory. Each node can buffer up to 128MB per partition before data replication is stopped on the master.

This means that if any of these features are enabled, you must account for the the overflow buffers in your memory calculations. In particular, memory used for rejoin and for data replication overflow consumes space in the Java heap. You must account for this variability to avoid running out of memory in the Java process.

For example, if you are using data replication on a master cluster with 8 partitions per node (that is sitesperhost=8) and replication stops, it is possible that DR overflow can consume up to a gigabyte (8X128MB) of the Java heap before replication queuing stops. It is also possible that, if the master cluster is K-safe, a node might fail and then rejoin the cluster. In that case, the rejoining node may also require a gigabyte to buffer the incoming data. However, any single node will never perform these two actions together. So you need only account for the overflow once, whether the cluster is configured for K-safety, data replication , or both.

This overflow memory should be added to your calculations of both total memory for the server and for the sizing of the maximum Java heap size.

contents