Using VoltDB

V3.2.1

This document and the software it describes is licensed under the terms of the GNU Affero General Public License Version 3 as published by the Free Software Foundation.

VoltDB is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License (http://www.gnu.org/licenses/) for more details.

Abstract

This book explains how to use VoltDB to design, build, and run high performance applications.


Table of Contents

Preface
1. Overview
1.1. What is VoltDB?
1.2. Who Should Use VoltDB
1.3. How VoltDB Works
1.3.1. Partitioning
1.3.2. Serialized (Single-Threaded) Processing
1.3.3. Partitioned vs. Replicated Tables
1.3.4. Ease of Scaling to Meet Application Needs
2. Installing VoltDB
2.1. Operating System and Software Requirements
2.2. Installing VoltDB
2.2.1. Upgrading an Existing VoltDB Installation
2.2.2. Performing a System-Wide Installation on Ubuntu
2.2.3. Building a New VoltDB Distribution Kit
2.3. Setting Up Your Environment
2.4. What is Included in the VoltDB Distribution
2.5. VoltDB in Action: Running the Sample Applications
3. Designing Your VoltDB Application
3.1. Designing the Database
3.1.1. Partitioning Database Tables
3.1.2. Replicating Tables
3.2. Designing the Data Access (Stored Procedures)
3.2.1. Writing VoltDB Stored Procedures
3.2.2. VoltDB Stored Procedures and Determinism
3.2.3. The Anatomy of a VoltDB Stored Procedure
3.2.4. Partitioning Stored Procedures
3.3. Designing the Application Logic
3.3.1. Connecting to the VoltDB Database
3.3.2. Invoking Stored Procedures
3.3.3. Invoking Stored Procedures Asynchronously
3.3.4. Closing the Connection
3.4. Handling Errors
3.4.1. Interpreting Execution Errors
3.4.2. Handling Timeouts
3.4.3. Interpreting Other Errors
4. Simplifying Application Development
4.1. Default Procedures for Partitioned Tables
4.2. Shortcut for Defining Simple Stored Procedures
4.3. Verifying Expected Query Results
5. Building Your VoltDB Application
5.1. Compiling the Client Application and Stored Procedures
5.2. Declaring the Stored Procedures
5.3. Building the Application Catalog
6. Running Your VoltDB Application
6.1. Defining the Cluster Configuration
6.1.1. Determining How Many Partitions to Use
6.1.2. Configuring Paths for Runtime Features
6.1.3. Verifying your Hardware Configuration
6.2. Starting a VoltDB Database for the First Time
6.2.1. Simplifying Startup on a Cluster
6.2.2. How VoltDB Database Startup Works
6.3. Starting VoltDB Client Applications
6.4. Shutting Down a VoltDB Database
6.5. Stopping and Restarting a VoltDB Database
6.5.1. Save and Restore
6.5.2. Command Logging and Recovery
6.6. Modes of Operation
6.6.1. Admin Mode
6.6.2. Starting the Database in Admin Mode
7. Updating Your VoltDB Database
7.1. Planning Your Application Updates
7.2. Updating the Database Schema on a Running Database
7.2.1. Validating the Updated Catalog
7.2.2. Managing the Update Process
7.3. Updating the Database Using Save and Restore
7.4. Updating the Hardware Configuration
8. Security
8.1. How Security Works in VoltDB
8.2. Enabling Authentication and Authorization
8.3. Defining Users and Roles
8.4. Assigning Access to Stored Procedures
8.5. Allowing Access to System Procedures, Ad Hoc Queries, and Default Procedures
9. Saving & Restoring a VoltDB Database
9.1. Performing a Manual Save and Restore of a VoltDB Cluster
9.1.1. How to Save the Contents of a VoltDB Database
9.1.2. How to Restore the Contents of a VoltDB Database
9.1.3. Changing the Database Schema or Cluster Configuration Using Save and Restore
9.2. Scheduling Automated Snapshots
9.3. Managing Snapshots
9.4. Special Notes Concerning Save and Restore
10. Command Logging and Recovery
10.1. How Command Logging Works
10.2. Enabling Command Logging
10.3. Configuring Command Logging for Optimal Performance
10.3.1. Log Size
10.3.2. Log Frequency
10.3.3. Synchronous vs. Asynchronous Logging
10.3.4. Hardware Considerations
10.4. Recovery Options in the VoltDB Community Edition
11. Availability
11.1. How K-Safety Works
11.2. Enabling K-Safety
11.2.1. What Happens When You Enable K-Safety
11.2.2. Calculating the Appropriate Number of Nodes for K-Safety
11.3. Recovering from System Failures
11.3.1. What Happens When a Node Rejoins the Cluster
11.3.2. Rejoining a "Live" Cluster (Enterprise Edition Feature)
11.3.3. Where and When Recovery May Fail
11.4. Avoiding Network Partitions
11.4.1. K-Safety and Network Partitions
11.4.2. Using Network Fault Protection
12. Database Replication
12.1. How Database Replication Works
12.1.1. Starting Replication
12.1.2. Replication and Existing Databases
12.1.3. Database Replication and Disaster Recovery
12.1.4. Database Replication and Completeness
12.1.5. Database Replication and Read-only Clients
12.2. Database Replication in Action
12.2.1. Starting Replication
12.2.2. Stopping Replication
12.2.3. Promoting the Replica When the Master Becomes Unavailable
12.2.4. Managing Database Replication
12.3. Using the Sample Applications to Demonstrate Replication
12.3.1. Replicating the Voter Sample Using the Enterprise Manager
12.3.2. Replicating the Voter Sample Using the Command Line
13. Exporting Live Data
13.1. Understanding Export
13.2. Planning your Export Strategy
13.3. Identifying Export Tables in the Schema
13.4. Configuring Export in the Deployment File
13.5. How Export Works
13.5.1. Export Overflow
13.5.2. Persistence Across Database Sessions
13.6. Using the Export Clients
13.6.1. Running the Export Client on the Database Server
13.6.2. Running the Export Client Remotely
13.7. The Export-to-JDBC Client (Enterprise Edition Only)
13.8. The Export-to-File Client
13.8.1. Understanding the Export-to-File Client Output
13.8.2. The Export-to-File Client Command Line
13.9. The Export-to-Hadoop Client (Enterprise Edition Only)
13.9.1. The Export-to-Hadoop Client Command Line
14. Logging and Analyzing Activity in a VoltDB Database
14.1. Introduction to Logging
14.2. Creating the Logging Configuration File
14.3. Enabling Logging for VoltDB
14.4. Customizing Logging in the VoltDB Enterprise Manager (Enterprise Edition Only)
14.5. Changing the Configuration on the Fly
15. Using VoltDB with Other Programming Languages
15.1. C++ Client Interface
15.1.1. Writing VoltDB Client Applications in C++
15.1.2. Creating a Connection to the Database Cluster
15.1.3. Invoking Stored Procedures
15.1.4. Invoking Stored Procedures Asynchronously
15.1.5. Interpreting the Results
15.2. JSON HTTP Interface
15.2.1. How the JSON Interface Works
15.2.2. Using the JSON Interface from Client Applications
15.2.3. How Parameters Are Interpreted
15.2.4. Interpreting the JSON Results
15.2.5. Error Handling using the JSON Interface
15.3. JDBC Interface
15.3.1. Using JDBC to Connect to a VoltDB Database
15.3.2. Using JDBC to Query a VoltDB Database
A. Supported SQL DDL Statements
CREATE INDEX — Creates an index for faster access to a table.
CREATE PROCEDURE AS — Defines a stored procedure composed of a SQL query.
CREATE PROCEDURE FROM CLASS — Defines a stored procedure associated with a Java class.
CREATE ROLE — Defines a role and the permissions associated with that role.
CREATE TABLE — Creates a table in the database.
CREATE VIEW — Creates a view into a table, used to optimize access to specific columns within a table.
EXPORT TABLE — Specifies that a table is for export only.
PARTITION PROCEDURE — Specifies that a stored procedure is partitioned.
PARTITION TABLE — Specifies that a table is partitioned and which is the partitioning column.
B. Supported SQL Statements
DELETE — Deletes one or more records from the database.
INSERT — creates a new row in the database, using the specified values for the columns.
SELECT — fetches the specified rows and columns from the database.
UPDATE — updates the values within the specified columns and rows of the database.
C. SQL Functions
ABS() — Returns the absolute value of a numeric expression.
ARRAY_ELEMENT() — Returns the element at the specified location in a JSON array.
ARRAY_LENGTH() — returns the number of elements in a JSON array.
AVG() — Returns the average of a range of numeric column values.
CAST() — explicitly converts an expression to the specified datatype.
CEILING() — Returns the smallest integer value greater than or equal to a numeric expression.
CHAR_LENGTH() — Returns the number of characters in a string.
CONCAT() — Concatenates two strings and returns the result.
COUNT() — Returns the number of rows selected containing the specified column.
DECODE() — Evaluates an expression against one or more alternatives and returns the matching response.
EXP() — Returns the exponential of the specified numeric expression.
EXTRACT() — Returns the value of a selected portion of a timestamp.
FIELD() — Extracts a field value from a JSON-encoded string column.
FLOOR() — Returns the largest integer value less than or equal to a numeric expression.
LEFT() — Returns a substring from the beginning of a string.
MAX() — Returns the maximum value from a range of column values.
MIN() — Returns the minimum value from a range of column values.
OCTET_LENGTH() — Returns the number of bytes in a string.
POSITION() — Returns the starting position of a substring in another string.
POWER() — Returns the value of the first argument raised to the power of the second argument.
REPEAT() — Returns a string composed of a substring repeated the specified number of times.
RIGHT() — Returns a substring from the end of a string.
SPACE() — Returns a string of spaces of the specified length.
SQRT() — Returns the square root of a numeric expression.
SUBSTRING() — Returns the specified portion of a string expression.
SUM() — Returns the sum of a range of numeric column values.
D. VoltDB CLI Commands
csvloader — Imports the contents of a CSV file and inserts it into a VoltDB table.
dragent — Starts the database replication agent (Enterprise Edition feature).
exporttofile — Starts the export client, connects to a database, and writes export data to one or more files.
sqlcmd — Starts an interactive command prompt for issuing SQL queries to a running VoltDB database
voltadmin — Performs administrative functions on a VoltDB database.
voltdb — Compiles or starts the VoltDB database
E. Deployment File (deployment.xml)
E.1. Understanding XML Syntax
E.2. The Structure of the Deployment File
F. System Procedures
@AdHoc — Executes an SQL statement specified at runtime.
@Explain — Returns the execution plan for the specified SQL query.
@ExplainProc — Returns the execution plans for all SQL queries in the specified stored procedure.
@Pause — Initiates admin mode on the cluster.
@Promote — Promotes a replica database to normal operation.
@Quiesce — Waits for all queued export data to be written to the connector.
@Resume — Returns a paused database to normal operating mode.
@Shutdown — Shuts down the database.
@SnapshotDelete — Deletes one or more native snapshots.
@SnapshotRestore — Restores a database from disk using a native format snapshot.
@SnapshotSave — Saves the current database contents to disk.
@SnapshotScan — Lists information about existing native snapshots in a given directory path.
@SnapshotStatus — Lists information about the most recent snapshots created from the current database.
@Statistics — Returns statistics about the usage of the VoltDB database.
@SystemCatalog — Returns metadata about the database schema
@SystemInformation — Returns configuration information about VoltDB and the individual nodes of the database cluster
@UpdateApplicationCatalog — Reconfigures the database by replacing the application catalog currently in use (Enterprise Edition feature).
@UpdateLogging — Changes the logging configuration for a running database.