Connecting VoltDB to the Big Data Ecosystem

written by John Piekos on June 23, 2014 with no comments

With its ability to act on data coming in at the rate of hundreds of thousands to millions of events a second, VoltDB is ideally suited as an operational database to process fast data.  Fast data creates Big Data, so moving data out of VoltDB becomes very important once that data has been processed and is no longer of immediate value. To move data out of VoltDB, use VoltDB Export. VoltDB Export allows you to transactionally push data from VoltDB into another system, similar to an ETL (extract, transform, load) process.

VoltDB lets you automate the export process by specifying certain tables in the schema as sources for export. At runtime, any data written to the specified tables is sent to the export connector, which queues the data for export, then sends it to the selected output target.

Release v4.4 of VoltDB supports the following Export Connectors:

    • CSV (flat file)
    • Kafka
    • JDBC
    • Netezza
    • Vertica

We’re rapidly building out new Export Connectors to integrate with other downstream systems.  Look for more connectors in the coming months. In the interim, if you need a specific export target, please contact us with your requirements at info@voltdb.com.  Or, if you’d like to build your own Export Connector, you’ll find it’s easy! See our recent Dev Center post for the steps as well as a basic code example.