Announcing VoltDB 4.0: Enhanced In-Memory Analytics and Online Elasticity
The VoltDB engineering team is pleased to announce that VoltDB 4.0 is now available!
Bruce Reading, the VoltDB CEO is fond of saying that VoltDB contains lots of “yummy goodness”, and, while that is not a term we engineers use often, VoltDB v4.0 does indeed include a lot of new features. The highlights of VoltDB v4.0 include:
- Enhanced in-memory analytics capabilities with a host of new SQL support.
- Greatly improved analytic read throughput performance.
- Clusters can grow elastically, increasing both throughput and capacity, by adding nodes to running clusters without blocking ongoing operations.
- Support for Groovy stored procedures, a message queue export connector, a MySQL migration utility and a host of other features.
- Online training, free, at Volt University, along with Volt Vanguard Certification.
Here’s the details on what’s new in VoltDB v4.0. For the crowd, download it here. For everyone else, read on…
Enhanced In-Memory Analytics
VoltDB is renown for its ability to execute very fast writes – we’ve benchmarked writes into the millions of transactions per second range, on small clusters, running on bare metal as well as cloud instances.
But fast writes without fast reads are less useful. Since its first version, VoltDB has allowed for transactional reads to support writes as well as provide a window into fast changing data.
At ingestion or processing time, stored procedures can transactionally perform lookups and queries as data is coming into the system, allowing for richer writes at scale. Separately, global transactional reads can trigger events, support dashboards and even live decisioning on immediate data. Mixing complex reads and writes transactionally, and at scale, has traditionally separated VoltDB from other write-heavy systems.
In 4.0, VoltDB has added both features and improved performance of analytic-focused read queries. We’re focused on helping users understand their data as soon as they have it.
First, VoltDB delivers major new SQL capabilities, now supporting SQL UNION, self/outer/explicit JOIN, CASE, HAVING, SQL IN, Group-by column functions and materialized view group-by column functions. Our SQL support is approaching SQL-92 compatibility, while also adding non-standard features to support our key use cases. For example, VoltDB can now build a materialized view that aggregates the value of a JSON field by 5-minute time windows.
Second, we’ve removed some of the transactional overhead when running many kinds of global read queries, including ad-hoc SQL. These queries are still reading a live, fully serializable view of committed data, but they’re now up to 50x faster. This directly translates into more powerful dashboards, more consumers of analytics and richer decisioning.
Online Elastic Database
As a natively clustered database, VoltDB can scale to meet the needs of almost any high-velocity application. While some users have megabytes of state, others have terabytes. While some users process hundreds of operations per second, others process millions. But what if your business is growing and you want your VoltDB cluster to grow with it? Since we shipped VoltDB 1.0, users have been asking to add nodes to the cluster without any interruption in service, to rebalance data in the background while their apps continue to work for them.
We call this feature “elasticity” and it’s shipping in VoltDB 4.0. VoltDB can now seamlessly add nodes to a running cluster, increasing storage and throughput with each new node. Since all topology changes and data movement are transactional and durable, your data is protected while the cluster is expanding or rebalancing. We’ve also carefully engineered this feature so that most customers will see no impact to their workload during expansion.
In addition to the new online elasticity, VoltDB also supports online catalog and schema updates. Tables can be added or dropped. Columns, indexes and materialized views can be added, dropped and modified. And of course, stored procedures can be added, updated, or deleted.
VoltDB upgraded its network partition and fault detection handling. In the real world, network partitions happen for a variety of reasons. Both hardware and software issues can cause a node to become unavailable, or unreachable to other nodes in a cluster. For version 4.0, VoltDB enhanced it’s fault detection and recovery functionality to remain available in the face even more kinds of failure. The result is increased availability of VoltDB clusters where links might break, for example, when VoltDB nodes are running in different availability zones.
Groovy – Our First Non-Java Stored Procedure Language
In VoltDB 4.0 we’ve added the ability to code the procedure implementation in the DDL itself with inline Groovy scripts. Check out our Groovy Voter sample for a familiar example of VoltDB processing in Groovy. We hope this is the first of many new stored procedure languages – please drop us a note if you have a favorite, or would like to contribute to this effort.
Integrations, Migrations and Miscellaneous Features
VoltDB v4.0 introduces a whole host of other features. I’ll run through them here quickly:
- VoltDB Export allows you to transactionally push data from VoltDB into another system, similar to an ETL (extract, transform, load) process. In 3.x, we could export to systems using JDBC, as well as to flat files. VoltDB 4.0 adds a new Export connector to leverage message queues. Presently available as a Beta, you can now export to a message queue using Kafka. Feel free to contact us if you’d like early access.
- We’ve added three new @Statistics selectors that can help identify performance hot spots in your application. They are PROCEDUREPROFILE, identifying the percentage of execution time each procedure takes, PROCEDUREINPUT, identifying the breakdown of data flow into stored procedures, and PROCEDUREOUTPUT, identifying the result set data flow from stored procedures. These new statistics help you to quickly identify what transactions are taking the most time or I/O in your application.
- You can now get a leg up porting your MySQL to VoltDB. Our new utility, fondly called Voltify, will extract your database schema from your MySQL database and create a VoltDB catalog automatically. This utility, coupled with our high performance CSV loader, enables you to rapidly move your MySQL database to VoltDB.
- The VoltDB JDBC driver has been enhanced to support parameterized ad hoc SQL statements, setting query timeouts, as well as additional metadata methods.
- We’ve partnered with New Relic and now offer a new system monitoring plugin supporting the New Relic SaaS monitoring platform. The plugin can be found at https://rpm.newrelic.com/accounts/382109/plugins/directory/133
VoltDB recently rolled out a new offering, Volt University, offering free online training on VoltDB key concepts. There are ten lessons, found at http://voltdb.com/resources/volt-university/tutorials/ that can help you come up to VoltDB-speed quickly. Additionally, should you wish formal Volt Vanguard certification, we are offering an online official certification course, which you can register for here: https://university.voltdb.com/
Download VoltDB 4.0
We here at VoltDB are very excited about the release and hope you are too. You can download VoltDB 4.0 here.
VoltDB is used in production in hundreds of customer sites across a wide range of applications, from financial market applications, mobile billing, sensor data processing, digital ad tracking and optimization to online multi-player gaming platforms. Reach us at email@example.com and let us know how we can help you to build your next generation application based on VoltDB.