Skip directly to content

Performance

877,000 TPS with Erlang and VoltDB

Friday, April 5, 2013 - 12:00am

written by Henning Diedrich on April 5, 2013 

-Edited 5/2/13 by Henning Diedrich to correct configuration typos.

Running on a suitable EC2 configuration (see details below), with our new VoltDB Erlang driver we achieved 877,519 transactions per second.

I am Henning Diedrich [1], CEO of Eonblast Corporation[2] a games company. I would like to introduce the new Erlang VoltDB driver we created, a piece of software that allows two genre-defining technologies to work together: VoltDB [3] and Erlang [4].

The Driver

I first came to VoltDB on the hunt for a better database for heavy duty online-game

Announcing VoltDB v3.0 — BETA

Friday, November 9, 2012 - 12:00am

written by John Piekos on November 9, 2012 

The VoltDB Engineering Team is excited to announce the availability of the VoltDB v3.0 BETA.
The following provides a brief overview and details on how you can download this very important release.
What is VoltDB v3.0?

VoltDB v3.0 offers a set of user-visible features, including new SQL, indexable column functions, improved ad hoc SQL execution performance, export enhancements, online schema changes, and a more streamlined application-development process.

Announcing VoltDB Version 2.5

Tuesday, April 10, 2012 - 2:30pm

written by John Piekos on April 10, 2012 

I’m very happy to announce the release of VoltDB 2.5.  Consistent with prior releases, 2.5 includes one major new feature, Database Replication, combined with other significant work in and around the VoltDB core engine.  I’d summarize our 2.5 engineering investments as follows:

  • Database Replication.  As I’d previously described here, Database Replication is the headline feature of 2.5 (until recently, we referred to the feature as WAN replication).  It allows VoltDB databases to be automatically replicated within and across data centers.

Debunking Myths about the VoltDB in-memory database

Monday, May 12, 2014 - 12:00am

I’d like to take a quick moment to address some myths and misconceptions about VoltDB. Many people selling products who view VoltDB as competition seem to be repeating them. As you’ll read, much of what’s said is just plain FUD.

 

VoltDB is an in-memory database that has benchmarked at over 3 million transactions a second on bare metal, and recently crushed previous performance records in the cloud, posting eye-popping YCSB (Yahoo! Cloud Service Benchmark) numbers on AWS, Amazon’s cloud platform.

Don’t Accept Data Loss when Upgrading to In-memory Database Performance

Tuesday, May 20, 2014 - 12:00am

Data loss is the enemy of businesses everywhere. Snapshots, replication, active-active architectures, even physical measures such as backup generators are used to ensure data is preserved as it enters and is used or warehoused by the enterprise. Most companies use a database to manage this data and extract intelligence from it; therefore, the database has an important role in preservation of data – durability in database-speak.

 

As data wends its way through ingestion, transactions and analysis, many databases rely on frequent writes to disk to ensure data is durable, even in the event of

In-memory database sizing – throw out conventional wisdom

Monday, June 2, 2014 - 12:00am

written by John Piekos on June 2, 2014 

Sizing an in-memory database does not follow conventional database sizing rules.

For traditional databases, you buy a decent server machine, likely one with many CPU cores and reasonable memory, and then focus on application IOPS (I/O Operations per Second). If you are really going to stress the database, you must choose disks that can support the I/O needs of your application, today and in the future. Because these systems often use many disks to achieve high I/O performance, capacity is usually an afterthought.

With in-memory databases, throw out

Key Value Benchmark FAQ

Tuesday, June 1, 2010 - 8:00pm

written by John Hugg on June 1, 2010

This is a follow up to a previous post on Benchmarking VoltDB against Cassandra on Key-Value-like workloads.

 

What’s the point of this benchmark?

 

Point 1: Demonstrate SQL can be fast. Say what you want about our numbers and benchmark, but the language used to manipulate data was SQL and it clearly wasn’t bottlenecking VoltDB performance. We wanted to show the assumption that dropping SQL is a precondition for performance and scalability is false.

Point 2: VoltDB competes well on simple workloads like the key-value puts and gets, as well as complex

Leaderboards: Optimization for Ranking-Related Queries

Friday, October 5, 2012 - 12:00am

written by Douglas Patten on October 5, 2012

My name is Xin Jia and I was one of the student interns at VoltDB this summer. For one of my projects, I worked on a feature that greatly improved the performance of ranking-related queries.

Finding the rank of an entry in a sorted list of data is a common operation in a lot of applications. Take a leaderboard within gaming application as an example: it is common to have a scoreboard that keeps track of users’ scores. Questions like, “what are the top-n users and their corresponding scores?”, or “what is the rank of a certain user?” are often

Optimizing Distributed Read Operations in VoltDB

Wednesday, August 3, 2011 - 12:00am

written by Ryan Betts on August 3, 2011

Many VoltDB applications, such as gaming leader boards and real-time analytics, use multi-partition procedures to compute consistent global aggregates (and other interesting statistics).  It’s challenging to efficiently process distributed reads operations, especially for performance sensitive applications.  Based on feedback from our users, we in VoltDB engineering have been enhancing the VoltDB SQL planner over the last few releases to improve this capability.

Executing global aggregates efficiently requires calculating sub-results at each partition

Picture it: Three Nodes, Highly Available Cluster, ~1 Million Transactions/second – with VoltDB

Tuesday, June 4, 2013 - 12:00am

written by Philip Rosegay on June 4, 2013

You’ve probably heard about the VoltDB, the super fast distributed ACID SQL RDBMS for OLTP, but you might not be aware of its throughput capabilities in detail. VoltDB achieves its high throughput by eliminating the locking and latching of conventional databases. It’s also distributed and can automatically shard your data, and it has a bunch of other really cool features that make transaction processing a snap.

We took a key­-value application and implemented it in VoltDB (the app is available with the distribution if you want to try it). We set the

Programming VoltDB – Easy, Flexible and Ultra-fast!

Monday, August 6, 2012 - 12:00am

written by John Piekos on August 6, 2012

You may believe that the only way to interact with VoltDB is through Java stored procedures.  To achieve maximum throughput, VoltDB stored procedures is the way to go.  You can achieve upwards of 100,000 transactions per second on a single node.  However, you can also achieve significant throughput by interacting with VoltDB conversationally, through ad hoc SQL statements, avoiding the need to pre-compile stored procedures.

This blog will discuss several approaches to interacting with VoltDB programmatically and cover the performance, in terms of

Q&A from My Talk at the Hamburg Javascript Meet-up

Friday, June 22, 2012 - 12:00am

written by Henning Diedrich on June 22, 2012

On June 18 I gave a talk to the Javascript meet-up group in Hamburg, Germany.  It was a very enjoyable evening.   I’d like to thank Martin Kleppe for organizing the event, and the meet-up group members for investing an evening to learn more about Node.js and VoltDB.

The group asked some interesting questions during the Q&A part of my talk and in the additional discussions afterward.  Below are some of the questions I remember, along with written responses.

Q:

What are the primary differences between VoltDB Community Edition (open

Scaling with VoltDB: The Clustered Database

Wednesday, January 9, 2013 - 12:00am

written by John Piekos on January 9, 2013

This is part 2 (of 2) of my Programming VoltDB – Easy, Flexible and Ultra-fast series. Blog post, part 1, showed how to build a VoltDB application using ad hoc queries and achieving thousands of transactions a second. It also showed how converting that logic to use VoltDB stored procedures allowed you to parallelize query execution and achieve 100,000+ transactions a second on a single node. In this blog post I’ll talk about scaling beyond 100,000 transactions per second by creating a VoltDB clustered database.

There are primarily two reasons why you

The Lifecycle of a Transaction

Thursday, June 10, 2010 - 12:00am

written by John Hugg on June 10, 2010

When learning about VoltDB for the first time, people often ask how VoltDB executes transactions in a distributed environment. So, here’s how…

Let’s say a developer, Dan, is working on a new website. He’s decided to use VoltDB as part of his data management layer. One of Dan’s users requests a dynamically generated page and the code that generates that page sends a request to Dan’s VoltDB cluster. The excitement begins.

Dan’s code asks the VoltDB client library to execute his procedure named “GetUserProfile” and provides the user’s handle, “dbNerd”, as a

Transactions on a Budget

Friday, September 24, 2010 - 12:00am

written by Tim Callaghan on September 24, 2010

We hand-built a “cluster” at VoltDB to perform a variety of long running tests. In this test cluster, one of the nodes performs all client application activity while the other 5 serve as VoltDB servers. The cost of the entire setup was under $3500 (six machines with 8GB RAM, 2 UPSs, and an 8-port switch). I’ll provide more technical details of the cluster hardware in a future blog entry.

After moving into our new office I kicked off my favorite test suite that includes a data generator, an export client (writing data to the file system), and a

Understanding VoltDB Data Partitioning

Monday, June 27, 2011 - 12:00am

written by Ryan Betts on June 27, 2011 

I’ve talked to hundreds of people about VoltDB over the past year. Data partitioning and its consequences are almost always the hardest parts to explain.

VoltDB is a clustered database. Each node of the cluster contains a subset of the data in the database. People seem to understand this quickly. However, explaining how data is distributed, how and when it can be accessed efficiently, and what operations are supported within and across data partitions is harder to describe succinctly.

Upserts in VoltDB

Friday, November 16, 2012 - 12:00am

written by Andrew Wilson 

 

The idea behind an upsert is that you try an update or an insert query first and if it fails, you then do the other query.

Why do an upsert? Upserts tend to be very fast in traditional databases because they can execute in as little as one query or as many as two. Consequently, a good upsert strategy has only two defined queries (insert and update) rather than three (insert, update and select). More importantly, “upsert” itself can be a keyword in which the database understands that it is responsible for figuring out whether to update or insert a record.

Using Node.js with VoltDB

Wednesday, May 9, 2012 - 12:00am

written by Andrew Wilson on May 9, 2012

We recently released the first supported version of a VoltDB client driver for Node.js applications. It was able to execute over 695K transactions per second. Since then we have been working on building a better driver and sought out help from the Node community and made a series of improvements.

Today I’m going to write about how to access VoltDB from a Node application. A sample application is included with the driver, the latest version of which can be downloaded from http://www.voltdb.com/tao-volt/downloads-home.php.

The integration process is short

VoltDB client for the Go Language

Monday, October 29, 2012 - 12:00am

written by Ryan Betts on October 29, 2012 

A while ago I published my Go VoltDB driver to github (https://github.com/rbetts/voltdbgo).  I wrote the driver for three reasons: to learn and write a little go; to be able to test and script against VoltDB using go; and to experiment with some different VoltDB client patterns.

We, and the community, have written several production quality drivers for VoltDB (available at https://www.voltdb.com/download). The go driver, however, has not been tested for production use.

With caveats complete, what’s up with voltdbgo?

Firstly, it only supports

VoltDB Community Users Share Knowledge

Monday, May 14, 2012 - 12:00am

written by Scott Jarr on May 14, 2012

VoltDB adoption and deployments are accelerating.  We’re working hard make our products easy to use in both cloud and privately-racked infrastructures.  We see strong adoption in five application categories – capital markets (mostly around trading systems), digital advertising, online games, network monitoring, and intelligence/surveillance (national security, fraud mitigation, etc.).

With product evaluations on the rise, organizations are increasingly asking about user experiences, particularly for deployed VoltDB applications.  There are some strong

VoltDB Explain Plan Command and Planner Testing Tool

Wednesday, September 12, 2012 - 12:00am

written by Douglas Patten on September 12, 2012

My name is Zheng Li, a UMass Lowell graduate student. I spent the summer as an internship at VoltDB. Over the summer, I primarily worked on two VoltDB features, both related to query plans. The features are:

  1. Explain plan command
  2. Planner testing tool

query plan is an ordered set of steps used to access or modify information in a SQL relational database management system (see http://en.wikipedia.org/wiki/Query_plan).  Understanding query plans is important because the first plan chosen to execute will directly affect the query execution

VoltDB in-memory database achieves best-in-class results, running in the cloud, on the YCSB Benchmark

Wednesday, May 7, 2014 - 9:00am

The development team at VoltDB recently ran VoltDB v4.2 against the Yahoo Cloud Serving Benchmark (YCSB), an industry-standard performance benchmark for cloud databases. We ran our test on as realistic a cluster setup as possible: commodity hardware that we could easily book as spot instances on EC2, with features like durability and high availability enabled. And we ran a varied mix of workloads with realistically sized, 1 KB rows, rather than focusing on a specific, overly favorable use case.

VoltDB – Simplified CSV Loader

Thursday, August 23, 2012 - 2:30pm

written by Douglas Patten on August 23, 2012

My name is Xin Jia, a Brown University student, and I’ve spent the summer interning at VoltDB. One of the first projects that I worked on this summer was a CSV Loader with fellow intern Zheng Li.

Prior to VoltDB 2.8, loading data from CSV files into the database could be a challenging task, especially for newcomers. There were a few steps that needed to be taken: you would need to write a client program to parse the CSV data, handle errors and ultimately invoke a stored procedure to insert the data into the database.

VoltDB’s End of Summer Engineering Update

Tuesday, August 28, 2012 - 12:00am

written by John Piekos on August 28, 2012

It’s been a busy summer here at VoltDB! Since spring the VoltDB Engineering team has released VoltDB four times (we’re operating on 3 week sprints) and delivered numerous product and performance enhancements.

Some of the major features we’ve recently released include:

  • Pause-less rejoin of failed nodes. Failed nodes can now be rejoined to the k-safe cluster without significant impact to the operational throughput of the cluster. This feature is available in the Enterprise edition of VoltDB.

What? VoltDB Stored Procedures?

Thursday, July 1, 2010 - 12:00am

written by Ryan Betts on July 1, 2010 

VoltDB is a relational store that uses SQL for its query language and stored procedures for its transactional unit of work. Clients invoke stored procedures to access the database. Stored procedures access data using parametrized SQL statements and can manipulate intermediate results in Java. Each stored procedure invocation is a transaction. A procedure’s data manipulations are atomic and isolated from other concurrently executing procedures.

Here’s a simple example, excerpted from the Voter sample application shipped with the VoltDB distribution kit.

What’s the plan?

Tuesday, May 7, 2013 - 12:00am

written by Ruth Morgenstein on May 7, 2013

“Why is this so slow?” Have you put your application into testing (you did this before going to production, right?) and wondered why you’re not getting VoltDB’s world-class performance? The problem might be with the SQL execution plan.

This article shows you how to look at the SQL execution plans and use the information to tune your application.

Getting the plans at compile time

In VoltDB, you can get execution plan information when you compile your stored procedures or later, when the database is up and running. When you build the application

Why is VoltDB So Fast?

Thursday, January 13, 2011 - 12:00am

written by John Hugg on January 13, 2011 

First and foremost, VoltDB is focused on specific workloads. Most existing RDBMSs are designed to be general purpose, one-size-fits-all systems. Recently, there have been a lot of new databases introduced that achieve better performance by specializing in areas like analytics, graphs or streaming data. Few of these specialized systems focus OLTP, and when they do, it’s often more about tuning, rather then a rethink.

Writing VoltDB Apps in Java Q & A

Friday, October 5, 2012 - 12:00am

written by Andrew Wilson on October 5, 2012

NewSQL and NoSQL databases present developers with a new and interesting programming model where there are fewer connections to the database, query parallelism, partitions and an easy model for supporting new clusters. Last month I presented a webinar introducing the basics of application development using VoltDB. We had a lot of great questions and I thought it would be handy to write them up and share the answers with everyone.

Q: Can you address how you would handle joining tables that have different partition keys?