What’s the plan?

“Why is this so slow?” Have you put your application into testing (you did this before going to production, right?) and wondered why you’re not getting VoltDB’s world-class performance? The problem might be with the SQL execution plan.… Read more


877,000 TPS with Erlang and VoltDB

-Edited 5/2/13 by Henning Diedrich to correct configuration typos.

Running on a suitable EC2 configuration (see details below), with our new VoltDB Erlang driver we achieved 877,519 transactions per second.

I am Henning Diedrich [1], CEO of Eonblast Corporation[2] a games company.… Read more










Part Four: You’d Better Do Fast Data Right – A Five Step approach

The last post defined what the Corporate Data Architecture of the future will look like and how “Fast” and “Big” will work together.Read more





Cost accounting for SSDs – it’s RAM, not disk

Most discussions I have seen about choosing SSDs vs. spinning disk arrays for databases tend to focus on SSDs as a replacement for disk. SSDs don’t replace disk; they replace the RAM you would be using to cache enough disk pages to make up for the terrible random IO performance of spinning disk arrays.… Read more

written by Ariel Weisberg on July 10, 2014 in Blog and Technical


Part Three: Designing a Data Architecture to Support Both Fast and Big Data

In post one of this series, we introduced the ideas that a Corporate Data Architecture was taking shape and that working with Fast Data is different from working with Big Data.… Read more


The benefits of real-time data via in-memory database technology

Having a lot of data pouring into your organization is one thing. Being able to store it, analyze it and visualize it in real-time is a whole different ball game. More and more organizations need to have real-time insights in order to fully understand what is going on within their enterprises – and with their customers.Read more

written by Mike Pogany on July 1, 2014 in Blog and Business

Big DataCamp LA 2014 Wrap-Up

Amazing. That’s about all I could say all day the Saturday before Father’s Day. I was asked to speak at the LA Big Data Camp (#BigDataCampLA) on that day, which I had happily agreed to before I actually realized it was _that_ weekend.… Read more

written by Scott Jarr on June 30, 2014 in Blog



Connecting VoltDB to the Big Data Ecosystem

With its ability to act on data coming in at the rate of hundreds of thousands to millions of events a second, VoltDB is ideally suited as an operational database to process fast data. … Read more

written by John Piekos on June 23, 2014 in Blog

Creating a custom export connector that runs inside VoltDB

Overview

  1. Implement a new export connector.
  2. Compile the connector and place its .class file in VoltDB’s classpath. ($VOLTDB/lib/extension).
  3. Configure the connector in the deployment file.
  4. Run the server.

Implement a new export connector

An export connector, known internally as an ExportClient, receives blocks of row data from a producer within the database.… Read more










Part One: The Imminent Fracture in Corporate Data Architecture: Fast + Big

written by Scott Jarr on May 28, 2014 in Blog and Business













VoltDB in-memory database achieves best-in-class results, running in the cloud, on the YCSB Benchmark

The development team at VoltDB recently ran VoltDB v4.2 against the Yahoo Cloud Serving Benchmark (YCSB), an industry-standard performance benchmark for cloud databases. We ran our test on as realistic a cluster setup as possible: commodity hardware that we could easily book as spot instances on EC2, with features like durability and high availability enabled.… Read more



M2M World Congress Wrap-Up

I attended M2M World Congress in London two weeks ago and have been pondering my “take-aways” since. While many VoltDB customers use our NewSQL, in-memory database to provide scalable transactions, decisions and analytics in M2M deployments (smart grids, mobile telco platforms, cloud PaaS products), putting the M2M/IoT space into a broader context is hard.… Read more




Impact of Java Garbage Collection on in-memory databases

Some users of Java, and specifically Java as it is used in in-memory database technology, worry Java’s ‘Garbage Collection’ can impact performance. Garbage Collection is intended to simplify memory management (allocation and de-allocation) and therefore reduce the amount of code written, speed development, and avoid memory management bugs.… Read more





Heartbleed and VoltDB: What You Need to Know (Spoiler Alert: VoltDB is Unaffected)

As you’ve no doubt heard, there’s a lot of buzz about a vulnerability in OpenSSL technology commonly referred to as the Heartbleed Bug. It’s present in sites powered by OpenSSL versions 1.0.1 through 1.0.1f, and affects more than 500,000 sites – enabling hackers to capture names and passwords, steal credit card data, and eavesdrop on communications.… Read more




NewSQL: the Cake You Can Eat

Formula for startup success: extract the lessons of the past and mix in present reality and macro-trends. That’s NewSQL in a nutshell.

NewSQL is growing in popularity because it preserves the value accumulated over the last 30 years of database development and deploys that capability on modern architectures and configurations.… Read more



















VoltDB, native memory, and you

(Originally published here.)

Target audience

This post targets other VoltDB developers who are going to be dealing with the various unconventional ways Volt now uses native memory in the Java portions of the database.… Read more

written by Ariel Weisberg on March 3, 2014 in Blog and Technical


Run Everywhere

VoltDB supports single-partition transactions, which as the name implies, run on only
one partition. VoltDB can run as many single partition transactions in parallel as
there are unique partitions. This enables high transaction throughput as well as
high transaction concurrency execution.… Read more







Planned Maintenance Windows

When you need to take the database down for maintenance, this is the recommended approach to do it safely with no data loss using a snapshot.

A snapshot is a point-in-time consistent copy of the entire contents of the database.… Read more

written by vdbdev on January 30, 2014 in Cookbook

Configuring Export

VoltDB’s export feature connects VoltDB to a downstream OLAP system
so that you can use VoltDB for data correlation, data quality, filtering,
real time enrichment – and pass completed records downstream to your
OLAP / HDFS for historical analytics and archival.… Read more

written by vdbdev on January 30, 2014 in Cookbook






















INSTALLING VOLTDB IN LOCAL MAVEN REPOSITORY

This is a topic that has come up quite often as to how to use VoltDB client libraries if I use Maven. Although VoltDB is not part of the Maven Central repository, we can work around this by installing the libraries locally in the local repository for maven.… Read more

written by vdbdev on October 24, 2013 in Cookbook

Real-time Analytics with JSON

This post describes how to combine VoltDB JSON functionality and materialized views
to perform high velocity real-time analytics. For this example, we’ll make some
simplistic modifications modify the voter application (found in
/examples/voter) to store vote values as JSON and
generate real-time analytics using views on the JSON vote data.
Read more

written by vdbdev on October 23, 2013 in Cookbook

Upsert: update or insert a record

There are many times when a database needs to maintain unique records, and as new data arrives it may not be known if it is a new record to be inserted, or an existing record to be updated.… Read more

written by vdbdev on October 22, 2013 in Cookbook

Scaling: 700,000 Simultaneous Connections to VoltDB

One of the original design goals for VoltDB was to support large numbers of connections per server. This kind of came for free with Volt’s event based design. The event based design was driven by a desire to keep the number of active software threads 1:1 with available hardware threads and this basically proscribed any kind of thread per client network IO implementation.… Read more


VoltDB SQL Command-Line Tool

sqlcmd is a SQL shell interpreter that allows you to execute VoltDB SQL and
Stored Procedures interactively as well as non-interactively, via scripts.

The following examples show how to execute SQL and Stored Procedures against
the VoltDB Voter example.… Read more

written by vdbdev on October 9, 2013 in Cookbook

VoltDB Real-Time Analytics with JSON

Consider the scenario where you are building an online multi-player game platform, where users can use the platform to create their own games.  Such a platform would have the following high-level requirements:

  1. It needs to be able to process tens of thousands of write transactions, game state changes, per second. 
  2. Read more







Configuring Durability

VoltDB achieves on-disk durability via its command logging feature. Command logging is enabled via the deployment.xml file. Command logging can not be enabled/disabled on a running cluster.

<deployment>
    <cluster hostcount="4" sitesperhost="2" kfactor="1" />
    <commandlog enabled="true"/>
</deployment>

Command logging has several configurable properties, including the path to the directory that log files will be written to, synchronous vs.… Read more


Starting a Cluster

VoltDB is usually configured to run in a cluster. Clusters provide both horizontal scaling (more transactions, more memory for storing data) and allow replication of data for high availability.

To start a VoltDB cluster, you specify two additional pieces of information when starting the database processes.… Read more


Configuring High Availability

VoltDB, when running on multiple hosts, can tolerate the failure of one or more hosts and continue running. To accomplish this, the database replicates partitions to multiple servers within a cluster.… Read more


Configuring Partitioning

VoltDB distributes data and processing across partitions. Other systems use the terms “shards” or “regions”. There are two steps to configuring partitioning in VoltDB.

  1. Tell the database how to distribute data.
  2. Read more





























JSON in VoltDB

VoltDB 3.0 introduces the use of JSON-encoded columns to allow more flexibility in how you structure and interact with your data. New SQL functions and index capabilities let you work more naturally with JSON data while maintaining the efficiency and transactional consistency of a relational database.… Read more





Stonebraker Live! – Tonight!

Register nowto join us via live streaming!

Can’t make it to Santa Clara tonight? That doesn’t mean you can’t attend Stonebraker Live!

Join us via live stream at 6:30 p.m. Pacific to hear from database legend Mike Stonebraker, as well as VoltDB’s VP of Market Strategy, Mark Hydar, and Co-founder, Scott Jarr.… Read more










Introducing VoltDB 3.0

The VoltDB engineering team is thrilled to announce that VoltDB 3.0 is now available!  Over the past six months we’ve added a ton of features to VoltDB 3.0. This blog post lists the highlights, but that just scratches the surface.… Read more












Upserts in VoltDB

Upserts in VoltDB

The idea behind an upsert is that you try an update or an insert query first and if it fails, you then do the other query.
Why do an upsert?… Read more







Writing VoltDB Apps in Java Q & A

NewSQL and NoSQL databases present developers with a new and interesting programming model where there are fewer connections to the database, query parallelism, partitions and an easy model for supporting new clusters.… Read more




Making an Impact: VoltDB Engineering Interns

Making an Impact: VoltDB Engineering Interns

Summer went by too fast in New England!  We are already missing our two engineering internship students, Xin Jia from Brown University and Zheng li from University of Massachusetts, Lowell.… Read more







Thoughts on NoSQL Now! 2012

One of the best parts about working for a growing start-up is the chance to leave keyboard and office behind and escape to meet users. I spent last week in San Jose at NoSQLNow – a conference designed to bring together users and vendors of “Not Only SQL” products.… Read more


VoltDB – Simplified CSV Loader

My name is Xin Jia, a Brown University student, and I’ve spent the summer interning at VoltDB. One of the first projects that I worked on this summer was a CSV Loader with fellow intern Zheng Li.… Read more







Using the Spring Converter API with VoltDB Data Objects

Mapping one type to another is a pretty common task. Hibernate and other ORM’s map a result from a data source’s native representation to an application specific representation. In English, I want to convert a JDBC result set, or data objects, into a collection of POJOs (plain old Java objects) using some kind of data mapping tool or API.… Read more



To Flash or Not to Flash: That is the Question

I am often asked about the value of flash memory in OLTP database applications.  This blog post discusses flash technology in this context.  First, I discuss the future of flash in general; then I turn to flash (and other future storage technologies) in the context of a main memory DBMS, such as VoltDB.… Read more




VoltDB Community Users Share Knowledge

VoltDB adoption and deployments are accelerating.  We’re working hard make our products easy to use in both cloud and privately-racked infrastructures.  We see strong adoption in five application categories – capital markets (mostly around trading systems), digital advertising, online games, network monitoring, and intelligence/surveillance (national security, fraud mitigation, etc.).… Read more


Using Node.js with VoltDB

We recently released the first supported version of a VoltDB client driver for Node.js applications. It was able to execute over 695K transactions per second. Since then we have been working on building a better driver and sought out help from the Node community and made a series of improvements.… Read more




695k TPS with Node.js and VoltDB

Hi, I’m Henning Diedrich, co-founder and CEO of Eonblast, Inc. I’m a guest contributor to VoltDB’s blog.

In February I was contacted by VoltDB about conducting a benchmarking project.  The company had recently released an updated version of a Node.js client driver that had originally been authored by Jacob Wright, one of VoltDB’s community members. … Read more


Announcing VoltDB Version 2.5

I’m very happy to announce the release of VoltDB 2.5.  Consistent with prior releases, 2.5 includes one major new feature, Database Replication, combined with other significant work in and around the VoltDB core engine. … Read more






VoltDB WAN Replication – Pre-release Users Invited

The subject of my last blog post was an introduction to VoltDB Replication, one of the major product additions that we’ve been working on.  Well, with this post, I’m happy to announce that we’ve recently pre-released VoltDB Replication to selected users to perform beta-level validation!… Read more


Intro to WAN Replication in VoltDB

This is the first in a series of posts about WAN replication, a VoltDB feature that’s currently under development.  It provides a high level overview of our direction.  More detailed posts will follow as we progress through our engineering iterations.… Read more


VoltDB 2.0 Features Summary

VoltDB 2.0 was released last week and I wanted to take this opportunity to summarize the important new features, including enhancements for database durability and recovery; key performance and interoperability improvements; and many enhancements geared at helping developers use VoltDB productively.… Read more




Optimizing Distributed Read Operations in VoltDB

Many VoltDB applications, such as gaming leader boards and real-time analytics, use multi-partition procedures to compute consistent global aggregates (and other interesting statistics).  It’s challenging to efficiently process distributed reads operations, especially for performance sensitive applications. … Read more


Intro to VoltDB Command Logging

VoltDB’s new command logging feature reduces the window of data loss during a cluster wide failure from single digit minutes (the window between snapshots) to zero. Command logging can be tuned to give you the same amazing latency and throughput you get from VoltDB, even on a single 7.2k SATA disk.… Read more


VoltDB’s New Command Logging Feature

We, at VoltDB, are excited to tell you about the command logging feature that we’ve been working on this summer. We’ve built this feature because our customers asked for it, and they’ve given us some great feedback on how it should work.… Read more



The Sqoop on VoltDB Export and Hadoop Integration

In the last couple of releases of VoltDB, we’ve made steady improvements to our Export feature. Export allows you to build into your VoltDB applications an automatic flow of data from VoltDB to companion datastores (for example, to an analytic database).… Read more


Berlin Bettswords

Getting away from the line-by-line details of code is necessary – the big picture matters. Getting away to Berlin, Germany, is a treat.  I’ve spent most of a week at the Berlin Buzzwords Conference http://berlinbuzzwords.de listening and talking about data management. … Read more




Upgrading to VoltDB V1.3

VoltDB V1.3 contains a number of new features, as well as changes and enhancements to existing features. In some cases, these enhancements involve changes to the behavior or the interface that existing users need to be aware of.

Read more







Visibility

“You cannot manage what you cannot measure”” - Peter Drucker

Most of us have heard this quote many times, though it’s probably more relevant to software development to say:

“You cannot manage what you do not measure”.Read more


Why is VoltDB So Fast?

First and foremost, VoltDB is focused on specific workloads. Most existing RDBMSs are designed to be general purpose, one-size-fits-all systems. Recently, there have been a lot of new databases introduced that achieve better performance by specializing in areas like analytics, graphs or streaming data.… Read more

Documenting Open Source Software

Someone asked what we use to produce the documentation for VoltDB. The simple answer is DocBook.

DocBook is an XML schema for structured documentation. Originally designed for UNIX-based hardware and software documentation, DocBook is generalized enough for most technical documentation purposes.… Read more


Building a Cluster on a Limited Budget

In April 2010 we built a “budget cluster” to facilitate additional VoltDB testing (we already had a 12-node Dell R710 cluster). The requirement was to build a complete 6 server VoltDB cluster for less than the cost of a single R710 (much less actually).… Read more


Franken-Switch

I did quite a bit of testing on one of the major VoltDB v1.2 features, Network Fault Protection. When enabled this feature detects a network partition and ensures that we don’t allow “split-brain” behavior (a single VoltDB cluster splitting into two independent clusters).… Read more


SGI/VoltDB Benchmark- More Details

Yesterday’s press release documented a 30 node SGI Rackable Cluster running VoltDB achieving 3.4 million transactions per second (at 10ms latency) running the “Voter” application.

Impressive numbers, indeed. And the numbers are even more interesting when you understand the transactions.… Read more


Overclocking Your Database Servers

It seems like I’ve been programming forever (we’re talking TS-1000, TRS-80, and Commodore64). I’ve always been looking to improve the performance of my programs, especially database applications. Better performance can usually be attributed to one of the following:

  • New Hardware[insert name of favorite hardware vendor] has just released new server technology that incorporates improved [RAM | CPU | Disk | RAID Controller].
  • Read more


Java and Kernal Bugs

Java Concurrency Bug

This is the first bug we found in a system outside VoltDB. We used LinkedBlockingDeque in Java’s concurrent package to maintain concurrent access in VoltDB’s network subsystem. Once in a while, we noticed cluster lockups when the system was under pressure.… Read more




Transactions on a Budget

We hand-built a “cluster” at VoltDB to perform a variety of long running tests. In this test cluster, one of the nodes performs all client application activity while the other 5 serve as VoltDB servers.… Read more


High Availability and Cloudy Problems

VoltDB, like many distributed systems, achieves high availability through redundant processing nodes. VoltDB calls this K-Safety. Essentially, the distributed system can answer any request at at least K+1 servers, so it can tolerate at least K hardware failures.… Read more

Living the Asynchronous LifeStyle, or how I Learned to Stop Threading and Love the Non-Blocking Call [back]

A 1:1 mapping between software threads and hardware threads is a core part of VoltDB’s approach to performance and also turns out to be the core of the asynchronous lifestyle. By multi-plexing work through a pool of threads appropriately sized for your hardware, you can obtain the maximum parallelism that your hardware can provide and efficiently process many discrete tasks (although parallelism != performance).… Read more

What? VoltDB Stored Procedures?

VoltDB is a relational store that uses SQL for its query language and stored procedures for its transactional unit of work. Clients invoke stored procedures to access the database. Stored procedures access data using parametrized SQL statements and can manipulate intermediate results in Java.… Read more


The Lifecycle of a Transaction

When learning about VoltDB for the first time, people often ask how VoltDB executes transactions in a distributed environment. So, here’s how…

Let’s say a developer, Dan, is working on a new website.… Read more





Categories