Living the Asynchronous LifeStyle, or how I Learned to Stop Threading and Love the Non-Blocking Call [back]

written by Ariel Weisberg on July 14, 2010 with one Comment

A 1:1 mapping between software threads and hardware threads is a core part of VoltDB’s approach to performance and also turns out to be the core of the asynchronous lifestyle. By multi-plexing work through a pool of threads appropriately sized for your hardware, you can obtain the maximum parallelism that your hardware can provide and efficiently process many discrete tasks (although parallelism != performance). In the VoltDB world, tasks tend to be very small in both size and duration, and there are thousands of them in flight at any given time in order to ensure that the execution pipeline never starves. This works for VoltDB because tasks are discrete, independent, and require no synchronization beyond what is necessary to acquire the task itself.

The importance of asynchronism extends beyond the server and shows up in the Java client library as well. There is a synchronous method for invoking procedures, and it’s even thread safe, but it’s also slow. Not an order of magnitude slower, but definitely on the order of 10s of percent. On one typical workload, a gigabit link can sustain around 180k transactions/sec, which means 180k thread sleeps/wake-ups along with the associated unnecessary cache thrashing in an application that uses the blocking API. See The asynchronous API is also advantageous when applications have other work to do. For instance an application doing key/value lookups can perform compression/decompression work, process other load/store requests, or perform cache maintenance while waiting for new procedures to invoke.

The asynchronous client API is only superficially different from the traditional connection pooling approach where connections to the database are dedicated to a single thread at a time. In both cases an application can have multiple outstanding transactions and won’t know the order those transactions execute. Nor will it know if an outstanding transaction is committed if a connection is lost before the database responds. The asynchronous approach does present thread-safety issues, as callbacks will be invoked from a different thread, and multiple callbacks may be invoked concurrently.

One way to make the asynchronous approach read more similarly to the synchronous approach is to use anonymous inheritance/implementation.

[sourcecode language="java"]
final long bar = 42; client.callProcedure( new ProcedureCallback() {
@Override public void clientCallback( ClientResponse response) {
if (response.getStatusCode() == bar) { //handle response
"foo",     arg1,     arg2);

Anonymous inheritance/implementation allows you to place the implementation of a callback in the same location as the code that invokes the procedure rather than in a separate class defined somewhere else. This allows a coding style more similar to synchronous APIs where result-handling code normally follows the procedure invocation. Another bonus is that an instance of an anonymous callback has visibility of any final variables in scope, which can be used to pack additional information into the callback such as the arguments to the procedure without requiring a constructor. Anonymous callbacks should be kept short for readability and act as a pointer to more complex code elsewhere if necessary.

A common “how to” question is how to access procedure arguments from within a callback. The Java client API defines an optional interface called ProcedureArgumentCacher that classes implementing ProcedureCallback can implement. The API invokes ProcedureArgumentCacher.setArgs() at invocation time if the callback implements the interface. This gives the callback an opportunity to save some or all of the arguments so they will be available when the callback is invoked. The API also provides the class AbstractProcedureArgumentCacher for subclassing. AbstractProcedureArgumentCacher implements ProcedureArgumentCacher and caches all the arguments and makes them available via the protected AbstractProcedureArgumentCacher.args() method.

Another common question is how to ensure transactions execute in a specific order. The easiest thing to do is to nest the invocation of a stored procedure with dependencies inside the callback of the dependency procedures. A semaphore can be used to track multiple dependencies, although it will not help you propagate errors or data. Consider procedures A, B, and C. C must be invoked after both A and B.

[sourcecode language="java"]
final Semaphore dependencies = new Semaphore(0);
final int numDependencies = 2;
final ProcedureCallback callback = new ProcedureCallback() {
@Override public void clientCallback( ClientResponse response) { \
if (dependencies.tryAcquire(numDependencies)) {
client.callProcedure( new NullCallback(), "A", arg1, arg2);
client.callProcedure( callback,"A",arg1,arg2);
client.callProcedure( callback,"B",arg1,arg2);

The same callback can be used for A and B because the data accessed by the callback is accessed in a thread-safe manner. This example also demonstrates the NullCallback for those who just don’t care about their data. Unfortunately, plain old Java null doesn’t fly due to the use of a variable length argument list in Client.callProcedure().

As delivered in VoltDB 1.0, the API doesn’t default to being truly asynchronous because Client.callProcedure() blocks until the invocation can be queued to the network layer. To go all the way, an application needs to invoke Client.configureBlocking(false). In non-blocking mode the return value of callProcedure indicates whether the procedure was actually queued. If the procedure was not queued, the application can go and do other work (compression/decompression, cache management etc.) or invoke Client.backpressureBarrier() to explicity block until the API thinks that callProcedure() is likely to succeed.

Go forth. Be fruitful. Multiply. But stop once you run out of hardware threads.

Ariel Weisberg
Software Engineer