Skip directly to content

Integrating VoltDB with the Spring Framework

Friday, March 30, 2012 - 12:00pm

Written by Andrew Wilson.

 

I’ve been writing Spring applications for a few years now and I’ve always been struck by the framework’s flexibility. A developer can write a web application in several distinct ways there is no discernible difference from an end user’s perspective.  Freeing the developer from strict implementation processes and design patterns greatly expands the choices a developer can make, such that the developer can write an implementation that fits the specifics of just one application and then take a completely different approach that is more suitable for another application.

 

Integrating external services, such as databases, file systems, third-party remote invocation services all become easy to implement because there is no specific right way to do it, though there are preferred ways.

 

Today I’m writing one possible method for integrating VoltDB into a Spring application. The implementation is for demonstration purposes and is relatively application agnostic. You’re free to change my implementation entirely provided that your application supports a few basic Spring configuration methods.

 

@Component
@Scope("singleton")
public class VoltClient {
protected Client client;
 
@Value("#{systemProperties.hostNames}")
private String hostNames;
 
public Client getClient() throws UnknownHostException, IOException {
if ( this.client == null ) {
String[] hostArray = hostNames.split(",");
this.initClient( hostArray );
}
return this.client;
}
 
private void initClient(String hostNames[])
throws UnknownHostException, IOException {
if (client == null) {
ClientConfig config = new ClientConfig("username", "password");
this.client = ClientFactory.createClient(config);
this.initConnections(hostNames);
}
}
 
private void initConnections(String hostNames[])
throws UnknownHostException, IOException {
if (hostNames != null && hostNames.length > 0) {
for (String host : hostNames) {
this.client.createConnection(host.trim());
}
} else {
throw new UnknownHostException("No hosts specified");
}
}
}

 

The above code initializes the org.voltdb.client.Client and connects to one or more VoltDB servers. This example expects that the hostnames will be in a single comma delimited string that will be split up into an array of String objects, and then connects to each server within a VoltDB cluster. The hostname string is passed in through a system property.

 

There are two Java clients for VoltDB. One is a standard JDBC driver that executes all queries synchronously. The other is a specialized client library that can run queries either synchronously or asynchronously, along with a number of other features. Synchronous queries perform well enough but their throughput is no match for asynchronous queries. Asynchronous query throughput is approximately four times greater than synchronous queries in a two node VoltDB cluster. For example, an application using asynchronous queries can run over 200K TPS (transactions per second) in a two node server cluster using a single client running on a Macbook Pro; a synchronous client running the same queries will achieve around 56K TPS.

 

Regardless of client, all database transactions occur at the VoltDB server level. Most VoltB applications use stored procedures written in Java, though you can write adhoc queries too. The transaction begins at the entry of the stored procedure and finish with commit or rollback at the end of that procedure. Consequently, Spring cannot manage the transactions because all management occurs at the stored procedure within the VoltDB and so you have more freedom to implement your integration such that it does not need to follow the DAO or even the Data frameworks.

 

That being said, let’s take a look at a Repository implementation that exposes a collection of VoltDB stored procedures.

 

public interface VoterRepository extends Repository {
 
// Call once during startup
void init( ProcedureCallback callback, int count, String candidates) throws Exception;
 
// Call at the end of voting
void getResults(ProcedureCallback callback)  throws Exception;
 
// Register each vote
void vote(ProcedureCallback callback, long phoneNumber, int contestantNumber, long maxVotesPerPhoneNumber)  throws Exception;
 
}
 
@Repository
public class VoterRepositoryImpl implements VoterRepository{
 
@Autowired
VoltClient voltClient;
 
public VoterRepositoryImpl() {
}
 
public void init(ProcedureCallback callback, int count, String candidates) throws NoConnectionsException, UnknownHostException, IOException  {
voltClient.getClient().callProcedure(callback, "Initialize", count, candidates);
}
 
public void getResults(ProcedureCallback callback) throws NoConnectionsException, UnknownHostException, IOException {
voltClient.getClient().callProcedure(callback, "Results");
}
 
public void vote(ProcedureCallback callback, long phoneNumber, int contestantNumber,
long maxVotesPerPhoneNumber) throws NoConnectionsException, UnknownHostException, IOException {
voltClient.getClient().callProcedure(callback, "Vote", phoneNumber, contestantNumber, maxVotesPerPhoneNumber);
}
}

 

This code sets up a basic repository for a “voter” example application. The voter example stored procedures are included in all VoltDB distributions. Note that the repository is based upon an ID of type long and a result of VoltTable. None of these stored procedures run synchronously, so the values are irrelevant. However, if we were to convert to a synchronous repository, then all the procedures would return an array of VoltTable objects. This raises an interesting point, a VoltDB stored procedure can return more than one resultset which is segregated into different VoltTable objects. It is also common practice to create new VoltTable instances that are populated entirely through the stored procedure’s user code rather than by the result of a particular query.

 

Asynchronously called stored procedures require a callback handler, which is called upon exit from the stored procedure. The following code demonstrates the invocation of a method within the VoterRepository class, which will call an asynchronous stored procedure.

 

@Autowired
private VoterRepository voterRepository;
 
public void start() throws Exception {
this.voterRepository.init(new ProcedureCallback() {
public void clientCallback(ClientResponse response)
throws Exception {
if (response.getStatus() == ClientResponse.SUCCESS) {
logger.info("Init executed");
semaphore.release();
} else {
logger.fatal("Application failed to initialize.");
System.exit(0);
}
}
}, 6, voteCandidates);
}

 

The code performs the initialization of the voter database. All asynchronously called stored procedures will invoke the callback handler and pass a ClientResponse object regardless of whether the stored procedure was able to execute successfully. The code then tests the ClientResponse status within the if statement.

 

We’ve seen all the components involved in this example. They are all wired together using just the @Autowired annotation. There is no need for extra accessor or factory methods for the application to execute and it runs quite fast. The above code was compared to an existing application that performs the same tasks and they produced almost identical throughput results. This means that the additional overhead of Spring’s injection had no impact on application performance.

 

The above sample could be further enhanced such that all the VoltDB client classes are completely abstracted out. It could also be expanded to include Spring DAO support. Again, this is likely unnecessary, as VoltDB manages all transaction states internally and so Spring’s persistence container has no bearing on VoltDB transactions.

Summary

This application demonstrates the basic process for integrating VoltDB into a Spring application. We see that the first step was to create a component to manage the connection between the application and the VoltDB server cluster. We then exposed the VoltDB stored procedures to the rest of the application, in this case using a Repository object. Finally, we invoked an asynchronously called stored procedure that flowed from the application level code to the repository and then to the VoltClient object. You can download the example application here.