Narrowing the data-to-decision gap with in-memory operational databases

written by Peter Vescuso on May 22, 2014 with no comments

We can all agree that data is an organization’s greatest asset, yet in many industries data is treated as a ‘fixed’ asset, collected and stored in data warehouses for later analysis. This ignores data’s most valuable moment: when it’s analyzed in real-time to inform business decisions.

Not all companies hoard data for later analysis and action, of course. In industries as diverse as telecoms, advertising technology, energy, financial services, retail and gaming, real-time analysis of data means the difference between profit and loss, pleasing customers or alienating them, and having a viable business model or losing to competitors.

As data volumes grow, pushed by new, high-velocity sources of data (smartphones and tablets, social media, wearable devices, smart thermostats, connected cars) even the most traditional companies are searching for tools to help close the data-to-decision gap. Big Data initiatives are underway at most enterprises, yet Gartner Research reports “90% of the information assets from big data analytic efforts will be siloed and un-leveragable across multiple business processes.” (Gartner, Inc. Predicts 2014: Big Data.)

Big Data’s promise, to make historical and real-time information actionable, has yet to be realized; a recent Gartner survey reveals that “while 64% of surveyed organizations either have invested in big data already (30%) or have plans to invest within 24 months (34%)”, a slim 25% are in production with Big Data initiatives. (Gartner, Inc. Survey Analysis: Big Data Adoption in 2013 Shows Substance Behind the Hype.)

One of the chief challenges to solving the Big Data puzzle is data management, using technologies architected to take in and analyze vast amounts of incoming data in real time.

The real-time challenge

First let me say that when I use the term ‘real time’ I mean “now.” In the data warehouse space the notion of real time has been greatly diluted and often means responses in ‘minutes’ versus the days or hours experienced with batch processing. Many real-time technologies focus on scalability, providing scalable caching or data capture, but lack the capability to respond with intelligent analysis or decisions in milliseconds. Contention can be an issue when multiple records are submitted, and it can be difficult to get visibility into related data. While relational databases meet many of these needs, most RDBMSs are based on 30-year-old architectures that don’t meet the scaling requirements of new applications. These new applications need ACID guarantees, and in many ways are considered OLTP (Online Transaction Processing). This new approach to OLTP is described by Gartner as an ‘operational database’.

Attributes of an operational database

Operational databases capable of handling data at scale share the following capabilities:

  • Ingest and interact on streams of inbound data
  • Make per-event, data-driven decisions
  • Real-time analytics on fast moving data
  • Integration with data warehouses to speed data export
  • High-speed serving of warehouse-derived analytics

On the other hand, the properties of a traditional RDBMS include:

  • SQL data querying
  • Relational data models
  • Ad Hoc querying
  • Durability, High Availability, and Disaster Recovery.

Key to creating and capturing business value with new, data-driven applications is the ability to gain faster insights and take faster action. Using real-time operational databases gets you there.

Traditional relational databases are ill-suited to the task. New approaches, including NoSQL and NewSQL databases, offer modern architectures built to handle most of the challenges of new, data-driven applications and businesses. But only NewSQL operational databases, such as VoltDB, offer users the familiarity of SQL data querying, the persistence and ACID compliance of RDBMSs, and the speed, scalability and analytical capacity necessary to handle millions of transactions per second.

See how VoltDB closes the data-to-decision gap for businesses including Sakura Internet, Shopzilla, and Social Game Universe, here, or download VoltDB here to see how it can meet the data management needs of your business.