In memory computing - a survey of implementations to date

In Memory Computing Blog Series
Part 1: In Memory Computing - A Walk Down Memory Lane

In the series’ previous post we took a walk down memory lane and reviewed the evolution of In Memory Computing (IMC) over the last few decades. In this blog post we will deep dive into a couple of specific implementations of IMC to better understand both the innovations that have happened to date and their limitations.

The Buffer Cache

Buffer caches, also sometimes referred to as page caches, have existed in operating systems and databases for decades now. A buffer cache can be described a piece of RAM set aside for caching frequently accessed data so that one can avoid disk accesses. Performance, of course, improves the more one can leverage the buffer cache for data accesses as opposed to going to disk.

Buffer caches are read-only caches. In other words only read accesses can potentially benefit from a buffer cache. In contrast, writes must always commit to the underlying datastore because RAM is a volatile medium and an outage will result in the data residing in the buffer cache getting lost forever. As you can imagine this can become a huge deal for an ACID compliant database. ☺ 

The diagrams below depict how the buffer cache works for both reads and writes.

For reads, the first access must always go to the underlying datastore whereas subsequent accesses can be satisfied from the buffer cache

For writes we always have to involve the underlying datastore. The buffer cache provides no benefits.

The fact that the buffer cache only helps reads is severely limiting. OLTP applications, for example, have a good percentage of write operations. These writes will track the performance of the underlying datastore and will most probably mask any benefits the reads are reaping from the buffer cache. 

Secondly, sizing the buffer cache is not trivial. Size the buffer cache too small and it is of no material benefit. Size it too large and you end up wasting memory. The irony of it all, of course, is that in virtualized environments the server resources are shared across VM and are managed by the infrastructure team. Yet, sizing the buffer cache is an application specific requirement as opposed to an infrastructure requirement. One therefore ends up doing this with no idea of what other applications are running on the host or sometimes even without an awareness of how much RAM is present in the server.

API based in memory computing

Another common way to leverage IMC is to leverage libraries or API within one’s applications. These libraries or API allow the application to use RAM as a cache. A prevalent example for this is memcached. You can learn more about memcached here. Here’s is an example of how one would use memcached to leverage RAM as a cache:

function get_foo(foo_id)     
foo = memcached_get("foo:" . foo_id)     
return foo if defined foo      
foo = fetch_foo_from_database(foo_id)     
memcached_set("foo:" . foo_id, foo)     
return foo 

This example shows you how you must rewrite your application to incorporate the API at the appropriate places. Any change to the API or the logic of your application means a rewrite.

Here is a similar example from SQL. 


In the SQL example note that you, as a user, need to specify which tables need to be cached in memory. You need to do this when the table is created. One presumes that if the table already exists you will need to use the ALTER TABLE command. As you know DDL changes aren’t trivial. Also, how should a user decide which tables to keep in RAM and which not to? And does such static, schema based definitions work in today’s dynamic world? Note also that it is unclear if these SQL extensions are ANSI SQL compliant.

In both cases enabling IMC means making fundamental changes to the application. It also means sometimes unnecessarily upgrading your database or your libraries to a version that supports the API you need. And, finally, it also means that each time the API behavior changes you need to revisit your application.

In Memory Appliances

More recently ISV have started shipping appliances that run their applications in memory. The timing for these appliances makes sense given servers with large amounts of DRAM are becoming commonplace. These appliances typically use RAM as a read cache only. In the section on buffer caches we’ve already discussed the limitations of read only caches. 

Moreover since many of these appliances run ACID compliant databases consistency is a key requirement. This means that writes must be logged in non-volatile media in order to guarantee consistency. If there are enough writes in the system then overall performance will begin to track the performance of the non-volatile media as opposed to RAM. The use of flash in place of mechanical drives mitigates this to some extent but if one is going to buy a purpose built appliance for in memory computing it seems unfortunate that one can only be guaranteed flash performance instead.

Most of these appliances also face tremendous startup times. They usually work by pulling all of their data into RAM during startup so that it can be cached. Instrad of using an ‘on demand’ approach they preload the data into RAM. This means that startup times are exorbitantly high when data sizes are large. 

Finally, from an operational perspective rolling in a purpose built appliance usually means more silos and more specialized monitoring and management that takes away from the tremendous value that virtualization brings to the table.


IMC as it exists today put the onus on the end user/application owner to make tweaks for storage performance when in fact the problem lies within the infrastructure. The result is tactical and proprietary optimizations that either make sub-optimal use of the infrastructure or simply break altogether over time. Wouldn’t the right way instead be to fix the storage performance problem, via in memory computing, at the infrastructure level? After all, storage performance is an infrastructure service to begin with. For more information, check out 

Infrastructure Level In Memory Computing Video

Watch Satyam Vaghani, CTO and Bala Narasimhan, VP of Products, at PernixData discuss Infrastructure level In-Memory Computing.
Learn how a new standard for in memory computing is forming in the industry where existing applications can take full advantage with zero changes, while future applications can become much easier to develop once they leverage the power of infrastructure level in memory computing.
Grab a cup a tea and enjoy!

In Memory Computing - A Walk Down Memory Lane

I'm excited once again to be hosting guest blogger Bala Narasimhan VP of Products at PernixData. He will be doing a new series dedicated to Infrastructure Level In Memory Computing. This is a new way to think and design your environment for the future. Enjoy! 

In Memory Computing (IMC) has seen a revival in recent years. In this blog post we will walk down memory lane (no pun intended) and study the evolution of IMC. We will specifically focus on the role of IMC in conjunction with storage systems.

Here’s a quick timeline capturing the evolution of IMC.

IMC initially manifested itself as buffer caches in operating systems and databases. Buffer caches remain integral parts of these software systems to date. A buffer cache is a read cache for most recently accessed data. A hit in the buffer cache makes reads go faster because it avoids a disk access. Writes however must be synchronized with the underlying datastore. This means a buffer cache help with reads but never help with writes. In Memory Databases (IMDB) that hold all of the data in memory for faster query processing were first introduced in 1990s but didn’t make much of a dent. Recently IMDB have seen a revival and have become an important discussion topic in the database industry. One of the reasons for this revival has been the dramatic increase in the amount of RAM that servers can hold; surpassing a terabyte of RAM per server. IMDB, like buffer caches, work well for reads but require synchronization with a persistent media for writes and therefore incur performance penalties. The early 2000’s saw the introduction of memory caching libraries such as Memcached that allow applications to leverage the performance of RAM as a cache within the application logic. Application owners, via API, can invoke these libraries at well defined locations to use RAM as a cache. Unfortunately, any change in the underlying memory caching libraries or to the application behavior would mean a rewrite of the application altogether.

All of these approaches leverage RAM as a volatile cache. This means all these approaches are read caches and only accelerate parts of an application, if at all. More importantly, all of these approaches push the in memory computing into the application. This has a number of unfortunate side effects:

  • Applications become unnecessarily complicated. They have to maintain complicated caching logic and ensure consistency with the backing store.
  • The onus is now on the end user to figure out how best to leverage IMC. Everything from determining the appropriate buffer cache size to deciding which tables to pin in RAM is now an application owner decision.
  • The application owners do not always control the infrastructure on which the application is deployed. It is therefore unfortunate that they must decide how much memory an application should use for caching without understanding how the infrastructure (for example, servers) are going to be used and will evolve. In addition, applications owners do not always have a holistic view of other applications are running on the same server.
  • Leveraging memory management innovations become challenging. NUMA is a good example of an innovation that cannot be leveraged until an application is enhanced to leverage it even though NUMA itself is an infrastructure level innovation.

All of this has meant that the strengths of RAM as a data layer have not been fully exploited. While this may not have been an issue until now because most servers had only little amounts of RAM, say 32GB or 64GB, this is quickly changing. Servers with terabytes of RAM are not uncommon these days and this trend is only going to continue. The time has therefore come to rethink IMC from the ground up. Enhancing applications to do better read caching and putting the onus on the end user is not the answer. Future blog posts in this series will introduce a high performance, highly scalable way to leverage IMC across one’s entire infrastructure.

My Role Transition

On May 1st, I started a new role in my career path at PernixData. It is with great delight that I’m now a Product Manager within the Product Management group. I look forward to working closely with Bala Narasimhan VP of Product Management. He and I have worked together on many other projects since the early alpha stages of FVP. So, in some ways this transition will feel like the early days!

With that said, I will be running many different projects in my new role. A renewed energy around blogging and staying close to the community is at utmost importance for me. In addition I will be helping with product collateral, documentation, roadmap items, and much more. It’s with this that I thank the wonderful teams and individuals that I have had the privilege to work with! Becoming the 1st customer to Product Manager is something that I could have only dreamed of.

Feel free to reach out and connect with me anytime, as part of my focus is to stay connected to a relevancy in the field, as this can only help any product-focused endeavors!

On a side note, I also thought it was very ironic that on May 1st, my father officially retired. He has been a minister and marriage and family therapist for many years. The timing couldn’t have been predicted, so I humbly congratulate him on his promotion to retirement and thank him for the many years of taking care of his family!


How Can Database Users Benefit - PernixData FVP

I'm pleased to introduce you to Bala Narasimhan, VP of Products at PernixData. He has a wealth of knowledge around databases, and has authored 2 patents for memory management in relational databases. It's my pleasure to have him featured in today's post. He is officially my first guest blogger! Enjoy! 

Databases are a critical application for the enterprise and usually have demanding storage performance requirements. In this blog post I will describe how to understand the storage performance requirements of a database at the query level using database tools. I’ll then explain why PernixData FVP helps not only to solve the database storage performance problem but also the database manageability problem that manifests itself when storage performance becomes a bottleneck. Throughout the discussion I will use SQL Server as an example database although the principles apply across the board.

Query Execution Plans

When writing code in a language such as C++ one describes the algorithm one wants to execute. For example, implementing a sorting algorithm in C++ means describing the control flow involved in that particular implementation of sorting. This will be different in a bubble sort implementation versus a merge sort implementation and the onus is on the programmer to implement the control flow for each sort algorithm correctly.

In contrast, SQL is a declarative language. SQL statements simply describe what the end user wants to do. The control flow is something the database decides. For example, when joining two tables the database decides whether to execute a hash join, a merge join or a nested loop join. The user doesn’t decide this. The user simply executes a SQL statement that performs a join of two tables without any mention of the actual join algorithm to use. 

The component within the database that comes up with the plan on how to execute the SQL statement is usually called the query optimizer. The query optimizer searches the entire space of possible execution plans for a given SQL statement and tries to pick the optimal one. As you can imagine this problem of picking the most optimal plan out of all possible plans can be computationally intensive.

SQL’s declarative nature can be sub-optimal for query performance because the query optimizer might not always pick the best possible query plan. This is usually because it doesn’t have full information regarding a number of critical components such as the kind of infrastructure in place, the load on the system when the SQL statement is run or the properties of the data. . One example of where this can manifest is called Join Ordering. Suppose you run a SQL query that joins three tables T1, T2, and T3. What order will you join these tables in? Will you join T1 and T2 first or will you join T1 and T3 first? Maybe you should join T2 and T3 first instead. Picking the wrong order can be hugely detrimental for query performance. This means that database users and DBAs usually end up tuning databases extensively. In turn this adds both an operational and a cost overhead.

Query Optimization in Action

Let’s take a concrete example to better understand query optimization. Below is a SQL statement from a TPC-H like benchmark. 

select top 20 c_custkey, c_name, sum(l_extendedprice * (1 - l_discount)) as revenue, c_acctbal, n_name, c_address, c_phone, c_comment from customer, orders, lineitem, nation where c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate >= ':1' and o_orderdate < dateadd(mm,3,cast(':1'as datetime)) and l_returnflag = 'R' and c_nationkey = n_nationkey group by c_custkey, c_name, c_acctbal, c_phone, n_name, c_address, c_comment order by revenue;

The SQL statement finds the top 20 customers, in terms of their effect on lost revenue for a given quarter, who have returned parts they bought. 

Before you run this query against your database you can find out what query plan the optimizer is going to choose and how much it is going to cost you. Figure 1 depicts the query plan for this SQL statement from SQL Server 2014 [You can learn how to generate a query plan for any SQL statement on SQL Server at]

Figure 1
You should read the query plan from right to left. The direction of the arrow depicts the flow of control as the query executes. Each node in the plan is an operation that the database will perform in order to execute the query. You’ll notice how this query starts off with two Scans. These are I/O operations (scans) from the tables involved in the query. These scans are I/O intensive and are usually throughput bound. In data warehousing environments block sizes could be pretty large as well. 

A SAN will have serious performance problems with these scans. If the data is not laid out properly on disk, you may end up with a large number of random I/O. You will also get inconsistent performance depending on what else is going on in the SAN when these scans are happening. The controller will also limit overall performance.

The query begins by performing scans on the lineitem table and the orders table. Note that the database is telling what percentage of time it thinks it will spend in each operation within the statement. In our example, the database thinks that it will spend about 84% of the total execution time on the Clustered Index Scan on lineitem and 5% on the other. In other words, 89% of the execution time of this SQL statement is spent in I/O operations! It is no wonder then that users are wary of virtualizing databases such as these.

You can get even more granular information from the query optimizer. In SQL Server Management Studio, if you hover your mouse over a particular operation a yellow pop up box will appear showing very interesting statistics. Below is an example of data I got from SQL Server 2014 when I hovered over the Clustered Index Scan on the lineitem able that is highlighted in Figure 1.

Notice how Estimated I/O cost dominates over Estimated CPU cost. This again is an indication of how I/O bound this SQL statement is. You can learn more about the fields in the figure above here.

An Operational Overhead

There is a lot one can learn about one’s infrastructure needs by understanding the query execution plans that a database generates. A typical next step after understanding the query execution plans is to tune the query or database for better performance.  For example, one may build new indexes or completely rewrite a query for better performance. One may decide that certain tables are frequently hit and should be stored on faster storage or pinned in RAM. Or, one may decide to simply do a complete infrastructure redo.

All of these result in operational overheads for the enterprise. For starters, this model assumes someone is constantly evaluating queries, tuning the database and making sure performance isn’t impacted. Secondly, this model assumes a static environment. It assumes that the database schema is fixed, it assumes that all the queries that will be run are known before hand and that someone is always at hand to study the query and tune the database. That’s a lot of rigidity in this day and age where flexibility and agility are key requirements for the business to stay ahead.

A solution to database performance needs without the operational overhead 

What if we could build out a storage performance platform that satisfies the performance requirements of the database irrespective of whether query plans are optimal, whether the schema design is appropriate or whether queries are ad-hoc or not? One imagines such a storage performance platform will completely take away the sometimes excessive tuning required to achieve acceptable query performance. The platform results in an environment where SQL is executed as needed by the business and the storage performance platform provides the required performance to meet the business SLA irrespective of query plans.

This is exactly what PernixData FVP is designed to do. PernixData FVP decouples storage performance from storage capacity by building a server side performance tier using server side flash or RAM. What this means is that all the active I/O coming from the database, both reads and writes, whether sequential or random, and irrespective of block size is satisfied at the server layer by FVP right next to the database. You are longer limited by how data is laid out on the SAN, or the controller within the SAN or what else is running on the SAN when the SQL is executed.

This means that even if the query optimizer generates a sub optimal query plan resulting in excessive I/O we are still okay because all of that I/O will be served from server side RAM or flash instead of network attached storage. In a future blog post we will look at a query that generates large intermediate results and explain why a server side performance platform such as FVP can make a huge difference.