In Memory Computing - A Walk Down Memory Lane

I'm excited once again to be hosting guest blogger Bala Narasimhan VP of Products at PernixData. He will be doing a new series dedicated to Infrastructure Level In Memory Computing. This is a new way to think and design your environment for the future. Enjoy! 

In Memory Computing (IMC) has seen a revival in recent years. In this blog post we will walk down memory lane (no pun intended) and study the evolution of IMC. We will specifically focus on the role of IMC in conjunction with storage systems.

Here’s a quick timeline capturing the evolution of IMC.

IMC initially manifested itself as buffer caches in operating systems and databases. Buffer caches remain integral parts of these software systems to date. A buffer cache is a read cache for most recently accessed data. A hit in the buffer cache makes reads go faster because it avoids a disk access. Writes however must be synchronized with the underlying datastore. This means a buffer cache help with reads but never help with writes. In Memory Databases (IMDB) that hold all of the data in memory for faster query processing were first introduced in 1990s but didn’t make much of a dent. Recently IMDB have seen a revival and have become an important discussion topic in the database industry. One of the reasons for this revival has been the dramatic increase in the amount of RAM that servers can hold; surpassing a terabyte of RAM per server. IMDB, like buffer caches, work well for reads but require synchronization with a persistent media for writes and therefore incur performance penalties. The early 2000’s saw the introduction of memory caching libraries such as Memcached that allow applications to leverage the performance of RAM as a cache within the application logic. Application owners, via API, can invoke these libraries at well defined locations to use RAM as a cache. Unfortunately, any change in the underlying memory caching libraries or to the application behavior would mean a rewrite of the application altogether.

All of these approaches leverage RAM as a volatile cache. This means all these approaches are read caches and only accelerate parts of an application, if at all. More importantly, all of these approaches push the in memory computing into the application. This has a number of unfortunate side effects:

  • Applications become unnecessarily complicated. They have to maintain complicated caching logic and ensure consistency with the backing store.
  • The onus is now on the end user to figure out how best to leverage IMC. Everything from determining the appropriate buffer cache size to deciding which tables to pin in RAM is now an application owner decision.
  • The application owners do not always control the infrastructure on which the application is deployed. It is therefore unfortunate that they must decide how much memory an application should use for caching without understanding how the infrastructure (for example, servers) are going to be used and will evolve. In addition, applications owners do not always have a holistic view of other applications are running on the same server.
  • Leveraging memory management innovations become challenging. NUMA is a good example of an innovation that cannot be leveraged until an application is enhanced to leverage it even though NUMA itself is an infrastructure level innovation.

All of this has meant that the strengths of RAM as a data layer have not been fully exploited. While this may not have been an issue until now because most servers had only little amounts of RAM, say 32GB or 64GB, this is quickly changing. Servers with terabytes of RAM are not uncommon these days and this trend is only going to continue. The time has therefore come to rethink IMC from the ground up. Enhancing applications to do better read caching and putting the onus on the end user is not the answer. Future blog posts in this series will introduce a high performance, highly scalable way to leverage IMC across one’s entire infrastructure.