Time Scale: Latency References

In a world where speed is of the utmost importance, it has become apparent to me that there is a notion of relativity in latency. In other words, one needs a form of measurement - a time scale to understand how fast something really was and/or is.

With high frequency low latency trading as depicted in this video, milliseconds is the name of the game. A loss of 2 seconds, as an example, can be a matter of losing millions of dollars or prevention of a catastrophic financial event.  

In using this example, how can one feel what 2 milliseconds feels like? Can one tell the difference between 2 or 3 milliseconds? I find it fascinating that we as humans sometimes base what is fast or slow on what it feels like. In fact, how do you measure a feeling anyway? We usually have to compare (base line) to determine if something is really faster or slower. I would argue that it’s sometimes the results or effect of latency that we measure against. In low latency trading, the effect or result can be devastating, and so there is a known threshold to not go past. However, this threshold is constantly being lowered or being challenged via competitive pressure. This means it’s important to constantly have latency references to measure against in order to determine if the combined effect will have positive or negative results.

This is why testing synthetic workloads (in order to determine what the performance is) can result in inaccuracies of what is truly fast or slow. When one tests only one workload, it’s not depicting the combined effect of all disparate workloads and their interactions as a whole.  Another inaccurate way to measure is to base decisions solely on what the end users feel is faster or slower. I know it can be interesting to see what the end-user thinks, but it’s not an accurate way to look at the whole system, as a way to measure. The results seen for all work done (may be based on a project) is a better way to measure the effect. This can obviously complicate the process of measuring, but there are places to focus that will give a more accurate look on latency effects as a whole if one follows what I call the time scale reference.

Contrasting what we deem fast historically and what is on the horizon is not just interesting but important for baselines. Proper latency measurements become important milestones to feel the full effect of speed and acceleration.

Let’s say for example you had to take a trip from Atlanta, GA to San Francisco, CA in a truck carrying peaches. You had two routes to choose from. One would take 3 days and the other route would take 6 months. Now, if you wanted to take the scenic route, and you had tons of time, without the peaches, you might want to take the longer route. However, if you took 6 months, those peaches would smell quite bad by the time you got to San Francisco!! Using real world analogies like this on a time scale that we can decipher is important in order to see the differences and the effect it may have. Now why did I choose 3 days vs. 6 months for this example? A typical Solid State Disk has an average latency around 100 microseconds. Compare that to a rotational standard hard drive at about 5 milliseconds. If I scale these as to compare how drastic the time difference is between the two, it’s 3 days for the SSD and 6 months for the standard hard drive. Now, I can really see and feel the variance between these two mediums, and why a simple choice like this can make a gigantic difference in the outcome. Let’s now take it up to another level. What if we now had the capability to travel with our truckload of peaches to San Francisco in 6 minutes instead of 3 days or better yet how about 40 seconds? Today 6 minutes is possible as it applies to standard DRAM, but 40 seconds isn’t too far off, as this is representative of the Intel and Micron announcement for 3DXPoint NAND.

If I take these latency numbers and plug them into my datacenter, then I can start to see how simple choices can really have a negative or positive impact. You may now be saying to yourself, “Well if I go with SSD’s today, then tomorrow I basically need to rip and replace my entire stack to take advantage of the newer thresholds of latency, like the new 3DXPoint NAND, or even whatever is next!!” The exciting part is that you don’t have to replace your entire stack to take advantage of the latest and greatest. Your truck carrying those peaches just needs a turbo boost applied to the engine. You don’t need to buy a new truck, which is why choosing the proper platform becomes very important. Choosing the right truck the first time doesn’t tie your hands with vendor lock-in when it comes to performance.

In conclusion, I hope that one can now understand why proper base lines need to be established, and real world measurements need to be referenced. It’s not just a feeling of what is faster. We now are past the point of recognizing the true speed of something as a feeling. It’s the cause and effect or result that determines the effective threshold. Then tomorrow the threshold could lower with new innovations, which is all the more reason to have a time scale of reference. Without a reference point people become accustomed to the world around them, missing out on what it really is like to travel from Atlanta to San Francisco in 40 seconds or less. Don’t miss out on the latency innovations of today and tomorrow; choose your platform wisely

                        My Daughter was inspired to draw this for me based on this post! :) 


In Memory Computing - A Walk Down Memory Lane

I'm excited once again to be hosting guest blogger Bala Narasimhan VP of Products at PernixData. He will be doing a new series dedicated to Infrastructure Level In Memory Computing. This is a new way to think and design your environment for the future. Enjoy! 

In Memory Computing (IMC) has seen a revival in recent years. In this blog post we will walk down memory lane (no pun intended) and study the evolution of IMC. We will specifically focus on the role of IMC in conjunction with storage systems.

Here’s a quick timeline capturing the evolution of IMC.

IMC initially manifested itself as buffer caches in operating systems and databases. Buffer caches remain integral parts of these software systems to date. A buffer cache is a read cache for most recently accessed data. A hit in the buffer cache makes reads go faster because it avoids a disk access. Writes however must be synchronized with the underlying datastore. This means a buffer cache help with reads but never help with writes. In Memory Databases (IMDB) that hold all of the data in memory for faster query processing were first introduced in 1990s but didn’t make much of a dent. Recently IMDB have seen a revival and have become an important discussion topic in the database industry. One of the reasons for this revival has been the dramatic increase in the amount of RAM that servers can hold; surpassing a terabyte of RAM per server. IMDB, like buffer caches, work well for reads but require synchronization with a persistent media for writes and therefore incur performance penalties. The early 2000’s saw the introduction of memory caching libraries such as Memcached that allow applications to leverage the performance of RAM as a cache within the application logic. Application owners, via API, can invoke these libraries at well defined locations to use RAM as a cache. Unfortunately, any change in the underlying memory caching libraries or to the application behavior would mean a rewrite of the application altogether.

All of these approaches leverage RAM as a volatile cache. This means all these approaches are read caches and only accelerate parts of an application, if at all. More importantly, all of these approaches push the in memory computing into the application. This has a number of unfortunate side effects:

  • Applications become unnecessarily complicated. They have to maintain complicated caching logic and ensure consistency with the backing store.
  • The onus is now on the end user to figure out how best to leverage IMC. Everything from determining the appropriate buffer cache size to deciding which tables to pin in RAM is now an application owner decision.
  • The application owners do not always control the infrastructure on which the application is deployed. It is therefore unfortunate that they must decide how much memory an application should use for caching without understanding how the infrastructure (for example, servers) are going to be used and will evolve. In addition, applications owners do not always have a holistic view of other applications are running on the same server.
  • Leveraging memory management innovations become challenging. NUMA is a good example of an innovation that cannot be leveraged until an application is enhanced to leverage it even though NUMA itself is an infrastructure level innovation.

All of this has meant that the strengths of RAM as a data layer have not been fully exploited. While this may not have been an issue until now because most servers had only little amounts of RAM, say 32GB or 64GB, this is quickly changing. Servers with terabytes of RAM are not uncommon these days and this trend is only going to continue. The time has therefore come to rethink IMC from the ground up. Enhancing applications to do better read caching and putting the onus on the end user is not the answer. Future blog posts in this series will introduce a high performance, highly scalable way to leverage IMC across one’s entire infrastructure.

The Server-Side Storage Intelligent System Revealed!

PernixData introduced yesterday a revolutionary step forward in storage performance with the release of PernixData FVP 2.0. Several innovative features were revealed and a technology first was dropped on the industry. Frank Denneman has already started a great series on some of the new features. As to not let him have all the fun, I will also be covering some aspects to this new version as well!

The first big reveal was FVP transforming itself into an all-encompassing platform for storage optimization. Adding NFS & DAS to the already supported iSCSi, FC, FCOE list; which now completes all available connectivity options for VMware environments.

NFS support is obviously a welcome treat for many. It’s the support of local disk that might actually surprise some. Optimizing DAS environments I think will provide some unique cases for customers. (Future Post Coming) However keep in mind that supporting DAS doesn’t mean it voids use cases for VSA (Virtual Storage Appliance) software. PernixData is only accelerating the reads and writes, so if you require data services, then you may need to look at a VSA type of solution for your underlying local data at rest tier.

The biggest news in my opinion that really dropped the mic on the industry was the reveal of the first ever-distributed fault tolerant solution utilizing server memory for read/write I/O acceleration. Yep, you heard it right; accelerate those very important writes without the potential of data loss on volatile server memory is a gigantic leap forward. Look for more details around DFTM (Distributed Fault Tolerant Memory) in the coming weeks!!

I’m excited for the future and look forward to telling you more about these new advancements!