FVP Color Blindness Accessibility

With much respect and detail our PernixData engineers are interested in every facet of the customer experience. Something that may seem small to others can be a big deal to some. It’s with this that PernixData thinks about every feature in a holistic manner that all can appreciate. One such feature that fits this model is providing visual accessibility to those with color blindness. With 1 in 12 men and 1 in 200 women having some form of color blindness, it becomes important that the FVP UI is readable and understandable no matter the impediment.

 It was in FVP 2.0 that we made modifications to the colors in our UI to deal with the most common forms of color blindness: Deuteranopia (~5% of males), Protanopia (~2.5% of males), and Tritanopia (~.3% of males and females). For example, the “Network Acceleration” line graph was made to be  lime green. In addition all colors were tested with “Color Oracle” and application that simulates different forms of color blindness.

In addition, we made it easy to recognize each line on a chart uniquely identifiable. This was accomplished by providing the ability to toggle lines on/off. For example, if you aren't sure which line is referring to the datastore, just toggle the others off or toggle the datastore selection off/on, and this will clearly show the datastore line.

When designing the FVP interface it was also recognized that color is used as a secondary source of information that provides further insight and impactful information to the primary source. For example, in the host/flash device visualization, the color of the tiles (red, green, yellow) indicates the state of the relevant object. If there is a problem however, alarms/warnings also show an exclamation point on the tile in addition to the coloring of the tiles.

 

In memory computing - a survey of implementations to date

In Memory Computing Blog Series
Part 1: In Memory Computing - A Walk Down Memory Lane

In the series’ previous post we took a walk down memory lane and reviewed the evolution of In Memory Computing (IMC) over the last few decades. In this blog post we will deep dive into a couple of specific implementations of IMC to better understand both the innovations that have happened to date and their limitations.

The Buffer Cache

Buffer caches, also sometimes referred to as page caches, have existed in operating systems and databases for decades now. A buffer cache can be described a piece of RAM set aside for caching frequently accessed data so that one can avoid disk accesses. Performance, of course, improves the more one can leverage the buffer cache for data accesses as opposed to going to disk.

Buffer caches are read-only caches. In other words only read accesses can potentially benefit from a buffer cache. In contrast, writes must always commit to the underlying datastore because RAM is a volatile medium and an outage will result in the data residing in the buffer cache getting lost forever. As you can imagine this can become a huge deal for an ACID compliant database. ☺ 

The diagrams below depict how the buffer cache works for both reads and writes.

For reads, the first access must always go to the underlying datastore whereas subsequent accesses can be satisfied from the buffer cache

For writes we always have to involve the underlying datastore. The buffer cache provides no benefits.

The fact that the buffer cache only helps reads is severely limiting. OLTP applications, for example, have a good percentage of write operations. These writes will track the performance of the underlying datastore and will most probably mask any benefits the reads are reaping from the buffer cache. 

Secondly, sizing the buffer cache is not trivial. Size the buffer cache too small and it is of no material benefit. Size it too large and you end up wasting memory. The irony of it all, of course, is that in virtualized environments the server resources are shared across VM and are managed by the infrastructure team. Yet, sizing the buffer cache is an application specific requirement as opposed to an infrastructure requirement. One therefore ends up doing this with no idea of what other applications are running on the host or sometimes even without an awareness of how much RAM is present in the server.

API based in memory computing

Another common way to leverage IMC is to leverage libraries or API within one’s applications. These libraries or API allow the application to use RAM as a cache. A prevalent example for this is memcached. You can learn more about memcached here. Here’s is an example of how one would use memcached to leverage RAM as a cache:

function get_foo(foo_id)     
foo = memcached_get("foo:" . foo_id)     
return foo if defined foo      
foo = fetch_foo_from_database(foo_id)     
memcached_set("foo:" . foo_id, foo)     
return foo 
end
This example shows you how you must rewrite your application to incorporate the API at the appropriate places. Any change to the API or the logic of your application means a rewrite.

Here is a similar example from SQL. 

CREATE TABLE FOO (…..) WITH (MEMORY_OPTIMIZED=ON);

In the SQL example note that you, as a user, need to specify which tables need to be cached in memory. You need to do this when the table is created. One presumes that if the table already exists you will need to use the ALTER TABLE command. As you know DDL changes aren’t trivial. Also, how should a user decide which tables to keep in RAM and which not to? And does such static, schema based definitions work in today’s dynamic world? Note also that it is unclear if these SQL extensions are ANSI SQL compliant.

In both cases enabling IMC means making fundamental changes to the application. It also means sometimes unnecessarily upgrading your database or your libraries to a version that supports the API you need. And, finally, it also means that each time the API behavior changes you need to revisit your application.

In Memory Appliances

More recently ISV have started shipping appliances that run their applications in memory. The timing for these appliances makes sense given servers with large amounts of DRAM are becoming commonplace. These appliances typically use RAM as a read cache only. In the section on buffer caches we’ve already discussed the limitations of read only caches. 

Moreover since many of these appliances run ACID compliant databases consistency is a key requirement. This means that writes must be logged in non-volatile media in order to guarantee consistency. If there are enough writes in the system then overall performance will begin to track the performance of the non-volatile media as opposed to RAM. The use of flash in place of mechanical drives mitigates this to some extent but if one is going to buy a purpose built appliance for in memory computing it seems unfortunate that one can only be guaranteed flash performance instead.

Most of these appliances also face tremendous startup times. They usually work by pulling all of their data into RAM during startup so that it can be cached. Instrad of using an ‘on demand’ approach they preload the data into RAM. This means that startup times are exorbitantly high when data sizes are large. 

Finally, from an operational perspective rolling in a purpose built appliance usually means more silos and more specialized monitoring and management that takes away from the tremendous value that virtualization brings to the table.

Summary

IMC as it exists today put the onus on the end user/application owner to make tweaks for storage performance when in fact the problem lies within the infrastructure. The result is tactical and proprietary optimizations that either make sub-optimal use of the infrastructure or simply break altogether over time. Wouldn’t the right way instead be to fix the storage performance problem, via in memory computing, at the infrastructure level? After all, storage performance is an infrastructure service to begin with. For more information, check out http://www.pernixdata.com 

Infrastructure Level In Memory Computing Video

Watch Satyam Vaghani, CTO and Bala Narasimhan, VP of Products, at PernixData discuss Infrastructure level In-Memory Computing.
 
Learn how a new standard for in memory computing is forming in the industry where existing applications can take full advantage with zero changes, while future applications can become much easier to develop once they leverage the power of infrastructure level in memory computing.
 
Grab a cup a tea and enjoy!


FVP Upgrades Using VUM

Starting with FVP version 2.5, a new upgrade process was introduced. As always the vSphere Update Manager could be used to deploy the FVP host extension in the perspective vSphere cluster of hosts. However prior to 2.5 the FVP upgrade process needed to be performed using the host CLI. This required the removal of the old host extension before the new host extension could be installed. Now we have a new supported method where VUM can be used to deploy a new FVP host extension and also upgrade an existing one already installed without the process of manually removing the host extension first! 

Before you begin the upgrade process for FVP, make sure you have downloaded the appropriate VIB from the PernixData support portal. These are VIBs signed and designed for only FVP upgrades using VUM. 

The upgrade also includes putting the host in maintenance mode as required for certified extension level installs. This becomes much more seamless since VUM can handle the transition in and out of maintenance mode. Additionally VUM needs to completely satisfy the compliance of the upgrade, this means a reboot is required for FVP upgrades when using the vSphere Update Manager. 

Using VUM for upgrades is different than using the simple uninstall and install method at a CLI prompt. Essentially VUM installations can not run /tmp/prnxuninstall.sh to uninstall a previous host extension version, as there is no API or scripting capabilities built-in to the VUM product. 

This is why there is a dedicated VIB strictly for the upgrade of FVP. There is no current way to perform a live installation on a running ESX boot partition. This means that a reboot is required since the backup boot partition /altbootbank is used to update the host extension. Then after a host reboot,  the new host extension will be installed to the primary boot partition /bootbank for a compliant running ESX host. 

Once the host extension has been uploaded into the patch repository, it then can be added to a custom VUM baseline, while making sure it’s selected for “Host Extension”, since any other selection would prevent the completion of the upgrade. 


Once VUM has finished scanning, staging against the custom “Host Extension” baseline, (I called mine PRNX) then remediation of the hosts can take place. This is based on each host that is labeled with an X as “non-compliant”. Once a reboot has finished the remediation process will check for host extension compliance, this will ensure that the new host extension has been fully deployed, and if that is the case VUM will report back a check mark for “compliancy” 
As you can see the new method of using VUM for not only new installations but upgrades has made it that much more seamless to have FVP start transforming your environment into an accelerated platform. 

Why I Decided Not To Put Flash In The Array

My story starts about 3 years ago, where at the time I was the Information Systems director for a large non-profit in Atlanta, GA. One of the initiatives at the time was to become 100% virtualized in 6 months; and there were obviously many tasks that needed to be accomplished before reaching that milestone. The first task was to upgrade the storage platform, as we had already surpassed the performance characteristics for the current workloads. As with any project, we looked at all the major players in the market, we ran trials, talked to other customers, and did our due-diligence for the project. It was not only important for us to be mindful of costs being a non-profit but we wanted also to be good stewards in everything we did. 

The current storage system that we were looking to upgrade was a couple 7.2K RPM, 24 TB chassis’. We had plenty of storage for our needs but latency was in the 50ms range encompassing only about 3000 IOPs. Obviously not the best to run a virtualized environment on as you can see!! We looked at the early All Flash Arrays that were just coming out and we also looked at the Hybrid Arrays, all of them promising increased IOPs and lower latency. The problem was that they were not an inexpensive proposition. So, the dilemma of being good stewards and at the same time needing single digit latency with more than 50K IOPs was a challenge to say the least. 

About the same I met a gentleman that told me some magical stories that sounded almost too good to be true! This man’s name is Satyam Vaghani, the PernixData CTO, creator of VVOLS, VAAI and VMFS.  Soon after meeting Satyam, I was given the privilege of getting my hands on an alpha build of PernixData FVP. I ran and tested the product during the alpha and beta stages at which I in turn immediately purchased and became PernixData’s first paying customer. I had never purchased a product in Beta before, but I felt this product was out of the ordinary. The value and the promise were proved even in beta, where I didn’t have to buy new storage just for performance reasons and thus saved the organization collectively over $100,000. This wasn’t a localized problem; it was an architecture problem that no array or multiple storage systems could solve. So, if I were in that position today, I’m sure the calculation over 3 years would be close to $500,000 worth of savings, do to the scale-out nature of the FVP solution. As the environment grew and became 100% virtualized I no longer would have had to think about storage performance in the same way. I no longer would have had to think about the storage fabric connections in the same way as well. Talk about a good feeling of not only being a good steward but also astonishing the CFO on what was achieved. 

This to me validated the waste and inefficiencies that occur when flash is being used at the storage layer. Disk is cheap when used for capacity and so it has never made sense to me to cripple flash performance by putting it behind a network in a monolithic box that can have it’s own constraints and bottlenecks. 

Fast forward to today where flash is now much more prominent in the industry. The story is even stronger today, how can anyone not be conscientious about spending over 100K on a single array that can only achieve 90,000 IOPs with single digit millisecond latency? When someone can buy a single enterprise flash drive for $500 that does over 50K IOPs with microsecond latency, then the question that must be asked, can you defend your decision from the CFO or CIO and feel good about it?

Don’t get me wrong; I’m not saying FVP replaces storage capacity, if you need storage capacity, then go and purchase a new storage array. However, this doesn’t mean that you have to buy an AFA for capacity reasons. There are many cost effective options out there that makes more economic sense, no matter what the dedupe or compression rates that are promised!  

My personal advice to everyone is to be a conscientious objector when deciding to put flash in the array. It didn’t make sense for me 3 years ago and still doesn’t make sense today. 

In Memory Computing - A Walk Down Memory Lane

I'm excited once again to be hosting guest blogger Bala Narasimhan VP of Products at PernixData. He will be doing a new series dedicated to Infrastructure Level In Memory Computing. This is a new way to think and design your environment for the future. Enjoy! 

In Memory Computing (IMC) has seen a revival in recent years. In this blog post we will walk down memory lane (no pun intended) and study the evolution of IMC. We will specifically focus on the role of IMC in conjunction with storage systems.

Here’s a quick timeline capturing the evolution of IMC.

IMC initially manifested itself as buffer caches in operating systems and databases. Buffer caches remain integral parts of these software systems to date. A buffer cache is a read cache for most recently accessed data. A hit in the buffer cache makes reads go faster because it avoids a disk access. Writes however must be synchronized with the underlying datastore. This means a buffer cache help with reads but never help with writes. In Memory Databases (IMDB) that hold all of the data in memory for faster query processing were first introduced in 1990s but didn’t make much of a dent. Recently IMDB have seen a revival and have become an important discussion topic in the database industry. One of the reasons for this revival has been the dramatic increase in the amount of RAM that servers can hold; surpassing a terabyte of RAM per server. IMDB, like buffer caches, work well for reads but require synchronization with a persistent media for writes and therefore incur performance penalties. The early 2000’s saw the introduction of memory caching libraries such as Memcached that allow applications to leverage the performance of RAM as a cache within the application logic. Application owners, via API, can invoke these libraries at well defined locations to use RAM as a cache. Unfortunately, any change in the underlying memory caching libraries or to the application behavior would mean a rewrite of the application altogether.

All of these approaches leverage RAM as a volatile cache. This means all these approaches are read caches and only accelerate parts of an application, if at all. More importantly, all of these approaches push the in memory computing into the application. This has a number of unfortunate side effects:

  • Applications become unnecessarily complicated. They have to maintain complicated caching logic and ensure consistency with the backing store.
  • The onus is now on the end user to figure out how best to leverage IMC. Everything from determining the appropriate buffer cache size to deciding which tables to pin in RAM is now an application owner decision.
  • The application owners do not always control the infrastructure on which the application is deployed. It is therefore unfortunate that they must decide how much memory an application should use for caching without understanding how the infrastructure (for example, servers) are going to be used and will evolve. In addition, applications owners do not always have a holistic view of other applications are running on the same server.
  • Leveraging memory management innovations become challenging. NUMA is a good example of an innovation that cannot be leveraged until an application is enhanced to leverage it even though NUMA itself is an infrastructure level innovation.

All of this has meant that the strengths of RAM as a data layer have not been fully exploited. While this may not have been an issue until now because most servers had only little amounts of RAM, say 32GB or 64GB, this is quickly changing. Servers with terabytes of RAM are not uncommon these days and this trend is only going to continue. The time has therefore come to rethink IMC from the ground up. Enhancing applications to do better read caching and putting the onus on the end user is not the answer. Future blog posts in this series will introduce a high performance, highly scalable way to leverage IMC across one’s entire infrastructure.

My Role Transition

On May 1st, I started a new role in my career path at PernixData. It is with great delight that I’m now a Product Manager within the Product Management group. I look forward to working closely with Bala Narasimhan VP of Product Management. He and I have worked together on many other projects since the early alpha stages of FVP. So, in some ways this transition will feel like the early days!

With that said, I will be running many different projects in my new role. A renewed energy around blogging and staying close to the community is at utmost importance for me. In addition I will be helping with product collateral, documentation, roadmap items, and much more. It’s with this that I thank the wonderful teams and individuals that I have had the privilege to work with! Becoming the 1st customer to Product Manager is something that I could have only dreamed of.

Feel free to reach out and connect with me anytime, as part of my focus is to stay connected to a relevancy in the field, as this can only help any product-focused endeavors!

On a side note, I also thought it was very ironic that on May 1st, my father officially retired. He has been a minister and marriage and family therapist for many years. The timing couldn’t have been predicted, so I humbly congratulate him on his promotion to retirement and thank him for the many years of taking care of his family!

 

Nashville VMUG Experience

 
 

Yesterday, I spent the day at the Nashville VMUG Converge event. I have to say that I been to a lot VMUG’s over the years and all have their strengths and weaknesses, but in IMHO the leadership team in Nashville have a model VMUG organization, that depicts forward thinking and focus.

This particular event wasn’t your typical user conference. It was a more focused event that the Nashville VMUG group organizes ever year. With close to 150 attendees, it’s not user conference size but also no small event either. I talked to some that drove over 4 hours to be part of the day activities!!

This year what really caught my attention was the pilot program the VMUG leadership team put together around educating the community. They invited ITT Technical Institute students & staff to attend and learn about virtualization. We ended up having over 20 students show up just to experience what a VMUG was all about and learn more about virtualization and the ecosystem that surrounds it. The plan is to expand this to other technical colleges and organizations and maybe even setup a mentoring program. This to me is one example of what the VMUG should be all about. I believe we all learn more when we mentor others and interact; it’s not all about who knows the most or who is the most popular speaker at today’s event. If you ask me, it’s about sharing with each other and becoming better at what do each day, no matter who we are.

Great organization and great event!! Thanks for the hospitality yesterday!! Oh, the food was good too!! Who doesn’t like an Ice Cream Sundae in the afternoon!!! :) 

 

 

How Can Database Users Benefit - PernixData FVP

I'm pleased to introduce you to Bala Narasimhan, VP of Products at PernixData. He has a wealth of knowledge around databases, and has authored 2 patents for memory management in relational databases. It's my pleasure to have him featured in today's post. He is officially my first guest blogger! Enjoy! 

Databases are a critical application for the enterprise and usually have demanding storage performance requirements. In this blog post I will describe how to understand the storage performance requirements of a database at the query level using database tools. I’ll then explain why PernixData FVP helps not only to solve the database storage performance problem but also the database manageability problem that manifests itself when storage performance becomes a bottleneck. Throughout the discussion I will use SQL Server as an example database although the principles apply across the board.

Query Execution Plans

When writing code in a language such as C++ one describes the algorithm one wants to execute. For example, implementing a sorting algorithm in C++ means describing the control flow involved in that particular implementation of sorting. This will be different in a bubble sort implementation versus a merge sort implementation and the onus is on the programmer to implement the control flow for each sort algorithm correctly.

In contrast, SQL is a declarative language. SQL statements simply describe what the end user wants to do. The control flow is something the database decides. For example, when joining two tables the database decides whether to execute a hash join, a merge join or a nested loop join. The user doesn’t decide this. The user simply executes a SQL statement that performs a join of two tables without any mention of the actual join algorithm to use. 

The component within the database that comes up with the plan on how to execute the SQL statement is usually called the query optimizer. The query optimizer searches the entire space of possible execution plans for a given SQL statement and tries to pick the optimal one. As you can imagine this problem of picking the most optimal plan out of all possible plans can be computationally intensive.

SQL’s declarative nature can be sub-optimal for query performance because the query optimizer might not always pick the best possible query plan. This is usually because it doesn’t have full information regarding a number of critical components such as the kind of infrastructure in place, the load on the system when the SQL statement is run or the properties of the data. . One example of where this can manifest is called Join Ordering. Suppose you run a SQL query that joins three tables T1, T2, and T3. What order will you join these tables in? Will you join T1 and T2 first or will you join T1 and T3 first? Maybe you should join T2 and T3 first instead. Picking the wrong order can be hugely detrimental for query performance. This means that database users and DBAs usually end up tuning databases extensively. In turn this adds both an operational and a cost overhead.

Query Optimization in Action

Let’s take a concrete example to better understand query optimization. Below is a SQL statement from a TPC-H like benchmark. 

select top 20 c_custkey, c_name, sum(l_extendedprice * (1 - l_discount)) as revenue, c_acctbal, n_name, c_address, c_phone, c_comment from customer, orders, lineitem, nation where c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate >= ':1' and o_orderdate < dateadd(mm,3,cast(':1'as datetime)) and l_returnflag = 'R' and c_nationkey = n_nationkey group by c_custkey, c_name, c_acctbal, c_phone, n_name, c_address, c_comment order by revenue;

The SQL statement finds the top 20 customers, in terms of their effect on lost revenue for a given quarter, who have returned parts they bought. 

Before you run this query against your database you can find out what query plan the optimizer is going to choose and how much it is going to cost you. Figure 1 depicts the query plan for this SQL statement from SQL Server 2014 [You can learn how to generate a query plan for any SQL statement on SQL Server at https://msdn.microsoft.com/en-us/library/ms191194.aspx]

Figure 1
You should read the query plan from right to left. The direction of the arrow depicts the flow of control as the query executes. Each node in the plan is an operation that the database will perform in order to execute the query. You’ll notice how this query starts off with two Scans. These are I/O operations (scans) from the tables involved in the query. These scans are I/O intensive and are usually throughput bound. In data warehousing environments block sizes could be pretty large as well. 

A SAN will have serious performance problems with these scans. If the data is not laid out properly on disk, you may end up with a large number of random I/O. You will also get inconsistent performance depending on what else is going on in the SAN when these scans are happening. The controller will also limit overall performance.

The query begins by performing scans on the lineitem table and the orders table. Note that the database is telling what percentage of time it thinks it will spend in each operation within the statement. In our example, the database thinks that it will spend about 84% of the total execution time on the Clustered Index Scan on lineitem and 5% on the other. In other words, 89% of the execution time of this SQL statement is spent in I/O operations! It is no wonder then that users are wary of virtualizing databases such as these.

You can get even more granular information from the query optimizer. In SQL Server Management Studio, if you hover your mouse over a particular operation a yellow pop up box will appear showing very interesting statistics. Below is an example of data I got from SQL Server 2014 when I hovered over the Clustered Index Scan on the lineitem able that is highlighted in Figure 1.

Notice how Estimated I/O cost dominates over Estimated CPU cost. This again is an indication of how I/O bound this SQL statement is. You can learn more about the fields in the figure above here.

An Operational Overhead

There is a lot one can learn about one’s infrastructure needs by understanding the query execution plans that a database generates. A typical next step after understanding the query execution plans is to tune the query or database for better performance.  For example, one may build new indexes or completely rewrite a query for better performance. One may decide that certain tables are frequently hit and should be stored on faster storage or pinned in RAM. Or, one may decide to simply do a complete infrastructure redo.

All of these result in operational overheads for the enterprise. For starters, this model assumes someone is constantly evaluating queries, tuning the database and making sure performance isn’t impacted. Secondly, this model assumes a static environment. It assumes that the database schema is fixed, it assumes that all the queries that will be run are known before hand and that someone is always at hand to study the query and tune the database. That’s a lot of rigidity in this day and age where flexibility and agility are key requirements for the business to stay ahead.

A solution to database performance needs without the operational overhead 

What if we could build out a storage performance platform that satisfies the performance requirements of the database irrespective of whether query plans are optimal, whether the schema design is appropriate or whether queries are ad-hoc or not? One imagines such a storage performance platform will completely take away the sometimes excessive tuning required to achieve acceptable query performance. The platform results in an environment where SQL is executed as needed by the business and the storage performance platform provides the required performance to meet the business SLA irrespective of query plans.

This is exactly what PernixData FVP is designed to do. PernixData FVP decouples storage performance from storage capacity by building a server side performance tier using server side flash or RAM. What this means is that all the active I/O coming from the database, both reads and writes, whether sequential or random, and irrespective of block size is satisfied at the server layer by FVP right next to the database. You are longer limited by how data is laid out on the SAN, or the controller within the SAN or what else is running on the SAN when the SQL is executed.

This means that even if the query optimizer generates a sub optimal query plan resulting in excessive I/O we are still okay because all of that I/O will be served from server side RAM or flash instead of network attached storage. In a future blog post we will look at a query that generates large intermediate results and explain why a server side performance platform such as FVP can make a huge difference.

 

FVP Management Database Design Decisions

When deciding which database model to use for FVP, it’s important to understand what the goals are in using FVP and the growth potential for the platform. Upon installation, FVP management service builds and connects to a “prnx” SQL database instance. This database is responsible for receiving, storing and presenting performance data. All time series data for all performance charts displayed in the FVP UI are stored in this database, in addition to management metadata as it relates to configurations. Keep in mind however neither the management server nor the FVP database needs to be operational for read/write acceleration to continue during downtime. 

PernixData management server is also responsible for managing fault domain configurations and the host peer selection process for write back fault tolerance. This information is also kept current in the “prnx” database so that any host or cluster changes can be kept accurate for FVP policy changes. This is why it’s imperative that FVP maintain a connection with the vCenter server, so that inventory information can be collected and maintained. 

It was decided early in the FVP design phase not to recreate the wheel and take advantage of already robust operations in SQL server. One of these decisions was to implement SQL rollup jobs into practice for FVP. The SQL rollup job is responsible for keeping only the current valuable data while providing an average for historical reference. Instituting the SQL rollup process lowers the latency and overhead of FVP having to implement the averaging operations. This means all data stored in SQL is not moved nor massaged outside the context of SQL, this provides the security and performance benefits to FVP as an acceleration platform. 

Since part of the SQL server responsibility is to store FVP performance data, it’s important to only store as much data that is relevant and useful. Currently FVP management server only requests 20-second performance samples on all FVP clustered VM’s on each enabled host. This is run using multiple threads so that multiple CPU cores can be utilized for efficiency. During a 24-hour period a large amount of data could be archived. In this case, FVP has a purging schedule that runs every hour to purge all 20-second samples older than 24 hours. This only happens after a SQL rollup has completed within each minute and hour time period averaging the 20-second samples. 

Every minute there are 3 samples (20 seconds each) that are averaged. At the 1 Hour mark, a SQL rollup job runs and at completion FVP will purge all 20-second samples older than 24 hours. In order to view the 20-second samples before the rollup, then look at the performance statistics that are 1 hour or less in the FVP performance UI.  After the 1-hour interval all 20-second samples are discarded after the first SQL rollup and then permanently removed after the purging operation 24 hours later. 

In order to determine a proper SQL capacity for this amount of data, one needs to know how many VM’s they plan to accelerate with FVP and what the potential is for continued expansion. Currently over 80% of the “prnx” database is used to store performance related metrics and this 80% also makes up the majority of data churn within the platform. This means calculating for the 80% will provide ample room for FVP’s operations. 

The PernixData Management Server will insert 1 row (record) in the DB table every 20 seconds for each VM. This can be approximated that each VM will store ~ 1.6KB amount of data every 20 seconds. This data also takes into account the index size for each VM that is referenced. 


If considering SQL Express with a 10GB limitation, knowing the effective data added each day becomes an important piece of information. This design decision could hamper long-term storage or the acceleration of a large number of VM’s. Whether SQL Express is chosen or not, it’s a best practice to either choose “Simple” Mode or have a regular scheduled SQL backups so that log truncation can help limit the continued growth of the SQL log. 

Knowing the approximate data added to the DB each day for said number of VM’s will provide the expectancy when one would reach a 10GB capacity for SQL Express. If for example you have 100 VM’s accelerated with FVP, it will take about 400 days, but for a 1000 VM’s the limitation will be reached in as little as 40 days! 

To understand how our UI displays the averages based on the samples and purging process, below is a chart that illustrates the number samples taken and the average based on the time displayed. Keep in mind whether choosing a custom time range or using the predefined time ranges in the FVP UI, all result in the same samples and averages as indicated in the chart below. 

As you can see it’s important to not only understand the metrics that you are referencing but design appropriately for database sizing and retention, taking into account PernixData FVP’s growth within your virtual environment. 

Shuttle DS81 Review

This past VMworld I had the opportunity to learn about a new server platform that really piqued my interest. The Shuttle DS81 received a refresh last year that gave it some nice features that I personally think puts it in a class all of its own. 

Shuttle was nice enough to send me the Shuttle DS81 to demo. So over the past couple of months I put the unit through it’s paces and even did some FVP testing against it!! 

Here are some of the unique features that caught my eye for making this a great home lab machine.  

•    VMware Ready
•    Only 43 mm thick. 
•    4K Support (2 Display Ports & 1 HDMI Port) w/ dual monitor support
•    Certified to operate in harsh, hot environments up to 122 °F or 50 °C – Great for my hot Atlanta summers!! ☺
•    Dual Gigabit Ports – Nice for teaming and you still have the USB ports. 
•    Low Noise, with 90W power.  I could hardly hear it run, it was very quiet. 
•    2 x Mini PCIe Slots 
•    1 bay for a 6Gbps SATA SSD drive- I used the built-in SD card slot for ESXi 

The configuration that I requested came with an Intel Haswell i7 3.2GHZ Quad Core processor and 16GB PC3-12800 1600MHZ DDR3 Memory. There are ton of different configuration options available. See list

When I first received the DS81 I installed ESXi 5.1 without issue. Then I decided to test ESXi 5.5 support, which is where I had to make some NIC driver adjustments. After doing some troubleshooting, I found that 5.5 didn’t include the correct NIC driver, a common problem with certain adapters. Since ESXi 5.5 at the time of testing didn’t include the Realtek 8111 LAN adapter drivers built-in, I went through the outlined process on a new 5.5 build as outlined in this post and all has been fine and without hiccup over the past 3 months since. 

The part that I really wanted to test and see what was possible with a unit this size was how well it stood up against heavy I/O demands. Naturally I wanted to test the performance using FVP.  So, I decided to use 12GB of available memory for my acceleration resource. I could have added an SSD to the unit, but since I had memory available, it was the easiest to test against. To my amazement the 16K, 50% Read workload on the Shuttle DS81 kicked out some impressive results. 

Over 115,000 IOPs at 20 microseconds is nothing short of spectacular. You may think that I have some fancy stuff on the storage backend. Well, if you call my $800 Synology box fancy then yes! ☺ You can also see in the screenshot that datastore latency isn’t impressive!! Any workload that I haven’t pernix’d is feeling the pain!! 

I truly believe there are many use cases for the Shuttle DS81. I hope to be adding some of these units to my lab as it grows. The power, cooling and sound pollution savings pays for itself at under $800 per configuration chosen. 

FVP Linked Clone Optimizations Part 2

In part 1 of this series, I talked about replica disk optimizations that FVP provides for your linked clone environment. In part 2 the focus will be on the different use cases for persistent and non-persistent disks and how it relates to the acceleration that FVP can provide to your VDI environment. 

I often hear confusing remarks about what some may call a persistent desktop and a non-persistent desktop. I have found that at times this terminology is based on confusion between a linked clone and a full clone. It also makes a difference what criteria one bases their understanding of a non-persistent or persistent desktop. For example, if you just look at linked clones, you will notice that several disks are non-persistent and persistent, depending on your design decisions. If one looks only at a dedicated linked clone with windows profile persistence then some may articulate this linked clone as a persistent desktop. 

The interesting thing is that Horizon View doesn’t refer to a linked clone in this context. The only time Horizon View refers to a persistent or non-persistent desktop is under the context of refreshing a cloned desktop. In other words, it doesn’t mean that just having a linked clone makes you a non-persistent or even persistent VDI environment. 

I also think some of the confusion revolves around the use of dedicated vs. floating assignment of linked clones. The dedicated configuration assigns each user a dedicated desktop, so, if the user has multiple sessions, they will always reconnect to the same desktop by default. In a floating configuration the user is assigned to a pool of desktops. This means they could login to a different desktop with each new session. The only way to keep windows profile persistence in the floating configuration scenario is use persona management solution outside the default configuration of view composer. 

So, when an admin decides to use a dedicated linked clone, view composer gives the option to redirect the windows profile to a persistent disk. This will provide user personalization persistence during refresh, recompose, and rebalance operations. This is an optional setting as seen in the screenshot. The default disk size is 2GB. 


When one chooses a floating assignment for linked clones, view composer does not provide an option for a persistent disk, this means that no user personalization will be retained after a refresh, recompose or rebalance operation. If you chose not to redirect the windows profile, then the data would be stored on the non-persistent delta disk. In either case, both read and write I/O will be accelerated with FVP. However, there will be a longer warm up time for read acceleration when using the non-persistent delta disk for user profiles, as this will depend on how frequent the refresh, recompose and rebalance cycles are. 

Whether you select floating or dedicated assignments and choose some level of windows profile persistence or not, FVP will automatically accelerate reads and writes for all disks that are part of the desktop VM. In the past, the choice on when to schedule a recompose, rebalance operation came with varied importance. Now with FVP offloading IO from the storage array, a refresh, recompose or rebalance operation can provide some breathing room for these tasks to finish without impact to the production environment.

Delta Disk:
The delta disk is probably where most desktop I/O will be seen from a linked clone. The delta disk becomes active as soon as the desktop is booted from the replica disk. Any desktop changes are stored on the delta disk and so depending on the user and the desktop use case, the I/O profile could vary drastically. This will not impact FVP negatively, as FVP will keep context on which disk is more active and thus provide the resource intelligence for acceleration no matter the use case. 

Disposable Disk:
A default configuration will have a separate non-persistent disposable disk 4GB in size. Having this as separate disk is recommended since it slows the growth of the delta disk between refresh, rebalance, and powered off tasks. This disk contains temp files, and the paging file, so FVP can help normalize OS operations by accelerating reads and writes associated with the disposable disk. If you choose not to redirect, then this data will reside on the delta disk. There is no negative impact to FVP on either option chosen. However it’s a best practice to help control the growth of the delta disk between refreshes, and so separating the non-persistent disk will help alleviate bloated delta disks. 

Internal Disk:
There is an Internal Disk that is created with each cloned desktop. This disk is Thick Provision Lazy Zeroed, with a default size of 20mb. This disk stores Sysprep, QuickPrep and AD account information, so very little IO will be realized from this disk. Keep in mind that this disk is not visible in windows, but it still has a SCSI address, so FVP will still recognize the disk and accelerate any I/O that comes from this disk. This is another advantage of being a kernel module, as FVP will recognize disks not mounted to the windows OS and yet FVP will still do its magic of acceleration. 

As you can see no matter the configuration, FVP will automatically capture all I/O from all disks that are part of a given desktop clone. Depending on the configuration, a desktop clone can have several disks and knowing when or which disks are active or are need of given resources at any given point, is not an easy task to determine. This is exactly why PernixData decided to develop FVP, a solution to take the guesswork out of each disks IO profile. This means the only item you are tasked with, is whether you accelerate the desktop or not! Talk about seamless and transparent, it doesn’t get any better than that!! 

FVP Linked Clone Optimizations Part 1

PernixData has some of the best minds in the industry working to provide a seamless experience for all differing types of workloads. Operational simplicity in a platform such as ours doesn’t mean there is a lack of complex functionality. Au contraire, FVP is truly a multiplex system that can dynamically adjust and enhance numerous workload attributes.  One popular example of this is the acceleration and optimization of Linked Clone technology by Horizon View.

The beauty of our linked clone optimizations is that it is completely seamless. This means you will not have to make any configuration changes to your existing VDI environment nor modify any FVP settings. No changes to the Horizon View Manager or other Horizon View products (e.g. ThinApp, Persona Management, Composer or client). It also doesn’t matter if you are using a persistent or non-persistent model for your linked clones, FVP will accelerate and optimize your entire virtual desktop environment.

It’s common to see a virtual desktop with many disks. Which may include an operating system disk, a disk for user/profile data, and a disk for temp data. No matter how many disks are attached to a virtual desktop, it doesn’t effect how FVP accelerates IO from the virtual machine. FVP’s intelligence automatically determines where IO is coming from and which disk is in need of additional resources for acceleration. The admin only decides which desktop (Linked or Full Clone) is part of the FVP cluster. This could comprise a mix of persistent, and non-persistent disks as seen in the diagram. FVP will automatically accelerate all IO coming from any persistent or non-persistent disk that is part of the desktop clone. 

As seen in the above diagram a linked clone environment can comprise several disks depending the configuration. This by far is the most confusing part IMHO. When you create a linked clone for the first time, you realize you have all these different disks attached to your clone and you may have no idea what they are for or even why the differing capacities.  Why are some persistent and some non-persistent, and which configuration works best for FVP acceleration? I will save these topics and more for part 2. However the replica (Base) disk is what I’m going to focus on in this post. 

In a Horizon View environment a full clone and a linked clone both have an OS disk, except that a linked clone will use a non-persistent delta disk for desktop changes and a replica (base) disk from the parent snapshot for image persistence. This delta disk will be active with any desktop operations, which makes it a prime candidate for acceleration. 
In addition to accelerating reads and writes for cloned desktops, FVP will automatically identify the replica disks in a linked clone pool and apply optimizations to leverage a single copy of data across all clones mapped to said replica disk. 

Note: Citrix's XenDesktop technology works essentially the same way with FVP instead of a replica disk, its called a master image. 

As seen in screenshot below, FVP will automatically place the replica disk (Linked Clone base disk) on an accelerated resource when using linked clones. In addition FVP only puts the active blocks of the desktop clone on the accelerated resource, which lowers the capacity required for the replica disk on the accelerated resources. It’s after the first desktops boot in the pool; that all ensuing clones will take advantage of reading the common blocks from the replica disk on the accelerated resource. If any blocks are requested that are not part of the replica disk, FVP will again fetch only the requested block and add it to already created replica disk. The same is true for any newly created linked cloned pools. A new replica disk will be added to the acceleration resource based on the new linked clone pool. This will be visible under FVP’s usage tab as a combined total of all active replica disks for a given acceleration resource. As you can imagine that only adding the active blocks of the replica disk provides a huge advantage for using memory as an acceleration resource. Windows 7 boot times in 5 -7 seconds are not uncommon in this scenario. 

screenshot.png

FVP maintains these optimizations during clustered hypervisor operations such as vMotion. This means that if desktop clones are migrated from one host to another the desktop clones are able to leverage FVP’s Remote access clustering to read blocks from the replica disk on the host where the desktop clones migrated. This only happens on a temporary basis, as FVP will automatically create a new active block replica disk on the new primary host’s acceleration resource. It’s through FVP’s Remote Access clustering and any new desktop clone reboots that a new replica disk for the desktop clones on the new host will be created or updated for local read access efficiency. 
If the desktop clones are in Write Back mode, write acceleration continues automatically once the desktop clones migrate successfully to a new host, irrespective to the replica disk optimizations. 

The diagram below outlines the process where a replica disk is first created on a primary host and then the first desktop clones migrate to a new host. This process of creating a new replica disk on the new host happens only one time per linked clone pool, all subsequent cloned desktops matched to designated replica disk will gain the benefit of any future migrations. 

 When the desktop clones are booted, (1) the clones will request blocks from the assigned replica disk in the pool. FVP will intercept the requested blocks, which will be (2) copied into the acceleration resource that has been assigned to the FVP cluster. All future desktop clone boot processes will read the blocks from the acceleration resource instead of traversing to the assigned datastore where the full replica disk resides. If any changes are made to the original replica disk through a recompose or rebalance operation, then this process will start all over again for the linked clones.  (3) When the desktop clones are migrated to a new host through DRS, HA or a manual vMotion operation, (4) FVP will send read requests to the host where the desktop clones migrated. (5) The blocks are coped back to the new hosts’ acceleration resource, (6) so that any future requests are acknowledged from the new local host. A reboot of any linked clone during this time will also copy all common blocks into the new local host’s acceleration resource. 

As you can see from a VDI perspective FVP can truly make a difference in your datacenter. Gone are the days when one had to architect a separate environment to run virtual desktops. FVP can break down the technology silos and allow the virtual admin to truly scale performance on demand without the worry of storage performance constraints.
 
More Information on using PernixData FVP with your VDI Infrastructure. 
 

 

A New Focus

It might be a surprise to some that I decided to discontinue my Cloudjock blog in name and design only. This change has been something on my mind for some time now. A new year and new goals propelled me to refocus my community engine to be more direct and frequent. The first task was to focus more on my own brand "Todd Mace". I have found it difficult at times to remember a blog name and/or the contributor/author behind it. This seems to create a disconnect between the blog and the real person behind it. I'm not saying that having a blog name different than your real name is a bad thing, for me it just makes it more personal and hopefully will help build my brand in the community more. 

"That's been one of my mantras - focus, and simplicity..."  Steve Jobs

.io doesn't mean I'm from the British Indian Ocean Territory! :) It was just a clever way to illustrate my intensions around the content featured! You will find going forward more focused posts on PernixData FVP, including many other passions of mine! It is my goal to grease your cerebral i/o for further discernment on this new platform! :) With this I invite you to reach out and give me feedback as we grow together. This will only further the enrichment of the community I believe! 

I will start forwarding cloudjock.com to this new site, but in the meantime, please spread the word about this new site. Thanks. 

Perception vs. Reality in Storage Performance

I’m sure we have all heard the saying that perception is reality, however this doesn’t mean it’s correct reality. In other words, it can be a false reality; the perceived notion doesn’t in of itself make it true.

 

A recent tweet response from Duncan Epping: 

I believe this assertion from Duncan is on the money. I talk to many customers that claim storage performance isn’t a problem. I then look at their storage latency and it averages around 20ms or more, even though we would most likely agree that 20ms is slow, the customer can seem perfectly happy with the current results.

This dichotomy I find fascinating as I think it also relates to the smartphone industry. For example, when I upgraded to the iPhone 6, I immediately noticed a performance increase; however after several weeks of use, I wasn’t aware of the performance increase, as I became accustom to the new effect. It wasn't until I picked up my daughters iPhone 4S and started using it, when I then realized again the app performance and usability that I gained from going with the newest device.

The old saying that you don’t know what you’re missing until you try it, can also apply when talking storage, and application latency.

Frank Denneman’s latest post on knowing the application I/O workload characteristics is one of the most difficult challenges I believe in today’s virtual datacenter. The true utopia as a virtual admin is to understand the full workload pattern over the application lifecycle as his post delves into. This is IMHO is an area where there is a false perception. Many believe that they understand their workloads, at least until something goes awry and it challenges them to think otherwise. There are a ton of reasons for this, i.e.. No good tools, App Dev. and/or Stakeholder communication disconnects, etc. As an application owner, how do you know what the app is capable of delivering? Do you know the perception of your end users or have they now accepted a false reality? 

Whether it’s a false perception or false reality the end result is the same. The admin doesn’t know what they are missing or has a capable tool to help them determine the actual reality and thus correct or challenge his or her perception.

It’s my hope that in 2015 more of us will test and challenge our perceptions, so that we can have a more accurate reality. Who knows what you may be missing!!!

 

 

2015 Goal: I'm going to be a Navy Seal

Over the holidays I spent several hours reading! One book that I thoroughly enjoyed was “The Way of the SEAL” by Mark Divine. I found the text very applicable for my vocation and good instruction for mental toughness to succeed in any personal or team endeavor.

As you know the U.S. Navy Seals are an elite group that only the select few have the privilege to serve. I seriously doubt I have the physical abilities to ever be part of the Navy Seals in my lifetime. :) However it’s my goal for 2015 to think like a Navy Seal and that’s exactly what Mark Divine (former Navy Seal) will help you achieve. If you can think and process like a Navy Seal, then it will help you be a better leader and also help one overcome difficulties that can be distracting at multiple levels.  

This is not your typical leadership book. You will find that Mark’s focus is on the character of leadership instead of developing a skill to be a better leader.


“If you lack an underlying commitment to self-mastery and growth, even the best theory won’t help yourself or a team to success.”

In other words a balanced heart and mind need to be in sync to achieve the integrity of greatness! The steps to achieve this are wonderfully illustrated and I can almost guarantee that you will be enlightened through the focus and execution of the process. 

Unless you are already a Navy Seal, I highly recommend this book to anyone who wants to make a difference in yourself and/or team!!

Happy New Year!! 

Storage Features Responses

In order to finish my Storage Features Survey, I have collected the responses from the community on how they view primary storage from a virtualization, enterprise worthy perspective. 

Most Popular Enterprise Storage Feature Responses (rounded):

Multiple Controllers - 70% of Respondents

Thin Provisioning - 65% of Respondents

Replication Capabilities - 60% of Respondents

Snapshot Capabilities - 60% of Respondents

High Availability - 60% of Respondents

Non-Disruptive Upgrades - 60% of Respondents

VAAI Support - 55% of Respondents

RAID Support - 50% of Respondents

Flash Technology - 50% of Respondents

Capacity/Shelf Expansion -50% of Respondents

There wasn't any big surprises, but there were a couple interesting responses. One of the higher responses came in for "Replication Capabilities", I knew this would be popular, however I did not envision it being as high on the list, since there are many other technologies in the market that can tackle replication from a different level in the stack. i.e. Zerto, Veeam. The other popular response that was interesting was "Flash Technology" at the array level. I know flash is very popular right now, but as you might know it's not something I agree with 100% of the time. PernixData can use flash at the host level for storage performance and thus can negate the use of flash at the array. I'm not saying this is the case for all environments, but it's enough to change the storage landscape. 

The biggest response was for multiple controller support! This isn't surprising, since high availability & non-disruptive upgrades was also listed high on the list from respondents. 

Did any of the responses surprise you or do you think this encompasses the most common requested enterprise features for primary storage? 

Thanks for participating in the Storage Features Survey! 

Poll: Storage Features

I was recently talking with some peers about what storage features fit into the category of enterprise worthy and are considered "must have’s" in regards to importance.

This got me thinking about market conditions today and how we are flooded with so many new storage systems/features. This not only can confuse consumers but I believe can change perceptions on what is really important or reality! It's my intent in this post to study what truly makes up a enterprise storage system and what are the must have features of today. 

The first part of this study is to publish a poll on what you believe to be an enterprise must have feature in the storage market. Please select only the must have’s and/or what is important to you and your company!

I will publish the anonymous results once everyone has had a chance to vote.

Keep in mind that these features are only intended to apply at the primary storage level for your array in a 100% VMware environment!  

This post is not sponsored nor endorsed by my employer! Personal Passion Only! 

http://goo.gl/forms/9HDCkknbvO

Synology NFS VAAI

A few days ago Synology released their latest DSM version 5.1-5004. I have been impressed with the amount of innovation and enterprise features Synology brings to the storage market. It’s with this new release that Synology adds NFS VAAI support to the already ISCSI VAAI support. The two currently supported primitives for NFS are Full File Clone and Reserve Space. I hope Synology also considers adding additional primitives for NFS support in the near future!

In order to take advantage of the two new primitives you need to install the NFS VAAI plugin. This is basically a VIB that can be installed on each host through update manager or esxcli commands.

1)    Copy the VIB to a temp location /tmp

2)    Run esxcli software vib install –v esx-nfsplugin.vib

3)    Reboot host

Once installation is complete you can check to see if the VIB was successfully installed by running:  esxcli software vib list | more

This will list all VIBs installed. As you can see I have the PernixData VIB installed in addition to the NFS VIB. 

 

You can also check in vCenter if “Hardware Acceleration” says “Supported” You can see that my ISCSI and NFS datastores from my Synology both are supported! Since PernixData now supports NFS, I can now start realizing IO accelerartion for VM's using NFS!