One Size Fits All?

Designing storage for enterprise applications such as databases, mail servers, and VDI involve benchmarking various storage devices during proof-of-concept (POC) phase. One of the intents of the POC is to evaluate behavior of storage under various load conditions, against various I/O profiles that are supposed to represent / simulate the actual applications. The storage that  stands out during the benchmarking will be a prime candidate for consideration.

The ability of the selected storage to meet the needs of the applications really hinges on selecting the right I/O profiles that most closely simulate the applications’ I/O behavior. Unfortunately, there is neither an easy way to determine an application’s I/O characteristics nor readily available I/O requirements spelt out clearly by the application vendor as is done for other resources (CPU, memory and network). Because of this, most data center architects often end up doing one of the followings –

  1. Use an I/O profile that a storage vendor tells them to use
  2. Use an I/O profile that is perceived to simulate an application (but really no proof to validate the assumption)
  3. Use a completely random profile that has got nothing to do with the application

In 2009, a blog was published on that attempted to disclose the I/O profiles of many enterprise applications. Although, this information is much better than random guesses, it does not capture all the phases of an application. Benchmarking with these profiles may result in selection of a storage that is sub-optimal to handle all the needs of an application. Various storage vendors have published their perceived view of the applications and are recommending certain IO profiles to use during a benchmark. A few vendors have started relying on Infrastructure-level analytics to learn about the behavior of various applications as viewed by their storage devices and started publishing the learnings (on their cloud portal and blogs). Using these machine learnings is certainly a good start.

One of the most important characteristics of an application’s I/O is the block size it uses when moving its data. My colleague Pete Koehler has done a very nice job of explaining what it is and why it matters in his blog. He also explains how one can get the information needed to understand the I/O profile of an application in this blog. As part of my day-to-day work, using the methods Pete explains in his blog, I have extracted the I/O profiles of many applications you run in your data centers. I thought these profiles can be useful to all those who benchmark their storage systems, but want to do so with realistic I/O profiles, understand their applications, or want to just look at some cool graphs.

What I will do is publish these profiles in separate blogs and discuss a bit about the profiles – why they are interesting? Why some of them could be devastating on your storage? etc. Here is the list of applications I have profiled:

  1. Microsoft SQL Server 2012
    1. User Database
  2. VMware Horizon 6.0
  3. Microsoft Exchange
  4. Cassandra
  5. Software Compilation

After looking at the profiles of all the above applications, one thing you will hopefully realize is that there is no one size fit all option when sizing storage for these applications. I know what you will be thinking after reading this – damn it! what should I use to benchmark my storage? I will try to summarize my findings and specify separate profiles that mostly closely resemble the profile of each application. But, more importantly, I want to educate the good folks who are tasked to run POCs to not fall for the blind recommendations that are no where close to reality, but to understand their applications using the tools that can help them do so and benchmark their storage based on what they learn.

Have fun!

About @_chethankumar
I currently work at PernixData as a Technical Marketing Engineer.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: