Performance of Off-the-Shelf Storage Systems is Leaving Their Enterprise Counterparts in the Dust

Everyone anecdotally knows that solid state disks (SSDs) are fast - like really fast when compared to hard disk drives (HDDs). It is just that the number of proof points coming from independent sources that conclusively demonstrate their performance advantage have been in short supply. Now proof points appearing on the SPECsfs website are confirming what people already suspect to be true: the performance of SSD-based systems is smoking fast with off-the-shelf SSD-based storage systems leaving their enterprise counterparts in the dust.

Quantifying performance on storage systems has always been a bit like trying to understand Russia. Winston Churchill once famously said in October 1939, "I cannot forecast to you the action of Russia. It is a riddle, wrapped in a mystery, inside an enigma; but perhaps there is a key. That key is Russian national interest."

Understanding published storage system performance metrics is very similar to understanding Russia. Performance metrics can be difficult for lay people to interpret and comprehend. However the key to understanding them is to realize that each storage system provider seeks to protect its own interests in the results that it publishes.

Keeping that thought in mind, a growing number of performance benchmarks being published on the SPECsfs website illustrate a significant gap up in terms of the number of I/Os per second (IOPS) that these systems can achieve as a result of more of them using either flash memory or SSDs. Whereas in 2010 the number of IOPS achieved were measured in tens or maybe hundreds of thousands of IOPS, in 2011 the industry saw for the first time published performance benchmarks in excess of one million IOPS per second.

In every case what made these numbers possible was the introduction of either SSDs or some type of performance acceleration module (such as what NetApp uses.) Coupling this technology with multiple front-end nodes (16 - 140) resulted in storage system providers achieving previously almost unheard of performance metrics in readily available commercial storage systems intended for business use.

Granted, organizations need to keep in mind that each of these providers had multiple opportunities to set up and test the performance of their storage systems, re-calibrate them to address bottlenecks, and then re-run the tests until they achieved the most optimal performance results. As such, organizations should not expect to see these levels of performance in their own environment.

But these benchmarks do suggest that if organizations can achieve even 50% of these published performance results, IOPS in the 500,000+ range may be well within their reach using these systems. Further, the fact that five enterprise storage systems from multiple providers broke the one million IOPS barrier in 2011 indicated that a fundamental technology change has occurred across storage systems as a whole.

Now in 2012 another performance breakthrough occurred. This time the three million IOPS barrier was broken by the Huawei OceanStor N8000 Clustered NAS Storage running the Symantec FileStore powered by Veritas Cluster File System (VxFS). Making this milestone particularly significant is that this performance benchmark was essentially achieved using off-the-shelf server and storage hardware and a commercially available clustered file system that may be installed on any hardware.

Making these high IOPS numbers achievable using the Veritas Cluster File System is its multi-volume file system. Using this feature, it may strategically place data on either SSDs or HDDs to optimize application performance while keeping hardware costs down.

These performance results should send more than a few shock waves through the storage industry as this was not an expensive high end storage system that had been architected and optimized for years or decades to deliver the best storage system performance possible. These results were essentially achieved using commercially available hardware and software that are, if my back-of-the-envelope calculations are correct, any where from one fifth to one tenth of the cost of the other storage systems listed above.

More notably, this configuration did not just edge these enterprise storage solutions out in terms of performance. It left them in the proverbial dust by more than doubling their performance. This suggests that the underlying architecture of enterprise storage systems is overdue for a dramatic design overhaul.

As everyone knows, storage system performance is a moving target and almost every year new thresholds are crossed. But the results published by Huawei and Symantec reflect a new reality in which almost any size enterprise with even a modest budget may acquire a storage solution that meets and exceeds their application performance requirements. More notably, these SPECsfs results also suggest that a broader revolution in storage system architectures is almost inevitable since such high IOPS may now be achieved at a comparatively low price point.