Picture from Google Images
All Flash Arrays: Dead Men Walking
All Flash Array (AFA) manufacturers may be rejoicing in the inevitable demise of spinning disk, but hyperconverged infrastructure (HCI) is increasingly upending the entire storage category. While an AFA may be faster and easier to manage than a traditional array, it’s still a SAN. Nutanix Enterprise Cloud is not only a better platform for flash than the AFAs, but also than other HCI solutions. Here are the 11 reasons why:
1) Dramatically reduced network latency effects
Nutanix HCI already bests AFA performance by eliminating network latency (see@vcdxnz001 post, Your Network is Too Slow and What to Do About it). Innovations such as NVMe and 3D XPoint amplify the advantage of storing data on flash or other Storage-Class Memory (SCM) next to the compute in a hyperconverged environment. Accessing data in a traditional model from an All Flash Array over a slower network negates the benefits of the faster flash/SCM.
Putting flash in a proprietary array at the end of a network designed for the latency of magnetic media instead of next to the compute intuitively makes no sense. This really boils down to simple physics where proximity matters. Flash should be directly connected, not remotely attached where it requires multiple hops, protocols, and performance-constraining controllers to be accessed. It’s just physics!
I/O path length for AFAs versus Nutanix
AFA vendors will often suggest faster networks and NVMe over fabrics that will offer lower latency and higher bandwidths. Nutanix enables customers to optimize the benefits of flash without the requirement to purchase expensive new storage fabrics that perpetuate legacy complexity.
Image from Long Virtual White Clouds by Michael Webster
2) Density advantage
Nutanix enables packing 92TB of flash, in addition to all of the server resources, into just 2U. AFAs require not just the array, but also the compute, the storage fabric and possibly lower-cost disk storage. All of this requires more power, rackspace, and cooling.
3) Commodity Hardware
Most AFAs utilize proprietary hardware, but this is a roadblock to quickly deploying new hardware innovations. All-flash arrays risk technological leaps that leave customers with obsolete products and facing forklift upgrades the next time they need more capacity. In today’s fast-paced technology environment, it’s the companies who leverage global economies of scale that succeed by leveraging the innovation driven by the world’s largest commodity hardware manufacturers.
Take the case of Sun Microsystems. Sun bet on proprietary hardware while the industry shifted to commodity servers utilizing the more cost-effective Intel-compatible microprocessors popularized in personal computers. Sun lost 80% of its value before being acquired by Oracle in a fire sale.
Violin Memory is another example. Violin was one of the first companies to introduce all-flash memory solutions to the marketplace. This was very cool and fast tech, with great engineering when they launched a decade ago.
But consumers had another idea. They loved the speed and reliability of solid-state drives (SSDs) which now can be found in almost every laptop, desktop and memory array. Even as the price of SSDs plummeted, Violin preferred to design its own proprietary field-programmable gate arrays. A sophisticated solution, perhaps, but no match for the rapid improvement of SSDs; Violin’s proprietary hardware quickly fell behind and the company has filed for bankruptcy.
The hyperconverged business is hardly the only example of a thriving enterprise technology built upon commodity hardware. All of the leading cloud providers also utilize commodity servers. Proprietary hardware, while once essential to protecting a company’s innovations, now hinders, or even destroys, a manufacturer’s ability to compete.
4) Distributed storage controllers
Most AFAs have physical, non-distributed, storage controllers that are easily saturated with traffic. Since the controllers are the bottleneck, adding more shelves of SSDs does not increase performance.
If we assume a single enterprise SSD is capable of delivering ~500MB/s of throughput, then a controller with dual 4Gb FC adapters is bottlenecked with only two SSDs. Even upgrading to dual 16Gb FC adapters only accommodates eight SSDs.
To overcome these limitations, AFAs must accommodate multiple adapters resulting in complex fabric configurations. But this inevitably hits the controller limits, forcing customers to purchase more AFA systems and creating more silos.
Contrast this with Nutanix where every time a node is added to a cluster, it’s also adding a virtual storage controller – enabling immediate enhanced performance. Resiliency is massively improved as loss of one controller has very little impact. This is why Nutanix can do non-disruptive 1-click upgrades and maintenance with very low impact.
5) Data locality
Imagine what would happen if 75% of the cars in Los Angeles were suddenly removed from the roads. Not only would traffic congestion quickly dissipate, but the city would benefit from other benefits such as fewer accidents, less road maintenance, reduced pollution, and so on.
Nutanix data locality similarly affects the data center environment by pulling the majority of read traffic off of the network; reads instead come from the local SSD within the node. Available network bandwidth is effectively increased for writes/end user applications improving not just storage performance, but also the application performance that the storage is servicing.
Capacity Performance: AFAs, which are typically limited to two physical storage controllers, hit a metadata bottleneck in scaling capacity that is limited by the amount of RAM/NVMRAM they have in a system. Adding SSDs, in most cases, does not improve performance.
At some point, the AFA customer must either upgrade to a bigger unit with more processing power, add complex fabric interconnection, or start creating silos. AFA manufacturers will say they can replace existing controllers with new faster ones, but despite the disruption and expense, that shifts the bottleneck to the network or possibly even to the existing flash medium.
Contrast this with Nutanix which, unlike AFAs, are not bottlenecked by two physical storage controllers. The VMs on every node are serviced by the Controller Virtual Machine (CVM) on that node. Every time a node is added to the cluster, a CVM is also added, thereby linearly scaling not just capacity, but also performance and resiliency, as well as expanding the management stack capabilities. Acropolis Block Services (ABS) and Acropolis File Services (AFS) enable Nutanix customers to scale physical and virtual workloads as well as file serving from the same Nutanix cluster, thereby eliminating silo inefficiencies.
Resiliency: Both resiliency and high availability are built-in across the entire Nutanix stack. Replication Factor 2 (RF2) or RF3 along with erasure coding (EC-X) enables superior fault tolerance for disk. Block awareness mitigates node failure, while sync and async-replication provides resiliency for entire datacenters.
All-Flash Storage-Only Nodes: Storage-Only nodes provide Nutanix customers the ability to scale compute and storage separately, thereby minimizing costs of their all flash environments.
Nutanix one-click upgrades reduce both complexity and risk involved with the upgrade process – there is no complex interoperability matrix or operational guidelines. Nutanix also simplifies the flash-based architecture by eliminating LUNs and their presentation by focusing on VMs rather than on storage constructs, and by including both centralized management and capacity planning.
Nutanix’s Simple and Intuitive Prism Management Dashboard
8) Workload Consolidation
AFAs must send information from the flash array across the network to the compute for processing. Beyond adding the aforementioned latency, this also requires additional queue management and overhead. CPUs can quickly become overloaded when simultaneously receiving small block, high IOPS and large block, high throughput application requests. To ensure consistent performance, AFA administrators must frequently separate OLTP & OLAP workloads from running on the same platform.
Nutanix gives the compute direct access to the storage. Servicing requests with limited overhead and consistent low latency enables mixing of workloads. And with Nutanix Acropolis Block Services, Nutanix becomes the storage backplane for bringing together different types of applications. Customers can even consolidate both physical workloads and virtualized workloads in the same cluster.
Additionally, AFAs tend to have block storage devices for blocks and flash arrays for blocks. With Nutanix, the storage is shared between block and file.
Nutanix enables optimal performance for critical apps right out of the box, even with multiple workloads. It eliminates the single point of failure challenge with storage access failover, self-healing, and ongoing data integrity checks. Storage performance is predictable, and no complex configuration or tuning is needed.
Non-disruptive software updates eliminates planned downtime, enhancing Nutanix’s appeal for hosting mission-critical applications. Maintenance windows for software upgrades and scaling become a thing of the past. Unlike almost all other HCI solutions, Nutanix has years of proven maturity and success in enterprise deployments of Splunk, Oracle, SAP, SQL Server, Exchange, and many other mission-critical applications (only Nutanix and VxRack are SAP-certified).
AFAs eventually run out of controller capacity, technology advances to the point where the existing AFA solution is comparatively uneconomical, or the equipment just gets old. In any of these cases, the AFA owner faces a forklift upgrade – a process that is typically expensive, complex and time-consuming. As a result, AFA owners typically purchase more capacity than required initially in hopes of having enough resources available to meet requirements four or five years down the road.
Nutanix owners never face a forklift upgrade, and therefore do not require purchasing more nodes than needed at any point in time. As technology changes, newer nodes can simply be added to the cluster with a mouseclick, and the software takes care of everything else. Nutanix eliminates the risk of under-buying.
Completely eliminating the need for storage arrays and storage fabric along with excess capacity up-front helps lower the CapEx cost for Nutanix. As the project footprint expands over the next few years, increasingly fewer nodes are required to run the same workload due to an increased density of VMs per node driven by both Moore’s Law and by performance enhancements in Nutanix software.
The CapEx for the project lifetime is thereby further reduced along with the associated rack space, power and cooling. Administrative requirements for Nutanix are also slashed – an IDC study found an average 71% reduction in administration time required for organizations migrating to Nutanix.
11) The Advantage of an Enterprise Cloud Platform
At the end of the day, it’s not just about the work, it’s about how you do it. Nutanix’s utilization of Web-scale architecture is a unique differentiator incorporating hyperconvergence as part of an Enterprise Cloud Platform. Distributed technologies such as Cassandra, NoSQL, MapReduce and Curator enable significantly higher performance and efficiency when optimizing all-flash environments.
Data Access: Classic tree-structured metadata query architectures (Btree & R&B) that work well in an array environments where metadata is stored in each physical controller are not optimal in all-flash HCI environments. In HCI, the metadata is distributed across many nodes – making tree-structured lookup inefficient. To combat this inefficiency, Nutanix utilizes big-data technologies such as Cassandra and NoSQL to enable very fast look-up and very high fault tolerance. No single point of failure exists.
Data Sorting: Unlike legacy 3-tier and other HCI approaches which sort data in the IO path, Nutanix evaluates it in the background, enabling better performance. The system scales as nodes are added allowing faster dedupe and compression with increased data locality. Seamless scalability enables rapid evaluation of whether to promote or demote data depending upon memory and storage tiers available.
Analytics: Even all-flash environments have different tiers of flash (performance & endurance). Metadata continues to grow and it can be difficult to cost-effectively keep it in memory or on the fastest tier.
Nutanix has again utilized a big data approach to solve this challenge. A custom-written version of MapReduce/Curator is used to determine key elements of the data including what is hot, compressible, and dedupeable. The same framework similarly determines what data needs to move to another node for data locality, what data has been deleted, and what data needs to be relocated or rebalanced – particularly in the event of failure.
These analytics enable deeper insight including trending, real-time analysis, proactive monitoring and root cause analysis, and alerting.
Timing In contrast to other solutions that rely solely on sub-optimal, inline compression and proprietary hardware for dedupe, Nutanix enables offline sorting with MapReduce/Curator. This enables more writes before deciding to compress or dedupe and avoids the requirement for a performance limiting centralized database.
Unified Cache Cache enables data locality. Deduplication makes it possible to store more data in this performance tier and maximize local cache hit potential. To maximize efficiency without limiting performance, Nutanix performs in-line local deduplication of context cache.
NVMe: Dead Man Running?
At least one of the legacy storage manufacturers is promoting NVMe as the future. But migration to NVMe is going to further amplify the advantages of putting the compute next to the data rather than across the network. It will accelerate the journey to extinction of all the fabric stretched monoliths – including AFAs.
Thanks for content and edits to @joshodgers @briansuhr @_praburam @Priyadarshi_Pd @sudheenair @binnygill @vcdxnz001 @RohitGoyal