.
Maximizing Benefits of External Flash Storage with NVMe !
June 2, 2021 | Written by: Deepak Yadav
Categorized: . | IT infrastructure
Share this post:
In external storage context, the age-old saying ‘Chicken or the egg’ holds true! Does high-speed storage media come first in order to drag high-speed connectivity or high-speed connectivity forces the storage media to rise to the occasion?
Regardless of the one which came first, the real need is for both to be present in the data center. The presence of both high-speed storage and high speed connectivity allows for modernization to genuinely move forward.
Let me share the brief tale of storage advancements in three phases:
Phase 1: SAS (Serial attached SCSI) based spinning media
In the early days of modern computing (1980s), as it involved external storage, we saw the birth of Small Computer System Interface which was simply referred to as SAS. It quickly became the de facto standard for facilitating communication between servers and disks, irrespective of whether the disks were local or remote.
The key point is that SAS was created to deal with disks which means that much of its code was designed to deal with rotating disk platters and physical disk head movement which could only handle a single request at a time. Thus, SAS was ideal for disks, but as a whole inefficient for dealing with solid-state devices.
SAS 3.0 launched in 2013 has 12 Gbps speed with transfer speed of 1200 MB/s and the best expected latency is about 3-5 millisecond.
Phase 2: Flash encapsulation as SSD (Solid State Drives)
With the invention of flash memory and its availability as non-volatile media, brought enormous capability of about 100x performance then a spinning media.
This kind of 100x performance was definitely not needed by our applications in the 2000-2010 era and that’s when media manufacturers introduced a encapsulate flash media called SSD that could deliver 10x performance as pluggable into existing external storage architecture, this improvement looked way sufficient for the most of the applications.
The flash memory was termed SSD because of the mimicking it did to integrate seamlessly into existing SAS compatible hardware connectors and software protocols.
“When Reading or Writing to an SSD, in spite of not having any moving parts, instructions must be sent which tell the non-existent drive head where it needs to move with respect to Data Tracks, Cylinders and Sectors. This is because it must follow the same communications protocol as the HDD it is mimicking”.
Let’s observe the difference in IOPS:
SAS 15K RPM Hard Drive can deliver about 200 IOPS whereas, an SSD can deliver about 2000-3000 IOPS. The response time varies around 1 millisecond.
Phase 3: Exploiting the real power of flash memory
The next era came from 2010 onwards, where the focus of storage performance shifted from only IOPS to Response times + IOPS (1 millisecond to 100 microseconds). The expectation of microsecond was not possible with spinning SAS drives or SAS SSD Flash. This incited the need for an efficient communication interface between host and the non-volatile flash memories.
In 2011, a group comprised of approximately 60 companies, formed an industry association to define a new storage interface protocol.
The purpose of NVMe (Non-Volatile Memory Express) was to eliminate old-school interface bottlenecks by introducing a more streamlined stack, with a new set of commands and a massive number of I/O queues for direct-connected storage. That streamlining means NVMe devices can achieve significant improvements over SATA and SAS in latency, IOPS, and bandwidth.
“There are over 200 Commands involved in SAS disk-based interfaces whereas NVMe only requires 13 Commands to communicate with Flash devices”
Below image depicts the way both protocols work and massive parallelism in NVMe protocol.
NVMe Flash devices can yield less than 100 microseconds of response time and more then 15000 IOPS per flash module.
To summarize here, NVMe defines an efficient interface protocol with three important benefits:
- The software code is leaner than SCSI and reduces the CPU cycle by about half
- It has massive IO parallelism exploiting multi-core CPU architecture
- It enables you to get the absolute most out of the flash
Disclaimer: The postings on this site are my own and don’t necessarily represent my organization’s positions, strategies or opinions.
Connect with me LinkedIn
Insurance Company Brings Predictability into Sales Processes with AI
Generally speaking, sales drives everything else in the business – so, it's a no-brainer that the ability to accurately predict sales is very important for any business. It helps companies better predict and plan for demand throughout the year and enables executives to make wiser business decisions.
Never miss an incident with an application-centric AIOps platform
Applications are bound to face occasional outages and performance issues, making the job of IT Ops all the more critical. Here is where AIOps simplifies the resolution of issues, even proactively, before it leads to a loss in revenue or customers.
How ICICI Prudential Life Insurance is Scaling Customer Care and Leveraging AI to Personalize Experiences
Organisations are constantly challenged to meet dynamic customer requirements and rethink ways to engage with them on their terms and as per their convenience. With customers at the core of decision making and business success, organisations are tuning to digital capabilities that can support new-age services. When done well, after sales service boosts the overall customer experience by providing […]