The Trusted Leader in High Performance Computing



SSD / Flash

Flash Options for All Conditions
As a relatively new storage option, SSDs have been called game changing by many storage pundits. Whether flash storage ever completely replaces spinning disks remains to be seen, and regardless is a long way out. In the mean time, SGI offers the best combination of flash, spinning disk, MAID, and tape technology.

Adopting flash as a storage medium is attractive for many reasons. Performance, density of performance, storage density, as well power and cooling savings can all be factors in choosing flash over spinning disk. For customers that will derive substantial benefit to their application from flash storage, there are cases where even the most expensive High performance Enterprise Class SSD will make economic sense from a density or $/IOPs perspective.

SSDs and Write Endurance

Write Endurance is defined as how much data can be written to the SSD over its lifetime. There are three parameters that affect write endurance of flash memory-based SSDs:

  1. A number of program/erase (p/e) cycles the raw flash can tolerate.
  2. Amount of flash in the SSD
  3. The amount of write amplification the SSD suffers.

Write Amplification is expressed as a ratio:

  • Number of bytes of flash that are programmed / Number of bytes of data written to the SSD

Factors that affect write amplification are:

  • Over provisioning percentage: the amount of raw flash versus the available capacity of the drive. When a block of flash is erased, there will often be valid data that must be preserved in that block. In general, the greater over provisioning the smaller the amount of valid data that will need to be preserved when a block is erased. For example, and SSD with 512GB of raw flash will suffer less write amplification if 400GB is available than if 480GB is available for user data.
  • Workload: Small random writes across the entire drive cause the most write amplification, because the drive will be forced to erase blocks in which most of the data is valid. Sequential writes, on the other hand, are the easiest to manage, as they will typically be programmed to sequential blocks of flash. Next time the blocks are erased, it may be the case that no data needs to be preserved. The less over provisioning there is, the more workload will affect write amplification.
  • Compression (if used by SSD): Compression has two effects. The obvious one is that compressed data will often consume less space than uncompressed data. A secondary, but often more important effect, is that more space is available for over provisioning. For example, if an SSD has 7% over-provisioning, but data written is compressed 20%, the effect would be the same as if over provisioning is 26%. That would increase random write performance by a factor of 3.
  • SSD hardware/firmware implementation: SSDs use sophisticated flash mapping schemes balancing several conflicting requirements, including performance, endurance, reliability, power, and cost. If the SSD uses XOR, which can greatly improve reliability, that can also increase write amplification.

    Typical write amplification for an SSD intended for a mixed read/write workload is 2-3.5. SSDs intended for client workloads or for mostly-read workloads in enterprise applications will suffer a write amplification of 10-70. In some cases, an SSD can be converted from a read-mostly workload to a mixed read/write workload merely be erasing it and then partitioning it to use 80% of the available capacity.

Sample endurance calculation:

  • Assume we have an SSD with 240GB of capacity and 256GiB (275 GB) of flash and 5000 P/E cycle MLC flash. Simple multiplication gives us an endurance of 1375TB. If write amplification is 8, then you could write a total of 171.8TB. Endurance specifications are typically provided assuming small random writes (either 4096 or 8192 bytes). Best case would be sequential writes, where write amplification could be close to 1. Worst case would be smaller writes, especially with an odd sector count.

In the list of SSDs in this document, endurance is specified assuming random 4K or 8K writes using the entire capacity of the drive. The writes are assumed to be aligned to either 4K or 8K. Where 4K endurance is provided, the 8K endurance will be very slightly higher. Where 8K endurance is given, 4K endurance will typically be between 50-75% of the 8K endurance, because the SSD is optimized for 8K and larger.

When partitions are not aligned, or I/O sizes are not a multiple of the specified size, endurance will be lower than specified. When extra over provisioning is used, writes are sequential, or the drive successfully compresses data, endurance will be higher - often much higher.

Endurance numbers for SSDs that feature compression are provided for the case where incompressible data is written (i.e. worst case). If such a drive is used in an environment where compression is successful (i.e. Boot/Root disk), then endurance can be much greater - as much as 5-10 times greater.