Test Procedures

Our usual SSD test procedure was not designed to handle multi-device tiered storage, so some changes had to be made for this review and as a result much of the data presented here is not directly comparable to our previous reviews. The major changes are:

  • All test configurations were running the latest OS patches and CPU microcode updates for the Spectre and Meltdown vulnerabilities. Regular SSD reviews with post-patch test results will begin later this month.
  • Our synthetic benchmarks are usually run under Linux, but Intel's caching software is Windows-only so the usual fio scripts were adapted to run on Windows. The settings for data transfer sizes and test duration are unchanged, but the difference in storage APIs between operating systems means that the results shown here are lower across the board, especially for the low queue depth random I/O that is the greatest strength of Optane SSDs.
  • We only have equipment to measure the power consumption of one drive at a time. Rather than move that equipment out of the primary SSD testbed and use it to measure either the cache drive or the hard drive, we kept it busy testing drives for future reviews. The SYSmark 2014 SE test results include the usual whole-system energy usage measurements.
  • Optane SSDs and hard drives are not any slower when full than when empty, because they do not have the complicated wear leveling and block erase mechanisms that flash-based SSDs require, nor any equivalent to SLC write caches. The AnandTech Storage Bench (ATSB) trace-based tests in this review omit the usual full-drive test runs. Instead, caching configurations were tested by running each test three times in a row to check for effects of warming up the cache.
  • Our AnandTech Storage Bench "The Destroyer" test takes about 12 hours to run on a good SATA SSD and about 7 hours on the best PCIe SSDs. On a mechanical hard drive, it takes more like 24 hours. Results for The Destroyer will probably not be ready this week. In the meantime, the ATSB Heavy test is sufficiently large to illustrate how SSD caching performs for workloads that do not fit into the cache.

Benchmark Summary

This review analyzes the performance of Optane Memory caching both for boot drives and secondary drives. The Optane Memory modules are also tested as standalone SSDs. The benchmarks in this review fall into three categories:

Application benchmarks: SYSmark 2014 SE

SYSmark directly measures how long applications take to respond to simulated user input. The scores are normalized against a reference system, but otherwise are directly proportional to the accumulated time between user input and the result showing up on screen. SYSmark measures whole-system performance and energy usage with a broad variety of non-gaming applications. The tests are not particularly storage-intensive, and differences in CPU and RAM can have a much greater impact on scores than storage upgrades.

AnandTech Storage Bench: The Destroyer, Heavy, Light

These three tests are recorded traces of real-world I/O that are replayed onto the storage device under test. This allows for the same storage workload to be reproduced consistently and almost completely independent of changes in CPU, RAM or GPU, because none of the computational workload of the original applications is reproduced. The ATSB Light test is similar in scope to SYSmark while the ATSB Heavy and The Destroyer tests represent much more computer usage with a broader range of applications. As a concession to practicality, these traces are replayed with long disk idle times cut short, so that the Destroyer doesn't take a full week to run.

Synthetic Benchmarks: Flexible IO Tester (FIO)

FIO is used to produce and measure artificial storage workloads according to our custom scripts. Poor choice of data sizes, access patterns and test duration can produce results that are either unrealistically flattering to SSDs or are unfairly difficult. Our FIO-based tests are designed specifically for modern consumer SSDs, with an emphasis on queue depths and transfer sizes that are most relevant to client computing workloads. Test durations and preconditioning workloads have been chosen to avoid unrealistically triggering thermal throttling on M.2 SSDs or overflowing SLC write caches.

Introduction SYSmark 2014 SE
POST A COMMENT

97 Comments

View All Comments

  • Valantar - Tuesday, May 15, 2018 - link

    Any chance you could test one of these drives with AMD's new caching solution? AFAIK the drives show up as regular NVME devices, so it should work in theory. Would be really interesting to see these solutions compared, and if Ryzen or Threadripper can make proper use of Optane caching through third-party software. Reply
  • Billy Tallis - Tuesday, May 15, 2018 - link

    I'll be setting up a Threadripper system this week to test both caching and NVMe RAID. Reply
  • Lolimaster - Tuesday, May 15, 2018 - link

    My only use for an optane drive would be for swap file, firefox/chrome cache/install/profiles and GTA5.

    But a 500GB 860EVO cost $169 with 300TB of endurance vs 365TB on optane, with the 860 offering 4x the storage... dunno.

    Their "low end" 118GB 800p needs to improve endurance to at least 1PB level to be a proper swapfile/browser/cache tool
    Reply
  • evernessince - Wednesday, May 16, 2018 - link

    So what's the point of this when AMD is giving away StoreMi with it's X470 boards? From what I've seen from reviews of the product, it works exceptionally well. It also doesn't require you to buy another drive and it can use much larger SSDs as a cache. Reply
  • CheapSushi - Wednesday, May 16, 2018 - link

    You can definitely ignore Intel's marketing pitch about these. But you can use ANY Optane drive, including ones mentioned here like ANY OTHER SSD out there. So you can make it work with StoreMi too. You have to decide which drive benefits your workload more and how and what your budget is. Optane has inherent benefits that beats out NAND is many ways. But again, just depends on what you want. The smaller GB ones are pretty damn cheap in my opinion. So worth just trying out. Reply
  • Svend Tveskæg - Wednesday, May 16, 2018 - link

    Reminds me of back in the days, when you could buy a weird plastic screen, that claimed it would turn your black and white television into a color-TV.... Reply
  • FunBunny2 - Wednesday, May 16, 2018 - link

    one of the distinguishing points, so to speak, of XPoint is its byte-addressable protocol. but I've found nothing about the advantages, or whether (it seems so) OS has to be (heavily?) modified to support such files. anyone know? Reply
  • Billy Tallis - Wednesday, May 16, 2018 - link

    The byte-addressability doesn't provide any direct advantages when the memory is put behind a block-oriented storage protocol like NVMe. But it does simplify the internal management the SSD needs to do, because modifying a chunk of data doesn't require re-writing other stuff that isn't changing. NVDIMMs will provide a more direct interface to 3D XPoint, and that's where the OS and applications need to be heavily modified. Reply
  • zodiacfml - Friday, May 18, 2018 - link

    Quite impressive but for 32GB Optane drive, I can have a 250 GB SSD.

    The Optane might improve performance for fractions of a second over SSDs for applications but it won't help during program/driver installations or Windows updates which needs more speed.

    I'd reconsider it for a 64 GB Optane as a boot drive for the current price of the 32GB.
    Reply
  • RagnarAntonisen - Sunday, May 20, 2018 - link

    You've got to feel for Intel. They spend a tonne of cash on projects like Larrabee, Itanium and Optane and the market and tech reviewers mostly respond with a shrug.

    And then everyone complains they're being complacent when it comes to CPU design. Mind you they clearly were - CPU performances increased at a glacial rate until AMD released a competitive product and then there was a big jump from 4 cores to 6 in mainstream CPUs with Coffee Lake. Still if the competition was so far behind you can afford to direct to R&D dollars to other areas.

    Still it all seems a bit unfair - Intel get criticised when they try something new and when they don't.

    And Itanium, Larrabee and Optane all looked like good ideas on paper. It was only when they had a product that it became clear that it wasn't competitive.
    Reply

Log in

Don't have an account? Sign up now