As a performance junkie, I’m less concerned about the security risks of the Spectre and Meltdown vulnerabilities—after all, there are no known exploits in use today—than I am about a performance hit from the fixes.
And from what I’m seeing, my concerns are warranted.
My sole experience with a fully updated platform so far is with Microsoft’s original Surface Book. It’s based on an Intel “Skylake” Core i7-6700U and has 16GB of LPDDR3 and a 512GB Samsung 950 Pro NVMe drive. The Surface Book is running the 64-bit Windows 10 Pro Fall Creator’s Update.
I basically drove the machine all week at CES, and on Friday morning when I fired it up at home, I found that Microsoft had pushed out two pairs of firmware updates that address the Spectre and Meltdown vulnerabilities.
I’ll admit, I’m still trying to play catch up on just what the hell is going on with Spectre and Meltdown, but this was a great opportunity to run before and after benchmarks on a production machine.
I know from reading Steve Walton’s write-up at TechSpot that the performance of games and most CPU-intensive apps doesn’t change. But Walton found storage read/write performance to be an issue, so this was my first point of investigation on the Surface Book. I started my tests with a not-yet-patched machine.
Before the Spectre and Meltdown fix
IDG
CrystalDiskMark 5.5.0 performance results on a Microsoft Surface Book before being patched for Meltdown and Spectre.
I started by running the default test for CrystalDiskMark 5.5.0. It’s the slightly older version, but the results are still valid for two areas: small 4K reads and writes as well as 4K reads and writes using a queue depth of 32. To the left you’ll find the result before Surface Firmware 91.1926.768.0 and 90.1837.256.0 were installed.
I made three repeated runs with two to five minutes of rest time in between to let the SSD return to normal temperature. SSDs, as you may know, can slow when heated. This is one particular result, but representative of the results I saw.
After the Spectre and Meltdown fix
Once I completed my tests, I was able to reboot the Surface Book and let the firmware patches install.
IDG
CDM 5.5.0 results on the Microsoft Surface Book once its been updated for the Spectre and Meltdown exploits.
As you can see in the chart to the right, the sequential read and write performance doesn’t change much (in fact, CrystalDiskMark no longer uses a low-queue-depth sequential test). But just as TechSpot found, 4K performance ain’t pretty. While 4K read performance was similar, the write performance dropped by 26 percent.
Far worse, though, 4K read and write with high queue depth take a performance hit of 42 percent and 39 percent, respectively. Ouch.
As with the pre-patch state, I ran the test three times with similar results.
A more relatable scenario
Storage performance hits in synthetic read/write tests, of course, can be difficult to relate to. So, to see how the patches manifested in performance that’s easy to relate to, I also ran Principled Technologies WebXPRT 2015 on the Surface Book, using the latest version of Microsoft Edge. WebXPRT 2015 is a browser-based benchmark that measures performance in various scenarios in HTML5 and JavaScript.
IDG
Before Meltdown and Spectre patch in WebXPRT 2015.
Unpatched, I saw an overall score of 450. Once patched, the score dropped, though not substantially, to 433. That’s about 4 percent slower. Intel’s own findings with that same test show about a 10 percent reduction.
IDG
After Meltdown and Spectre patch in WebXPRT 2015.
So is the situation not as dire as the synthetic storage benchmarks make it out to be? Yes and no. We’re still very early in testing the patches, but it’s safe to assume that performance drops will be dependent on what you actually do with your machine.
As TechSpot found, most classic tests (such as pure 3D rendering) and most games won’t see a change. But the greater-than-20-percent storage penalty that both TechSpot and I observed will rear its head occasionally.
In fact, one test that Intel presents is BAPCo’s SYSMark 2014 SE. One of the most advanced benchmarks around, SYSMark uses real-world applications such as Word and Photoshop, and then runs them through tasks that mirror real-world activity.
IDG
Two sets of patches for the Surface Book address the Spectre and Meltdown exploits.
Even better, BAPCo has developed a method for testing where only the response time of an action is measured. In Word, for example, old-time benchmarks would type and perform actions in Word well beyond what even the fastest typist could ever hit.
But SYSMark 2014 SE measures the things that can truly annoy you, like how long it takes to start the application, or have it perform a search and replace, or import photos.
As you can imagine, SYSMark will lean more heavily on storage responsiveness.
Intel’s tests on SYSMark 2014 SE mostly indicate about an 8 percent overall hit, but in the details for the System Responsiveness test (where you’d expect the storage performance to matter more), Intel says its seeing a 21 percent drop. Ouch.
We’ll try to independently verify Intel’s results on our own builds, but everything I’m seeing so far says the performance penalty will probably run the gamut from “no big deal” to “this is really testing my patience.” Again: It’ll come down to what you’re doing, and how you’re doing it.
If we’re talking an extra 500ms to launch an application that takes 1,500ms to launch, no big deal. But if we’re talking 34 seconds to import or copy photos instead of 27 seconds, it’s going to get annoying really fast and that’s what scares me.