Chip-level vulnerability issues restricted to high-end workloads

The Spectre/Meltdown disclosures in January last year shook long-held assumptions about processor hardware security.

But even though the problems forced fundamental redesigns, it has not resulted in the industry taking the performance hit that some initially expected, The Daily Swig can reveal.

The Meltdown and Spectre vulnerabilities both stemmed from security shortcomings in computer processor design.

In order to make computers work faster, modern hardware allows instructions to be run out of sequence or in parallel across the different processing units within the CPU.

This set up the possibility of a CPU ‘race condition’, where an instruction might be executed before their privileges are checked.

Controls normally restrict user applications from accessing protected kernel memory, but the Meltdown vulnerability offered a means to circumvent these protections, opening the door for malicious code to steal passwords and other sensitive information on vulnerable systems.

The Spectre vulnerability created a means to tear down the isolation between different applications, to similar malign effect.

Meltdown was largely restricted to Intel processors. Spectre – which is harder to exploit but potentially even more damaging – impacted a much wider range of processor makers, including AMD and ARM.

Operating system developers and cloud service providers scrambled to develop and roll out fixes to defend against Meltdown/Spectre before the vulnerabilities became public knowledge in early January last year.

Most unusually, patching the vulnerabilities incurred a performance hit. This was especially inconvenient since the risks to cloud-based environments, in particular, were too severe to disregard.

Performance hits of between 5-30% (depending on workload) were seen in early testing of software fixes, but the effect has been nowhere as severe in practice, according to experts including the original Meltdown authors and others quizzed by The Daily Swig.

Professor Alan Woodward, a computer scientist at the University of Surrey, UK, commented: “The Spectre and Meltdown problems have been mitigated reasonably well by the chip makers and the operating system vendors. To be honest, I’m not sure the great slow down that was forecast has happened. Certainly, most users wouldn’t have noticed much if any difference.”

Daniel Gruss, a security researcher at Graz University of Technology and member of one of the three teams that uncovered the Meltdown vulnerability, agreed with this assessment. “We said, from the start, that the performance hit won't be significant and [would] remain unnoticeable for most users and use cases,” he told The Daily Swig.

Variations on a theme

Gruss added that research into chip-level security has become a fertile field since Meltdown broke in January 2018. “There are tons of follow up works discovering new variations of these attacks or putting these attacks into context,” he explained.

Professor Woodward added: “The only real developments are that further exploits and vulnerabilities in the same family have emerged – that’s where the research went.

“Some of the exploits (including the originals, to be honest) are fairly low down on the list of problems that system operators/home users need to worry about. But having said that, it is a risk and continues to be addressed by Intel, Microsoft, et al, and those mitigations haven’t fully plugged the gaps – a few cracks are still there if you wanted to try to exploit it.

“However, the return on your efforts in trying to exploit these remaining chinks is likely to be fairly low – [there are] easier pickings elsewhere,” he concluded.

Gruss backed the assessment that Spectre was somewhat esoteric, but disagreed when it came to Meltdown.

“For Spectre, I fully agree,” Gruss told The Daily Swig. “The case for Meltdown is different though. Meltdown was so trivial to exploit – hadn’t it been patched, we would have seen real-world attacks.”

Werner Haas, a security researcher at Cyberus Technology and member of a separate team also credited with uncovering Meltdown, said that “office-/home-users likely do not experience any slow down” as a result of measures put in place to address the vulnerability.

“Out of curiosity we did a cursory check on our own systems (including compute[r] servers) and noted no significant difference. Some of our customers, however, especially those paranoid about security, they suffer a lot because they have to be careful with hyperthreading.”

Benchmark tests

Jani Joki, engineering director of UL Benchmarks, a specialists computer performance assessment firm, said that gauging the effects of the Spectre and Meltdown fixes one year on was difficult – not least because so many other changes have taken place in the market in the last 12 months.

“We did run a fair amount of numbers to evaluate the impact at the time, but unfortunately not in a server context – all our focus was on PC laptops/desktops,” Joki told The Daily Swig.

“Some of that data might be interested in a server evaluation, primarily in our Servermark VDI product. However, given that there have been a number of other flaws fixed, new hardware generations, and for desktops an upcoming Windows update, the comparison data would have to be recreated.”

The Meltdown fix for the Linux kernel introduced a technology called Kernel Page Table Isolation (KPTI).

Software defined (virtual) storage firm Virtuozzo published before and after Meltdown fix performance data on its own technology. Container performance impact from KPTI ranged from no measurable impact to as high as 10-15% (depending on system configuration), the firm found.

Virtuozzo 7 hypervisor, pre and post Spectre patching metrics


Such real-world performance benchmarking on the implications of the Meltdown and Spectre fixes is very thin on the ground, so it’s hard to draw any firm conclusion other than to state that the performance of consumer setups were not much affected by the CPU bug fixes.

Haas offered the suggestion that, on the performance front, cloud providers were probably hit the hardest, and they also have the scale to solidly quantify low-impact effects.

Cloud platforms – such as Amazon Web Services and Microsoft Azure – have not (publicly, at least) disclosed their findings, but independent experts have seen an effect on cloud-based systems that they think are attributable to the Meltdown and Spectre security fixes.

Daniel Ayers, a security consultant and computer forensics expert, told The Daily Swig: “On Intel E5 & Gold I was seeing a huge impact with KVM-QEMU on Linux. Closer to 30% than 5%.

“Context switches for I/O (esp. 10G eth) were especially an issue. I have seen it break cloud providers so badly they had to turn mitigations off to have a functional system.”

Hoopla

Haas criticised the “hoopla” in news coverage and elsewhere that ensued in the aftermath of the Meltdown revelations. “I have seen tons of citations of the Meltdown/Spectre paper, some with questionable connections. It appears as if it has become ‘in’ to quote Meltdown/Spectre in the academic world,” he said.

The publicity has, however, also had some beneficial effects of raising awareness about processor-level security among users of high-performance computing.

“In industrial context, we are approached because people are curious about the inner workings of CPUs,” Haas explained. “It used to be treated as black box component and now we observe genuine interest in basic understanding of the hardware architecture.

“We have also seen interest in a more principled approach instead of simply piling features onto an existing design and we are aware of people worrying about the security properties in the ARM universe as it has more diversity on platform level, compared to IA [Intel Architecture].”

Higher up the stack

Haas said that in a “world of sloppy authentication and open ports there are way easier attack vectors” in most scenarios, while noting that “things are different in the high security domain”.

“People realized that they have to look beyond the ISA [Intel’s Instruction Set Architecture] specification,” he explained. “SGX [Intel’s Software Guard Extensions], in particular, was hit pretty hard, and side-channel attacks are very effective there. But this is not the kind of mass malware you will see popping up in your anti-virus scanner.”

A year on there aren’t any documented cases of the Meltdown/Spectre vulnerabilities that have been applied, even in targeted nation-state attacks.

Further exploits and vulnerabilities in the same family of Meltdown and Spectre have emerged in the year since the landmark discoveries, but Haas described these as the “low-hanging fruit”, saying that deeper, less derivative work is underway but will take some time to arrive.

“I believe there is more research going on that is simply not visible yet, simply because hardware-level developments require more time,” Haas concluded.

Responding to The Daily Swig’s request for comment, Intel referred us to a recent blog post outlining its ‘security-first’ approach to hardware development.

Meltdown and Spectre: From discovery to disclosure

  • August 2016 – Moritz Lipp of Graz University of Technology and other researchers presented ‘ARMageddon: Cache Attacks on Mobile Devices’ (PDF) at the 25th USENIX security symposium. Although focused on ARM chipsets, the research opened up the field of research, into which Meltdown and Spectre would land around a year later.
  • Early 2017 – Security researchers from three teams independently discover and report what became known as the Meltdown vulnerability. A related CPU security vulnerability, which came to be called Spectre, was discovered by some of the same researchers and others at around the same time.
  • Late July 2017 – The bugs get confirmed. Affected hardware and operating system developers briefed on the issue. OS developers secretly start working on mitigations for the vulnerabilities.
  • November 2017 – Security researchers Alex Ionescu publicly said changes in the latest build of Windows 10 would impact speed, without explaining why the changes were nonetheless needed. He mentions that similar changes in Linux are in the pipeline.
  • January 3, 2018 – News of the vulnerabilities breaks, forcing Intel to go public, several days earlier than planned.
  • January 9, 2018 – Early versions of the Spectre fix cause issues in Microsoft server installations. These teething problems are gradually resolved, leaving only a dim memory the inconvenience caused in the first place.
  • March 15, 2018 – Intel said it planned to redesign its CPU processors (performance losses to be determined) to help defend against Meltdown and Spectre vulnerabilities, ahead of the release of processors in late 2018.
  • May 2018 – Eight further Spectre-type flaws (dubbed Spectre-NG) are disclosed.

  • October 8, 2018 – Intel confirmed that it had added hardware and firmware mitigations against Meltdown and Spectre vulnerabilities to its processors.

Research team roster

Meltdown earned its name because the vulnerability was seen as ‘melting’ the security boundaries normally enforced by silicon chip hardware. Spectre relates to a technology called ‘speculative branch execution’.

Meltdown was independently discovered and reported by three teams: Jann Horn from Google’s Project Zero; Werner Haas and Thomas Prescher from Cyberus Technology; and Daniel Gruss, Moritz Lipp, Stefan Mangard, and Michael Schwarz from Graz University of Technology, Austria.

Spectre was independently discovered and reported by two teams: Jann Horn and Paul Kocher from Cryptography Research, in collaboration with Daniel Genkin from University of Pennsylvania and University of Maryland; and Mike Hamburg from Rambus, Moritz Lipp, and Yuval Yarom from University of Adelaide and Data61.


RELATED Remote control? New Spectre-based attack carried out via network