[Beowulf] New Spectre attacks - no software mitigation - what impact for HPC?

[Chris Samuel]

Hi all,

This is a few days old now, but it passed me by until now.

https://www.tomshardware.com/news/intel-arm-new-spectre-flaws,37436.html

The researchers noted in their paper that currently no effective static
analysis or compiler instrumentation can even detect or mitigate Spectre
1.1.

and

What the researchers are actually implying is first that software
mitigations largely depend on app developers to implement them, which means
that most applications won’t be protected, if history is any guide; second,
hardware changes will be necessary for true long-term fixes that can stop
Spectre flaws from appearing.

I will be interesting to see what happens around this one, as they say that if
we don't get hardware fixes we could face decades of different variations on
this as software folks play whack-a-mole.

So the two HPC related issues that come to mind will be:

1) It'll be interesting to see what performance impacts hardware fixes for this
class of attacks will be, and whether we see vendors decide that the only way
to really avoid them is to drop speculative execution. Perhaps if that
penalty is large then would vendors look to have separate processor lines, one
set with speculative execution for performance (but without protection) and
one for security instead?

2) Will people start to look at delaying purchasing decisions until it becomes
clearer how the chip vendors are going to deal with this?

This might be a more pressing concern for the cloud crowd given the higher
immediate exposure, but even in HPC we can't avoid the need to address this in
some way (even if it's just "we did a risk assessment and we judge it to be a
low risk").

Currently these new vulnerabilities are demonstrated on Intel & ARM, it will
be interesting to see if AMD is also vulnerable (I would guess so).

cheers!
Chris

[Chris Samuel]

Post by Chris Samuel
Currently these new vulnerabilities are demonstrated on Intel & ARM, it will
be interesting to see if AMD is also vulnerable (I would guess so).

Interestingly RISC-V claims immunity, and that looks like it'll be one of the
two CPU architectures blessed by the Europeans in their Exascale project
(along with ARM).

https://riscv.org/2018/01/more-secure-world-risc-v-isa/

All the best,
Chris

[John Hearns via Beowulf]

I guess I am not going to explain myself very clearly here. Maybe I wont
make a coherent point.
I think I read on The Next Platform at the time a comment along the lines
of - "as CPU Mhz speeds cannot continue to rise,
smart engineers who design CPUs have had to come up with mechanisms to
increase performance continually, in the face of software developers who
will not modernise their code."
Indeed of course we all run a huge base of legacy codes, and these will nto
be retired any time soon.
So speculative execution is one mechanism to 'save the bacon' of users
wanting more and more performance.

We in Beowulfery have taken advantahe of general purpose CPUs. Indeed,
with CPUs themselves having wide vector units and several units per package
(AMD) they actual CPU package looks like a supercomputer of old.
I digress. The big companies produce CPUS for the datacentre market, which
is of course their biggest market. I get the impression that HPC is no
small market either, and CPU varieties are engineered specifically for the
HPC market.
To be clear, my argument is NOT that HPC is some sort of leftover from the
data centre market - indeed it is a prime and growing market.

However, in the past there were specific architectures and instruction sets
for supercomputing. What I am going to throw out to the floor is:

* The IT industry goes round in cycles. Is the time right for HPC specific
processors again?

* Yes, you would need a Linux kernel and distribution ported to that CPU
instruction set

* Regarding compiler technology, how important is speculative execution for
HPC style codes?

* For the point above, can anyone point to studies where retired
instructions versus unused have been counted for HPC codes?
https://software.intel.com/en-us/vtune-amplifier-help-instructions-retired-event

* What would HPC specific processors look like? Would they have speculative
execution? What else would be missing - or added?

Post by Chris Samuel

Post by Chris Samuel
Currently these new vulnerabilities are demonstrated on Intel & ARM, it

will

Post by Chris Samuel
be interesting to see if AMD is also vulnerable (I would guess so).

Interestingly RISC-V claims immunity, and that looks like it'll be one of the
two CPU architectures blessed by the Europeans in their Exascale project
(along with ARM).
https://riscv.org/2018/01/more-secure-world-risc-v-isa/
All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[John Hearns via Beowulf]

In a partial answer to my own question it looks like Likwid can count
Issued versus Retired instructions
https://github.com/RRZE-HPC/likwid/wiki/TutorialStart

Post by John Hearns via Beowulf
I guess I am not going to explain myself very clearly here. Maybe I wont
make a coherent point.
I think I read on The Next Platform at the time a comment along the lines
of - "as CPU Mhz speeds cannot continue to rise,
smart engineers who design CPUs have had to come up with mechanisms to
increase performance continually, in the face of software developers who
will not modernise their code."
Indeed of course we all run a huge base of legacy codes, and these will
nto be retired any time soon.
So speculative execution is one mechanism to 'save the bacon' of users
wanting more and more performance.
We in Beowulfery have taken advantahe of general purpose CPUs. Indeed,
with CPUs themselves having wide vector units and several units per package
(AMD) they actual CPU package looks like a supercomputer of old.
I digress. The big companies produce CPUS for the datacentre market, which
is of course their biggest market. I get the impression that HPC is no
small market either, and CPU varieties are engineered specifically for the
HPC market.
To be clear, my argument is NOT that HPC is some sort of leftover from the
data centre market - indeed it is a prime and growing market.
However, in the past there were specific architectures and instruction
* The IT industry goes round in cycles. Is the time right for HPC specific
processors again?
* Yes, you would need a Linux kernel and distribution ported to that CPU
instruction set
* Regarding compiler technology, how important is speculative execution
for HPC style codes?
* For the point above, can anyone point to studies where retired
instructions versus unused have been counted for HPC codes?
https://software.intel.com/en-us/vtune-amplifier-help-
instructions-retired-event
* What would HPC specific processors look like? Would they have
speculative execution? What else would be missing - or added?

Post by Chris Samuel

Post by Chris Samuel
Currently these new vulnerabilities are demonstrated on Intel & ARM, it

will

Post by Chris Samuel
be interesting to see if AMD is also vulnerable (I would guess so).

Interestingly RISC-V claims immunity, and that looks like it'll be one of the
two CPU architectures blessed by the Europeans in their Exascale project
(along with ARM).
https://riscv.org/2018/01/more-secure-world-risc-v-isa/
All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[Scott Atchley]

Hi Chris,

They say that no announced silicon is vulnerable. Your link makes it clear
that no ISA is immune if the implementation performs speculative execution.
I think your point about two lines of production may make sense. Vendors
will have to assess vulnerabilities and the performance trade-off.

Personally, I do not see a large HPC system being built out of
non-speculative hardware. You would need much more hardware to reach a
level of performance and the additional power could lead to a lower
performance per Watt (i.e., exceed the facility's power budget).

Scott

Post by Chris Samuel

Post by Chris Samuel
Currently these new vulnerabilities are demonstrated on Intel & ARM, it

will

Post by Chris Samuel
be interesting to see if AMD is also vulnerable (I would guess so).

Interestingly RISC-V claims immunity, and that looks like it'll be one of the
two CPU architectures blessed by the Europeans in their Exascale project
(along with ARM).
https://riscv.org/2018/01/more-secure-world-risc-v-isa/
All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[John Hearns via Beowulf]

This article is well worth a read, on European Exascale projects

https://www.theregister.co.uk/2018/07/17/europes_exascale_supercomputer_chips/

The automotive market seems to have got mixed in there also!
The main thrust dual ARM based and RISC-V

Also I like the plexiglass air shroud pictured at Barcelona. I saw
something similar at the HPE centre in Grenoble.
Damn good idea.

Post by Scott Atchley
Hi Chris,
They say that no announced silicon is vulnerable. Your link makes it clear
that no ISA is immune if the implementation performs speculative execution.
I think your point about two lines of production may make sense. Vendors
will have to assess vulnerabilities and the performance trade-off.
Personally, I do not see a large HPC system being built out of
non-speculative hardware. You would need much more hardware to reach a
level of performance and the additional power could lead to a lower
performance per Watt (i.e., exceed the facility's power budget).
Scott

Post by Chris Samuel

Post by Chris Samuel
Currently these new vulnerabilities are demonstrated on Intel & ARM, it

will

Post by Chris Samuel
be interesting to see if AMD is also vulnerable (I would guess so).

Interestingly RISC-V claims immunity, and that looks like it'll be one of the
two CPU architectures blessed by the Europeans in their Exascale project
(along with ARM).
https://riscv.org/2018/01/more-secure-world-risc-v-isa/
All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[Scott Atchley]

I saw that article as well. It seems like they are targeting using RISC-V
to build an accelerator. One could argue that you do not need speculation
within a GPU-like accelerator, but you have to get your performance from
very wide execution units with lots of memory requests in flight as a GPU
does today.

On Tue, Jul 17, 2018 at 8:19 AM, John Hearns via Beowulf <

Post by John Hearns via Beowulf
This article is well worth a read, on European Exascale projects
https://www.theregister.co.uk/2018/07/17/europes_exascale_
supercomputer_chips/
The automotive market seems to have got mixed in there also!
The main thrust dual ARM based and RISC-V
Also I like the plexiglass air shroud pictured at Barcelona. I saw
something similar at the HPE centre in Grenoble.
Damn good idea.

Post by Scott Atchley
Hi Chris,
They say that no announced silicon is vulnerable. Your link makes it
clear that no ISA is immune if the implementation performs speculative
execution. I think your point about two lines of production may make sense.
Vendors will have to assess vulnerabilities and the performance trade-off.
Personally, I do not see a large HPC system being built out of
non-speculative hardware. You would need much more hardware to reach a
level of performance and the additional power could lead to a lower
performance per Watt (i.e., exceed the facility's power budget).
Scott

Post by Chris Samuel

Post by Chris Samuel
Currently these new vulnerabilities are demonstrated on Intel & ARM,

it will

Post by Chris Samuel
be interesting to see if AMD is also vulnerable (I would guess so).

Interestingly RISC-V claims immunity, and that looks like it'll be one of the
two CPU architectures blessed by the Europeans in their Exascale project
(along with ARM).
https://riscv.org/2018/01/more-secure-world-risc-v-isa/
All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[Lux, Jim (337K)]

Perhaps it’s more about “system management” for HPC – these sort of vulnerabilities only occur when you have one process able to see what’s going on in another process. From a “security” standpoint, the answer is simple – don’t share the same entity between processes owned by different users. (presumably a given user doesn’t care about “leaking” from one of his/her processes to another).

Sure, in the abstract, it would be nice to design leakproof processors – hey, it would be nice to make processors and hardware that don’t radiate EMI, which can also provide a leakage path.

I’m not sure that for the *vast majority* of HPC applications this is a problem – sure, in “the cloud” or in a heavily shared resource with very little control over who is sharing (i.e. Azure or AWS kind of server farming) you’ve got a potential problem. And, on the desktop where a seemingly innocuous program can theoretically get data from another. Neither of the latter, though, is anything like what one would consider “secure computing”.

If the data YOU are processing is sufficiently valuable that it is worth it to try and exploit via a Spectre type attack, maybe it’s worth it for you to have a dedicated computing resource? Are HPC jobs sufficiently fine grained and disperseable that you would have multiple user’s processes running on a single CPU in a 1000 node cluster? Or would you say “User 1, you get nodes 1-500, User 2 you get nodes 501-1000”? I find it hard to believe that we *must* distribute processes more finely grained than this (user 1 gets cores 1,2,3,4 on node1, cores 1,2 on node2, user 2 gets cores 3,4, on node 2, cores 1,2,3,4 on node 3)

And really, these Spectre type attacks are sort of a theoretical vulnerability – there’s a long way from having iron ore and reading about its processing to having machine tools and swords in your hands. Sure, there are adversaries with sufficient resources to figure it out (maybe it’s cheaper to steal secrets than to do it yourself, if the original secrets cost billions), and it’s of great theoretical interest to figure out how to make processors that are immune to this. But really, is this a significant threat? Or, even more “conspiracy theory”, you create this potential threat which is *very* expensive to counter, so you throw all your resources at it, and starve the mitigation of the other threats.

For all those millions of dollars you’d invest in “industrializing” an attack like Spectre, you could go out and bribe employees and probably achieve the same end result.

James Lux
Task Manager, DARPA High Frequency Research (DHFR) Space Testbed
Jet Propulsion Laboratory (Mail Stop 161-213)
4800 Oak Grove Drive
Pasadena CA 91109
[phone] (office)
[phone] (cell)

From: Beowulf [mailto:beowulf-***@beowulf.org] On Behalf Of Scott Atchley
Sent: Tuesday, July 17, 2018 6:38 AM
To: John Hearns <***@googlemail.com>
Cc: Beowulf Mailing List <***@beowulf.org>
Subject: Re: [Beowulf] New Spectre attacks - no software mitigation - what impact for HPC?

I saw that article as well. It seems like they are targeting using RISC-V to build an accelerator. One could argue that you do not need speculation within a GPU-like accelerator, but you have to get your performance from very wide execution units with lots of memory requests in flight as a GPU does today.

On Tue, Jul 17, 2018 at 8:19 AM, John Hearns via Beowulf <***@beowulf.org<mailto:***@beowulf.org>> wrote:
This article is well worth a read, on European Exascale projects

https://www.theregister.co.uk/2018/07/17/europes_exascale_supercomputer_chips/

The automotive market seems to have got mixed in there also!
The main thrust dual ARM based and RISC-V

Also I like the plexiglass air shroud pictured at Barcelona. I saw something similar at the HPE centre in Grenoble.
Damn good idea.







On 17 July 2018 at 13:07, Scott Atchley <***@gmail.com<mailto:***@gmail.com>> wrote:
Hi Chris,

They say that no announced silicon is vulnerable. Your link makes it clear that no ISA is immune if the implementation performs speculative execution. I think your point about two lines of production may make sense. Vendors will have to assess vulnerabilities and the performance trade-off.

Personally, I do not see a large HPC system being built out of non-speculative hardware. You would need much more hardware to reach a level of performance and the additional power could lead to a lower performance per Watt (i.e., exceed the facility's power budget).

Scott

Post by Chris Samuel
Currently these new vulnerabilities are demonstrated on Intel & ARM, it will
be interesting to see if AMD is also vulnerable (I would guess so).

Interestingly RISC-V claims immunity, and that looks like it'll be one of the
two CPU architectures blessed by the Europeans in their Exascale project
(along with ARM).

https://riscv.org/2018/01/more-secure-world-risc-v-isa/

All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC

_______________________________________________
Beowulf mailing list, ***@beowulf.org<mailto:***@beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


_______________________________________________
Beowulf mailing list, ***@beowulf.org<mailto:***@beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


_______________________________________________
Beowulf mailing list, ***@beowulf.org<mailto:***@beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

[Jonathan Engwall]

Docker claims to have patches which have no degradation in performance:
https://success.docker.com/article/how-does-spectre-meltdown-affect-my-docker-installs
Here is an interesting video about making a simple container and what that
container might do:

I am not saying this solves any issue. It is only a current software
direction. It could be significant because there is a low level exchange of
information to these vulnerabilities, in the video (she does not use docker
by the way) she basically tells her OS to lie about the contents of her
directory.
It is a direction of interesting development. You could for example isolate
a job within a container which carries all sorts of fakery.
Jonathan

Post by Chris Samuel
Hi all,
This is a few days old now, but it passed me by until now.
https://www.tomshardware.com/news/intel-arm-new-spectre-flaws,37436.html

The researchers noted in their paper that currently no effective static
analysis or compiler instrumentation can even detect or mitigate Spectre
1.1.

and

What the researchers are actually implying is first that software
mitigations largely depend on app developers to implement them, which

means

that most applications won’t be protected, if history is any guide;

second,

hardware changes will be necessary for true long-term fixes that can stop
Spectre flaws from appearing.

I will be interesting to see what happens around this one, as they say that if
we don't get hardware fixes we could face decades of different variations on
this as software folks play whack-a-mole.
1) It'll be interesting to see what performance impacts hardware fixes for this
class of attacks will be, and whether we see vendors decide that the only way
to really avoid them is to drop speculative execution. Perhaps if that
penalty is large then would vendors look to have separate processor lines, one
set with speculative execution for performance (but without protection) and
one for security instead?
2) Will people start to look at delaying purchasing decisions until it becomes
clearer how the chip vendors are going to deal with this?
This might be a more pressing concern for the cloud crowd given the higher
immediate exposure, but even in HPC we can't avoid the need to address this in
some way (even if it's just "we did a risk assessment and we judge it to be a
low risk").
Currently these new vulnerabilities are demonstrated on Intel & ARM, it will
be interesting to see if AMD is also vulnerable (I would guess so).
cheers!
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[Douglas Eadline]

I saw that as well. I'm always a bit skeptical about
some of these theoretical attacks. IMO there should
be a "degree of difficultly" (of sorts) assigned to
these hardware issues. Then you can decide on a
risk strategy.

Multicore really introduced a lot of issues. For those
that can remember, when a process owned the whole
(single) processor things seemed bit simpler.

In any case, I believe XCD summs up the issue quite nicely

https://xkcd.com/538/

--
Doug

Post by Chris Samuel
Hi all,
This is a few days old now, but it passed me by until now.
https://www.tomshardware.com/news/intel-arm-new-spectre-flaws,37436.html

The researchers noted in their paper that currently no effective static
analysis or compiler instrumentation can even detect or mitigate Spectre
1.1.

and

What the researchers are actually implying is first that software
mitigations largely depend on app developers to implement them, which means
that most applications won’t be protected, if history is any guide;
second,
hardware changes will be necessary for true long-term fixes that can stop
Spectre flaws from appearing.

I will be interesting to see what happens around this one, as they say that if
we don't get hardware fixes we could face decades of different variations on
this as software folks play whack-a-mole.
1) It'll be interesting to see what performance impacts hardware fixes for this
class of attacks will be, and whether we see vendors decide that the only way
to really avoid them is to drop speculative execution. Perhaps if that
penalty is large then would vendors look to have separate processor lines, one
set with speculative execution for performance (but without protection) and
one for security instead?
2) Will people start to look at delaying purchasing decisions until it becomes
clearer how the chip vendors are going to deal with this?
This might be a more pressing concern for the cloud crowd given the higher
immediate exposure, but even in HPC we can't avoid the need to address this in
some way (even if it's just "we did a risk assessment and we judge it to be a
low risk").
Currently these new vulnerabilities are demonstrated on Intel & ARM, it will
be interesting to see if AMD is also vulnerable (I would guess so).
cheers!
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
MailScanner: Clean