registers to “roll back” to previous values after a context switch.
I know that sounds unbelievable – but I have a reliable reproducer!
$ grep -m1 ‘model name’ /proc/cpuinfo
model name : AMD Ryzen Threadripper PRO 3945WX 12-Cores
The bug occurs when there is a context switch after a ucomiss
instruction. The YMM registers are “rolled back” to some previous state.
It’s not clear to me how or why this is happening, or if it can happen
across a process boundary.
To reproduce:
$ nasm -felf64 -O0 zenymmasm.asm
$ ld -o zenymmasm zenymmasm.o
If you run it it should just print some nuls:
$ ./zenymmasm
$
That is the expected, correct result.
However, if you run the attached program hammer.c (it just runs
sched_yield() in a loop), and then pin this testcase to the same core as
that:
$ taskset -c 1 ./zenymmasm
SECRETSECRET
The previous register values are restored.
I think this should be impossible.
The code does this:
vmovdqu ymm0, [rel secret] <--- put SECRET into ymm0
mov rax, SYS_sched_yield
syscall
vpxor ymm0, ymm0, ymm0 <--- Here the value of ymm0 should be lost
It’s not related to to VPXOR, you can use VZEROALL or whatever else.
ucomiss xmm0, dword [rel space]
mov rax, SYS_sched_yield
syscall
It’s the UCOMISS r128,m32 instruction that triggers the bug, the value
of the m32 must be < 0x80000. As far as I know, this should only ever
change condition flags, but if we dump the value of ymm0:
mov rax, SYS_write
mov rdi, 1
lea rsi, [rel regstate]
mov rdx, 32
syscall
It will have reverted back to the pre-vpxor SECRET value !?!?
In the original C program, we were seeing register values randomly
restored from significantly earlier in the program execution (like, from
ld.so), sometimes values we don’t recognize and can’t explain.
We’ve reproduced on multiple Zen2 machines.
Obviously, not great if your registers are randomly time travelling 🙂
Thanks, Tavis.
—
_o) $ lynx lock.cmpxchg8b.com
/\ _o) _o) $ finger taviso@sdf.org
__V _( ) _( ) @taviso
#define _GNU_SOURCE
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
int main(int argc, char **argv)
{
cpu_set_t set;
CPU_ZERO(&set);
CPU_SET(1, &set);
if (sched_setaffinity(0, sizeof(set), &set) != 0) {
err(EXIT_FAILURE, “failed to set cpu affinity”);
}
while (true) {
asm volatile (“syscall” :: “a”(SYS_sched_yield) : “rcx”, “r11”);
}
return 0;
}
BITS 64
global _start
%define SYS_sched_yield 0x18
%define SYS_write 0x01
%define SYS_exit 0x3c
section .data
align 32
secret: times 4 dq ‘SECRET’
align 32
regstate: dq 0,0,0,0
align 32
space: dd 1
section .text
_start:
vmovdqu ymm0, [rel secret]
mov rax, SYS_sched_yield
syscall
vpxor ymm0, ymm0, ymm0
mov rax, SYS_sched_yield
syscall
ucomiss xmm0, dword [rel space]
mov rax, SYS_sched_yield
syscall
vmovdqu [rel regstate], ymm0
mov rax, SYS_write
mov rdi, 1
lea rsi, [rel regstate]
mov rdx, 32
syscall
mov rax, SYS_exit
mov rdi, 0
syscall
int3
Leave A Comment