gdb reverse debugging avx2

Question

So I have a new fancy cpu that supports avx2 instruction set. This is great, but breaks gdb reverse debugging. When compiling with no optimisations code still uses shared libraries, eg calls memset() which then goes and invokes an avx2 optimised version of memset. This is great but avx2 is not supported by gdb record.

process record does not support instruction 0xc5 at address 0x7ffff690dd80.

0xc5 is the vex prefix here.

reverse debugging works great with a cpu that does not support avx2. How do I get libc etc to not use the avx2 optimised versions of library calls so I can use gdb record, step backwards etc?

I've tried

LD_BIND_NOW=1
LD_HWCAP_MASK=0
compiling with -static

And short of debugging on an old machine, I'm out of ideas.

Answer 1

For your glibc 2.23 as shipped on ubuntu 16.04 amd64 there is adaptation of my binary kludge (1 bit patch) made exactly for the same reason. Package libc6 (2.23-0ubuntu7) was downloaded from https://packages.ubuntu.com/xenial/amd64/libc6 and file ld-2.23.so was edited (keep copy of the original, or save patched into different path and change INTERP section of your own binary to use different path):

 83 3D 5B C9 20 00 06   cmpl $0x6, smth...
 7E 21                  jle  some_forward_label
 B8 07 00 00 00         mov $0x7, %eax
 31 C9                  xor %ecx,%ecx
 0F A2                  cpuid

There is code of the get_common_indeces: if (cpu_features->max_cpuid >= 7) __cpuid_count (7, 0, ... called from __get_cpu_features. EAX=7 leaf of cpuid has all info needed to detect AVX2 support and enable it, so I just skipped fragment with cpuid eax=0x7,ecx=0 and saving its results into some parts of memory by changing 0x7e 0x21 into 0x7f 0x21.

So, the binary patch is like replacing 83 3D xx xx xx xx 06 7E xx B8 07 00 00 00 31 C9 0F A2 (where xx may be any byte) into 83 3D xx xx xx xx 06 7F xx B8 07 00 00 00 31 C9 0F A2. You may do this with any Hex Editor or by some binary diff. In 2.23-0ubuntu7 this code is at 0x0193B0 - 0x0193B9 is 7e to be changed into 7f.

The patch it dumb and don't use patched file globally if your root fs may be started with CPU without eax=7 cpuid leaf support (pre Intel Core CPU) or in virtual machine emulating such pre Intel Core CPU ("Pentium D 8xx/9xx", Pentium 4, Pentium M - will fail).

You may put patched file at path which name is equal in length or shorter to the original path of /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (which symlinks into /lib/x86_64-linux-gnu/ld-2.23.so file). For example as /lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2. Then use same hexeditor to replace the string "/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2" of your program executable (ELF) into "/lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2", or use patchelf tool from the patchelf package:

cp /lib/x86_64-linux-gnu/ld-2.23.so /lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2

bless  /lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2
# or any other hex editor

patchelf --set-interpreter /lib_x86_64-linux-gnu_ld-linux-no-AVX2.so.2 ./my_program