ELF executable issues

Question

I'm having some weird issues with ELF executables on linux.

This is my system (uname -a):

Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u2 (2016-01-02) x86_64 GNU/Linux

I have the following program (test.asm), I assemble it using NASM:

; program just exits with code 0 using linux INT 80H

SECTION .data
SECTION .text
GLOBAL _start

_start:
    MOV EAX, 1
    XOR EBX, EBX
    INT 0x80

I create three different executables:

nasm -f elf32 -o test32-i386.o test.asm
ld -m elf_i386 -o test32-i386 test32-i386.o

nasm -f elfx32 -o test32-x86_64.o test.asm
ld -m elf32_x86_64 -o test32-x86_64 test32-x86_64.o

nasm -f elf64 -o test64-x86_64.o test.asm
ld -m elf_x86_64 -o test64-x86_64 test64-x86_64.o

This is the output of the file command:

test32-i386:   ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
test32-x86_64: ELF 32-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
test64-x86_64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped

Makes sense to me. However, running them causes trouble.

• ./test32-i386: no problem, runs fine.
• ./test64-x86_64: same, runs fine.
• ./test32-x86_64 however, gives bash: ./test32-x86_64: cannot execute binary file: Exec format error

Also, Valgrind produces... interesting results.

• valgrind ./test32-i386: OK
• valgrind ./test64-x86_64: raises SIGILL (?!)
• valgrind ./test32-x86_64: gives me ./test32-x86_64: 1: ./test32-x86_64: Syntax error: word unexpected (expecting ")")

So, to summarize:

Question 1: Why does Valgrind raise SIGILL when it runs ./test64-x86_64 even though the program seems to work fine without Valgrind?

Question 2: Why can't I run ./test32-x86_64? The error Valgrind gives for that binary is very obscure...

Answer 1

First of all, I could not reproduce your error on ./test32-x86_64. Though I used the exact same code and command lines to compile it.

I am running on Linux 4.3.3 x86_64 (Debian).

Question 1: Why does Valgrind raise SIGILL when it runs ./test64-x86_64 even though the program seems to work fine without Valgrind?

My version of Valgrind (3.11.0) do not raise a SIGILL on this one, but raise this message (and, then, execute the program as expected):

valgrind: wrong ELF executable class (eg. 32-bit instead of 64-bit)

But, when running text64-x86_64 Valgrind throw out the following message:

vex amd64->IR: unhandled instruction bytes: 0xCD 0x80 0x0 0x0 0x0 0x0 0x0 0x0
vex amd64->IR:   REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0
vex amd64->IR:   VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE
vex amd64->IR:   PFX.66=0 PFX.F2=0 PFX.F3=0

As, stated in the Valgrind FAQ:

3.3. My program dies, printing a message like this along the way:

vex x86->IR: unhandled instruction bytes: 0x66 0xF 0x2E 0x5

One possibility is that your program has a bug and erroneously jumps to a non-code address, in which case you'll get a SIGILL signal. Memcheck may issue a warning just before this happens, but it might not if the jump happens to land in addressable memory.

Another possibility is that Valgrind does not handle the instruction. If you are using an older Valgrind, a newer version might handle the instruction. However, all instruction sets have some obscure, rarely used instructions. Also, on amd64 there are an almost limitless number of combinations of redundant instruction prefixes, many of them undocumented but accepted by CPUs. So Valgrind will still have decoding failures from time to time. If this happens, please file a bug report.

In our precise case, it simply means that the VEX intermediate language is not recognizing the int 0x80 instruction as part of a x86_64 architecture.

Question 2: Why can't I run ./test32-x86_64? The error Valgrind gives for that binary is very obscure...

Unfortunately, I cannot reproduce your error with my Valgrind.

Answer 2

For question 1: there's a bug opened against valgrind that it doesn't support the int80 instruction in x86_64. I was able to reproduce this under my own valgrind (v3.11.0) and from a browse of the source it appears as though it's not supported.

For question 2: the file type is not supported by your ELF loader. In order to provide compatibility of 32bit binaries on linux, it has to make some checks of the binary when it tries to execute it.

When we use readelf on the test32-x86_64, it reveals a header of:

ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x400060
  Start of program headers:          52 (bytes into file)
  Start of section headers:          288 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         1
  Size of section headers:           40 (bytes)
  Number of section headers:         5
  Section header string table index: 2

i.e. class is 32bit, and machine type is x86_64. i.e. it's an x32 ABI binary

The problem is that this requires your kernel to be configured with CONFIG_X86_X32_ABI, otherwise you'll fall foul of the check:

#define compat_elf_check_arch(x)                                        \
         (elf_check_arch_ia32(x) ||                                      \
          (IS_ENABLED(CONFIG_X86_X32_ABI) && (x)->e_machine == EM_X86_64))

which only supports 32bit binaries without the config option. This option is set if you have the kernel options: CONFIG_X86_X32=y and CONFIG_X86_X32_DISABLED unset (this is linux kernel 4.3 source I'm looking at).

So you need your kernel configured with this support to get the code to run - the reason perror wasn't seeing the issue is that his kernel seems to be compiled with the correct options for running x32 code.

valgrind is probably hopeless confused by the binary format - it's not considered particularly common.