Linker's emulation vs OUTPUT_FORMAT

Question

When I specify the output format to be i386, my execute got a SIGSEGV. However, when I use -m elf_i386 option, it worked. Checking man page, these two are different, since OUTPUT_FORMAT is equivalent to -oformat option.

So, what are the differences between the two and which should I use in which cases?

Example code:

File hello.c:

int a = 1;
int b;
void _start() {
  /* exit system call */
  asm("movl $1,%eax;"
      "xorl %ebx,%ebx;"
      "int  $0x80"
    );
}

script.lds: OUTPUT_FORMAT and OUTPUT_ARCH seem to do nothing to help my program running.

/* OUTPUT_FORMAT("elf32-i386"); */
/* OUTPUT_ARCH(i386); */
OUTPUT(hello);
ENTRY(_start);

SECTIONS
{
  .text 0x10000:
  {
    *(.text)
  }
  .data 0x8000000:
  {
    *(.data)
  }
  .bss :
  {
    *(.bss)
  }
}

Commands executes:

gcc -m32 -nostdlib -g -c hello.c -o hello.o
ld -m elf_i386 -T script.lds hello.o

Answer 1

The difference really is that emulation means way more than just OUTPUT_ARCH and OUTPUT_FORMAT. Some of the details are almost obvious, like the difference in default linker scripts that can be seen with --verbose option, some are described in this document, but most of the answers could only be found in sources, like compare emulation script for elf_i386 and emulation script for elf_x86_64. The difference doesn't seem to be that high, but that's not the only difference and what actually bites you in your particular case can't even be seen with a diff between generated (on ld build) ld/eelf_i386.c and ld/eelf_x86_64.c files, because that boils down to the constant that comes from the bfd library and that also depends on emulation.

So, let's drill down a little bit and see what happens. Everywhere down below by script.lds I mean your script with OUTPUT_ARCH and OUTPUT_FORMAT uncommented.

Now, let's take a look at differences in results first:

$ ld -T script.lds hello.o
$ LC_ALL=C objdump -p hello

hello:     file format elf32-i386

Program Header:
    LOAD off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**21
         filesz 0x00010048 memsz 0x00010048 flags r-x
    LOAD off    0x00200000 vaddr 0x08000000 paddr 0x08000000 align 2**21
         filesz 0x00000004 memsz 0x00000008 flags rw-
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4
         filesz 0x00000000 memsz 0x00000000 flags rw-
$ ld -m elf_i386 -T script.lds hello.o
$ LC_ALL=C objdump -p hello

hello:     file format elf32-i386

Program Header:
    LOAD off    0x00001000 vaddr 0x00010000 paddr 0x00010000 align 2**12
         filesz 0x00000048 memsz 0x00000048 flags r-x
    LOAD off    0x00002000 vaddr 0x08000000 paddr 0x08000000 align 2**12
         filesz 0x00000004 memsz 0x00000008 flags rw-
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4
         filesz 0x00000000 memsz 0x00000000 flags rw-

Notice that the "bad" binary has a PT_LOAD segment with virtual address of zero and alignment of 0x00200000. Virtual address 0 doesn't sound right, but let's see why it really fails. Debugging that is a real fun. If one tries to use gdb, he gets this:

(gdb) run
Starting program: /somewhere/hello 
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb) bt
No stack.
(gdb) info registers 
The program has no registers now.

So the program doesn't even start actually running. Let's look at strace then:

$ strace ./hello
execve("./hello", ["./hello"], [/* 108 vars */]) = -1 EPERM (Operation not permitted)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} ---
+++ killed by SIGSEGV +++

We see that execve() returns EPERM. What kind of permission could fail? Well, that's exactly because of that virtual address of zero, the kernel tries to load our ELF, tries to map file for virtual address 0 and fails to do that because around Linux 2.6.23 times there was a security feature introduced that forbids doing that. But that can be configured, so after a simple

$ echo 0 > /proc/sys/vm/mmap_min_addr

the "bad" binary suddenly starts working.

But we're not about making something work here (yay!), we're about the differences in ld behaviour. What also differs between our "bad" and "good" binaries is the alignment of the loadable segments. And if you think about it for a while, you'll see that ld behaviour is actually absolutely correct, when it has an alignment constraint of 0x1000 it uses virtual address of 0x10000 for the segment start that is correct for this alignment, but when it has an alignment constraint of 0x200000, given that we have instructed it to put our .text into address 0x10000 it has no other choice but to use base virtual address of zero!

So where this alignment requirement comes from? Here we return to our emulation stuff, because the default alignment for both elf_i386 and elf_x86_64 is the maximum page size (got from bfd via bfd_emul_get_maxpagesize()), but that page size is different for these architectures.

You actually can build your binary without elf_i386 emulation, but in order to do that you need to specify the maximum page size via parameter, like:

$ ld -T script.lds -z max-page-size=0x1000 hello.o

This resulting binary will not only work without mmap_min_addr tweaks, but it will also be bit-by-bit identical to the one built with proper elf_i386 emulation.

Getting back to the original questions — the difference is huge and subtle in its details. You definitely want to use the right emulation when you build your software. 99.99% of the time your OUTPUT_FORMAT is going to be something very similar to your emulation parameter.

But. Well. There are some cases. Things you normally don't do. But you can do if you're careful and there is need to, like, for example:

$ head -n 1 script.lds
OUTPUT_FORMAT("srec");
$ ld -T script.lds hello.o 
$ file hello
hello: Motorola S-Record; binary data in text format

Exactly the case where your emulation is one thing and OUTPUT_FORMAT is really about output format that you need for some (strange) reason.

But don't try that at home, please, use proper emulations and forget about all this nightmare.