Linker error on relocating a program above 2GB in x86_64 linux?

Question

I have a user program which normally compiles to have an entry point at 0x400460 which I have to relocate to have an entry point starting at within 2GB of the shared libraries loaded in Linux. e.g

linux-vdso.so.1 =>  (0x00007fff109cd000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcd195e6000)
lib64/ld-linux-x86-64.so.2 (0x00007fcd199af000)

I am using gcc command line argument -Wl,-Ttext=0x80000000 to specify the start address for the .text segemnt.

The issue is that when I am giving an address above 2GB in this argument I am getting a linker error which is:

gcc test.c -ggdb -Wl,-Ttext=0x80000000 -o test1
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 1 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 4 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 5 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 6 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 7 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 8 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 9 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 10 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 11 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 12 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 13 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 14 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 15 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 16 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 17 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 18 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 19 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 20 has invalid symbol index 20
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): relocation 0 has invalid symbol index 2
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x12): relocation truncated to fit: R_X86_64_32S against symbol `__libc_csu_fini'   defined in .text section in /usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS)
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x19): relocation truncated to fit: R_X86_64_32S against symbol `__libc_csu_init' defined in .text section in /usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS)
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): relocation truncated to fit: R_X86_64_32S against symbol `main' defined in    .text section in /tmp/ccFshK69.o
/var/services/homes/adabral/elider/gc/a1/lib/gcc/x86_64-unknown-linux-  gnu/4.8.2/crtbegin.o: In function `deregister_tm_clones':
crtstuff.c:(.text+0x8): relocation truncated to fit: R_X86_64_32S against  `.tm_clone_table'
 /var/services/homes/adabral/elider/gc/a1/lib/gcc/x86_64-unknown-linux- gnu/4.8.2/crtbegin.o: In function `register_tm_clones':
 crtstuff.c:(.text+0x38): relocation truncated to fit: R_X86_64_32S against  `.tm_clone_table'
 collect2: error: ld returned 1 exit status

: The reason for this as far as I can make out after going through several SO questions and forums can be that some of the sections are still mapped to low 2GB address space.

This is the output of the readelf -a for a binary compiled with text segment just below 2GB (at 0x79990000).

Dynamic section at offset 0x190310 contains 24 entries:
Tag        Type                         Name/Value
0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
0x000000000000000c (INIT)               0x4003f0
0x000000000000000d (FINI)               0x79990204
0x0000000000000019 (INIT_ARRAY)         0x79b902f8
0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
0x000000000000001a (FINI_ARRAY)         0x79b90300
0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
0x0000000000000004 (HASH)               0x400278
0x0000000000000005 (STRTAB)             0x400318
0x0000000000000006 (SYMTAB)             0x4002a0
0x000000000000000a (STRSZ)              72 (bytes)
0x000000000000000b (SYMENT)             24 (bytes)
0x0000000000000015 (DEBUG)              0x0
0x0000000000000003 (PLTGOT)             0x79b904e8
0x0000000000000002 (PLTRELSZ)           72 (bytes)
0x0000000000000014 (PLTREL)             RELA
0x0000000000000017 (JMPREL)             0x4003a8
0x0000000000000007 (RELA)               0x400390
0x0000000000000008 (RELASZ)             24 (bytes)
0x0000000000000009 (RELAENT)            24 (bytes)
0x000000006ffffffe (VERNEED)            0x400370
0x000000006fffffff (VERNEEDNUM)         1
0x000000006ffffff0 (VERSYM)             0x400360
0x0000000000000000 (NULL)               0x0

You can see that the INIT and some other sections still starts at the low 2GB address space. So the dynamic linker can not offset the relocation address at run time since the relocation type is R_X86_64_32.

• So I tried compiling my code with gcc -mcmodel=large flag but I am still getting the same linker error. Using the large model should have rectified this error but it is not.

• I am stuck at this point, any help is highly appreciated.

I am working on x86_64 ubuntu. gcc version 4.8.2

Answer 1

You should first understand that the ABI for x86_64 has several different "models": small, kernel, medium, and large. These are described under the GCC -mcmodel option here: http://gcc.gnu.org/onlinedocs/gcc/i386-and-x86-64-Options.html

What you're experiencing is that crt1.o, the startup code that gets linked into every program that's responsible for taking the initial ELF register/stack state and passing them into the libc startup code which eventually calls main, seems to be using the small model. You can see here:

/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): relocation truncated to fit: R_X86_64_32S against symbol `main' defined in    .text section in /tmp/ccFshK69.o

What's happening is that crt1.o has a relocation for the address of main that only allows for a 32-bit address to be filled in. (Note: Even if main were defined in a shared library rather than the executable, there would be a PLT entry in the executable, and the address of this PLT entry would be the official address of main to which the relocation would be resolved.)

To solve this problem, you would need a crt1.o that can handle full 64-bit addresses. One way to get this might be by using Scrt1.o, which is normally only used for PIE executables, instead of crt1.o. You could probably achieve this with -nostartfiles and manually specifying all the start files on the link command line. It might be worth filing a bug report against glibc requesting that the x86_64 crt1.o be converted to "large model" so that it works with main programs not linked in the 32-bit range.

Note that you'll probably also need -mcmodel=large (or perhaps -fPIE would work) for all of your own code to make it link and work correctly at high addresses. This could make it considerably larger and slower. You might want to rethink why you're doing this and whether you really need to.