Shared library injection: _dl_relocate_object segfaults

Question

I was recently experimenting with shared library injection in Linux and decided to write my own program to do it (instead of using say GDB to inject the library).

My program uses pthread to overwrite the first 0x25 bytes of a loaded program program (0x40000-0x400025) with assembly code to allocate space for the filename and call dlopen. Once all of this is done, it restores the program state and detaches from it.

Here's the assembly:

global inject_library
global nullsub

section .data
section .text

inject_library:
; rdi -> Pointer to malloc()
; rsi -> Pointer to free()
; rdx -> Pointer to dlopen()
; rcx -> Size of the path to the .so to load

; Create a new stack frame
push rbp

; Save rbx because we're using it as scratch space
push rbx
; Save addresses of free & dlopen on the stack
push rsi
push rdx

; Move the pointer to malloc into rbx
mov rbx, rdi
; Move the size of the path as the first argument to malloc
mov rdi, rcx
; Call malloc(so_path_size)
call rbx
; Stop so that we can see what's happening from the injector process
int 0x3

; Move the pointer to dlopen into rbx
pop rbx
; Move the malloc'd space (now containing the path) to rdi for the first argument
mov rdi, rax
; Push rax because it'll be overwritten
push rax
; Second argument to dlopen (RTLD_NOW)
mov rsi, 0x2
; Call dlopen(path_to_library, RTLD_NOW)
call rbx
; Pass control to the injector
int 0x3

; Finally, begin free-ing the malloc'd area
pop rdi
; Get the address of free into rbx
pop rbx
; Call free(path_to_library)
call rbx

; Restore rbx
pop rbx

; Destory the stack frame
pop rbp

; We're done
int 0x3
retn

nullsub:
retn

There's also a C program which calls this assembly routine and uses pthread to handle these breakpoints.

This setup works just fine for small, single threaded programs like the following.

#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>

int main(int argc, char* argv) {
    pid_t my_pid = getpid();
    printf("PID: %ld\n", my_pid);
    getchar();
    return 0;
}

I used a simple shared library that just did puts("Hi"); in its constructor. As stated above, everything upto here works perfectly.

However, when I try to inject the same library into a much bigger (external, closed-source program), I run into a segfault.

Here's the backtrace:

#0  0x00007f6a7985d64d in _dl_relocate_object (scope=0x21fbc08, reloc_mode=reloc_mode@entry=0, consider_profiling=consider_profiling@entry=0)
    at dl-reloc.c:259
#1  0x00007f6a79865723 in dl_open_worker (a=a@entry=0x7fff82d7cbf8) at dl-open.c:424
#2  0x00007f6a793cf5d4 in __GI__dl_catch_error (objname=objname@entry=0x7fff82d7cbe8, errstring=errstring@entry=0x7fff82d7cbf0, 
    mallocedp=mallocedp@entry=0x7fff82d7cbe7, operate=operate@entry=0x7f6a798654c0 <dl_open_worker>, args=args@entry=0x7fff82d7cbf8)
    at dl-error-skeleton.c:198
#3  0x00007f6a79865069 in _dl_open (file=0x21fb830 "/home/umang/code/insertion/test_library.so", mode=-2147483646, caller_dlopen=0x40001a, nsid=-2, 
    argc=<optimized out>, argv=<optimized out>, env=0x7fff82d7cfe8) at dl-open.c:649
#4  0x00007f6a7964ef96 in dlopen_doit (a=a@entry=0x7fff82d7ce08) at dlopen.c:66
#5  0x00007f6a793cf5d4 in __GI__dl_catch_error (objname=objname@entry=0x7f6a798510f0 <last_result+16>, 
    errstring=errstring@entry=0x7f6a798510f8 <last_result+24>, mallocedp=mallocedp@entry=0x7f6a798510e8 <last_result+8>, 
    operate=operate@entry=0x7f6a7964ef40 <dlopen_doit>, args=args@entry=0x7fff82d7ce08) at dl-error-skeleton.c:198
#6  0x00007f6a7964f665 in _dlerror_run (operate=operate@entry=0x7f6a7964ef40 <dlopen_doit>, args=args@entry=0x7fff82d7ce08) at dlerror.c:163
#7  0x00007f6a7964f021 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#8  0x000000000040001a in ?? ()
#9  0x00000000021fb830 in ?? ()
#10 0x00007f6a79326a90 in ?? () at malloc.c:3071 from /lib64/libc.so.6
#11 0x00007f6a796488a0 in ?? () from /lib64/libc.so.6
#12 0x0000000000000d68 in ?? ()
#13 0x00007f6a7931e938 in _IO_new_file_underflow (fp=0x7f6a7964efe0 <__dlopen>) at fileops.c:600
#14 0x00007f6a7931fa72 in __GI__IO_default_uflow (fp=0x7f6a796488a0 <_IO_2_1_stdin_>) at genops.c:404
#15 0x00007f6a7931a20d in getchar () at getchar.c:37
#16 0x00000000004005d7 in main ()

This backtrace tells me something went (horribly) wrong in the dlopen call. Specifically, the error lies at glibc dl-reloc.c:259.

Here's the questionable glibc code.

254          l->l_lookup_cache.value = _lr; }))                   \
255      : l)
256 
257 #include "dynamic-link.h"
258 
259     ELF_DYNAMIC_RELOCATE (l, lazy, consider_profiling, skip_ifunc);
260 
261 #ifndef PROF
262     if (__glibc_unlikely (consider_profiling)
263     && l->l_info[DT_PLTRELSZ] != NULL)

ELF_DYNAMIC_RELOCATE is a macro defined in dynamic-link.h as the following -

/* This can't just be an inline function because GCC is too dumb
   to inline functions containing inlines themselves.  */
# define ELF_DYNAMIC_RELOCATE(map, lazy, consider_profile, skip_ifunc) \
  do {                                        \
    int edr_lazy = elf_machine_runtime_setup ((map), (lazy),              \
                          (consider_profile));        \
    ELF_DYNAMIC_DO_REL ((map), edr_lazy, skip_ifunc);                 \
    ELF_DYNAMIC_DO_RELA ((map), edr_lazy, skip_ifunc);                \
  } while (0)

#endif

elf_machine_runtime_setup returns just fine, so I'm assuming that the problem lies with ELF_DYNAMIC_DO_REL. This is the source for the mentioned macro. The problem here is that the called method is inline, so GDB only displays the macro name and not the underlying source.

Using ni in GDB, I see the following after elf_machine_runtime_setup returns:

ELF_DYNAMIC_RELOCATE (l, lazy, consider_profiling, skip_ifunc);
ELF_DYNAMIC_RELOCATE (l, lazy, consider_profiling, skip_ifunc);
ELF_DYNAMIC_RELOCATE (l, lazy, consider_profiling, skip_ifunc);

Stepping through assembly, the segfault happens after the following instruction: movaps %xmm0,-0x70(%rbp).

info local isn't of much help:

(gdb) info local
ranges = {{start = 140072440991568, size = 0, nrelative = 0, lazy = 670467104}, {start = 0, size = 140072438891376, nrelative = 140072441065920, 
    lazy = 672664367}}
textrels = 0x0
errstring = 0x0
lazy = <optimized out>
skip_ifunc = 0

Interestingly enough, when I use GDB to inject the shared library (using this code I found somewhere on the net), the library loads perfectly.

sudo gdb -n -q -batch \
  -ex "attach $pid" \
  -ex "set \$dlopen = (void*(*)(char*, int)) dlopen" \
  -ex "call \$dlopen(\"$(pwd)/libexample.so\", 1)" \
  -ex "detach" \
  -ex "quit"
)"

Thanks in advance!

Answer 1

After days of scratching my head and ripping off my hair, I decided to Google "MOVAPS segfault".

MOVAPS is a SIMD instruction (and here, it is used to quickly zero out a quadword). Here's some more info about the same.

On taking a closer look, I noticed the following paragraph:

When the source or destination operand is a memory operand, the operand must be aligned on a 16-byte boundary or a general-protection exception (#GP) is generated.

Hmm. So I read the value of the offending address.

(gdb) print $rbp - 0x70
$2 = (void *) 0x7ffecd32e838

There. The address isn't aligned to a 16-byte boundary and thus the segfault occurs.

Fixing this was easy.

; Create a new stack frame
push rbp
sub rsp, 0x8
; Do stuff
; Fix the stack pointer
add rsp, 0x8
; Destroy stack frame, return, etc.

I'm still doubtful if this is the right way to do it, but it works.

Oh, and GDB got it right the whole time - it made sure that the stack was aligned.