Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Segfault when running cross-compiled HelloWorld to armv5te on a board with CPU arm926ej-s

I have a board that has this CPU:

# uname -a
Linux gw-9167 4.4.24 #1 Thu Mar 28 17:52:19 UTC 2019 armv5tejl GNU/Linux
# cat /proc/cpuinfo
processor       : 0
model name      : ARM926EJ-S rev 5 (v5l)
BogoMIPS        : 226.09
Features        : swp half fastmult edsp java 
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant     : 0x0
CPU part        : 0x926
CPU revision    : 5

Hardware        : Freescale MXS (Device Tree)
Revision        : 0000
Serial          : 0000000000000000

I'm trying to cross-compile a simple Rust hello world to it:

[0] [05:56:25] ~/r/gw-test HEAD > /bin/cat .cargo/config
[target.armv5te-unknown-linux-gnueabi]
linker = "arm-none-eabi-gcc"

[target.armv5te-unknown-linux-musleabi]
linker = "arm-none-eabi-gcc"
[0] [05:56:34] ~/r/gw-test HEAD > env RUSTFLAGS="-C target-cpu=arm926ej-s" cargo build --target=armv5te-unknown-linux-musleabi --release
   Compiling gw-test v0.1.0 (/home/cecile/repos/gw-test)
    Finished release [optimized] target(s) in 0.28s
[0] [05:56:51] ~/r/gw-test HEAD > qemu-arm -L /usr/arm-linux-gnueabi -cpu arm926 target/armv5te-unknown-linux-musleabi/release/gw-test
Hello, world!

When I run it on the board itself, it segfaults immediately:

[0] [05:57:09] ~/r/gw-test HEAD > scp ......
gw-test                         100% 1781KB 906.4KB/s   00:01    
[0] [05:58:27] ~/r/gw-test HEAD > ssh ...... /tmp/gw-test
Segmentation fault
[139] [05:58:47] ~/r/gw-test HEAD > ssh ..... strace /tmp/gw-test
execve("/tmp/gw-test", ["/tmp/gw-test"], [/* 9 vars */]) = -1 EINVAL (Invalid argument)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} ---
+++ killed by SIGSEGV +++
Segmentation fault

What is the content of your main.rs?

Nothing! It's just the default hello world generated by Cargo

EINVAL An ELF executable had more than one PT_INTERP segment (i.e., tried to name more than one interpreter).

Now that's really weird because I also read in the doc: "If the executable is a dynamically linked ELF executable, the interpreter named in the PT_INTERP segment is used to load the needed shared libraries." and I have compiled with musl. It should be static (at least it says it is when I run ldd from the host machine on it).

Try to attach a debugger, either directly via gdb or remotely by using gdbserver.

I understand. I will try to see how to do it remotely as you suggested.

It seems like an error from strace and not from your process.

It makes sense.

What happens when you run it without strace? Is there a core dump?

It's in the log I already shared.

[0] [05:58:27] ~/r/gw-test HEAD > ssh ...... /tmp/gw-test
Segmentation fault

You are using arm-none-eabi-gcc, which is not targetting any operating system. Check readelf --headers ~/r/gw-test and compare it with a binary that does work, like /bin/cat or something.

Here is the output of readelf --headers on the binary I made and on cat:

[0] [08:11:23] ~ > cd ~/repos/gw-test/
[0] [08:15:26] ~/r/gw-test HEAD > ls
Cargo.lock  Cargo.toml  gw.json  lib-root  resources  src  target
[0] [08:15:27] ~/r/gw-test HEAD > readelf --headers target/armv5te-unknown-linux-musleabi/release/gw-test
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0x8278
  Start of program headers:          52 (bytes into file)
  Start of section headers:          1822748 (bytes into file)
  Flags:                             0x5000200, Version5 EABI, soft-float ABI
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         5
  Size of section headers:           40 (bytes)
  Number of section headers:         27
  Section header string table index: 26

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .init             PROGBITS        00008000 008000 00000c 00  AX  0   0  4
  [ 2] .text             PROGBITS        00008010 008010 0281b8 00  AX  0   0  8
  [ 3] .fini             PROGBITS        000301c8 0301c8 00000c 00  AX  0   0  4
  [ 4] .rodata           PROGBITS        000301e0 0301e0 005174 00   A  0   0 16
  [ 5] .ARM.extab        PROGBITS        00035354 035354 000bf4 00   A  0   0  4
  [ 6] .ARM.exidx        ARM_EXIDX       00035f48 035f48 0009e8 00  AL  2   0  4
  [ 7] .tdata            PROGBITS        00046930 036930 000018 00 WAT  0   0  8
  [ 8] .tbss             NOBITS          00046948 036948 000034 00 WAT  0   0  8
  [ 9] .data.rel.ro      PROGBITS        00046948 036948 000f98 00  WA  0   0  8
  [10] .got              PROGBITS        000478e0 0378e0 0000cc 04  WA  0   0  4
  [11] .data             PROGBITS        000479b0 0379b0 0000bc 00  WA  0   0  8
  [12] .bss              NOBITS          00047a70 037a6c 000a78 00  WA  0   0  8
  [13] .comment          PROGBITS        00000000 037a6c 00003b 01  MS  0   0  1
  [14] .debug_aranges    PROGBITS        00000000 037aa8 0012f0 00      0   0  8
  [15] .debug_info       PROGBITS        00000000 038d98 07d9b8 00      0   0  1
  [16] .debug_abbrev     PROGBITS        00000000 0b6750 00a574 00      0   0  1
  [17] .debug_line       PROGBITS        00000000 0c0cc4 04e271 00      0   0  1
  [18] .debug_frame      PROGBITS        00000000 10ef38 0101f4 00      0   0  4
  [19] .debug_str        PROGBITS        00000000 11f12c 02bc3f 01  MS  0   0  1
  [20] .debug_loc        PROGBITS        00000000 14ad6b 015230 00      0   0  1
  [21] .debug_macinfo    PROGBITS        00000000 15ff9b 000007 00      0   0  1
  [22] .debug_ranges     PROGBITS        00000000 15ffa8 033418 00      0   0  8
  [23] .ARM.attributes   ARM_ATTRIBUTES  00000000 1933c0 000031 00      0   0  1
  [24] .symtab           SYMTAB          00000000 1933f4 01e590 10     25 7245  4
  [25] .strtab           STRTAB          00000000 1b1984 00b595 00      0   0  1
  [26] .shstrtab         STRTAB          00000000 1bcf19 000103 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  y (purecode), p (processor specific)

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x035f48 0x00035f48 0x00035f48 0x009e8 0x009e8 R   0x4
  LOAD           0x000000 0x00000000 0x00000000 0x36930 0x36930 R E 0x10000
  LOAD           0x036930 0x00046930 0x00046930 0x0113c 0x01bb8 RW  0x10000
  TLS            0x036930 0x00046930 0x00046930 0x00018 0x0004c R   0x8
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x10

 Section to Segment mapping:
  Segment Sections...
   00     .ARM.exidx
   01     .init .text .fini .rodata .ARM.extab .ARM.exidx
   02     .tdata .data.rel.ro .got .data .bss
   03     .tdata .tbss
   04
[127] [08:19:01] ~/r/gw-test HEAD > scp -r -i ~/Downloads/hub_keys/sshKeys/0200000100009167 [email protected]:/bin/cat ./
cat                                                                                                                                                                                                                                                         100%  651KB 581.0KB/s   00:01
[0] [08:19:18] ~/r/gw-test HEAD > stat cat
  File: cat
  Size: 666788          Blocks: 1304       IO Block: 4096   regular file
Device: 18h/24d Inode: 4034474     Links: 1
Access: (0755/-rwxr-xr-x)  Uid: ( 1000/  cecile)   Gid: ( 1000/  cecile)
Access: 2019-04-07 10:19:17.131861726 +0200
Modify: 2019-04-07 10:19:18.251842664 +0200
Change: 2019-04-07 10:19:18.251842664 +0200
 Birth: 2019-04-07 10:19:17.131861726 +0200
[0] [08:19:22] ~/r/gw-test HEAD > readelf --headers cat
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0xc914
  Start of program headers:          52 (bytes into file)
  Start of section headers:          665748 (bytes into file)
  Flags:                             0x5000002, Version5 EABI, <unknown>
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         8
  Size of section headers:           40 (bytes)
  Number of section headers:         26
  Section header string table index: 25

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .interp           PROGBITS        00008134 000134 000013 00   A  0   0  1
  [ 2] .note.ABI-tag     NOTE            00008148 000148 000020 00   A  0   0  4
  [ 3] .hash             HASH            00008168 000168 000990 04   A  4   0  4
  [ 4] .dynsym           DYNSYM          00008af8 000af8 0015b0 10   A  5   1  4
  [ 5] .dynstr           STRTAB          0000a0a8 0020a8 000b33 00   A  0   0  1
  [ 6] .gnu.version      VERSYM          0000abdc 002bdc 0002b6 02   A  4   0  2
  [ 7] .gnu.version_r    VERNEED         0000ae94 002e94 000020 00   A  5   1  4
  [ 8] .rel.dyn          REL             0000aeb4 002eb4 000050 08   A  4   0  4
  [ 9] .rel.plt          REL             0000af04 002f04 000a60 08   A  4  11  4
  [10] .init             PROGBITS        0000b964 003964 00000c 00  AX  0   0  4
  [11] .plt              PROGBITS        0000b970 003970 000fa4 04  AX  0   0  4
  [12] .text             PROGBITS        0000c914 004914 07dcd8 00  AX  0   0  4
  [13] .fini             PROGBITS        0008a5ec 0825ec 000008 00  AX  0   0  4
  [14] .rodata           PROGBITS        0008a5f8 0825f8 01f208 00   A  0   0  8
  [15] .ARM.exidx        ARM_EXIDX       000a9800 0a1800 000008 00  AL 12   0  4
  [16] .eh_frame         PROGBITS        000a9808 0a1808 000004 00   A  0   0  4
  [17] .init_array       INIT_ARRAY      000b2000 0a2000 000004 00  WA  0   0  4
  [18] .fini_array       FINI_ARRAY      000b2004 0a2004 000004 00  WA  0   0  4
  [19] .jcr              PROGBITS        000b2008 0a2008 000004 00  WA  0   0  4
  [20] .dynamic          DYNAMIC         000b200c 0a200c 0000e8 08  WA  5   0  4
  [21] .got              PROGBITS        000b20f4 0a20f4 000544 04  WA  0   0  4
  [22] .data             PROGBITS        000b2638 0a2638 00014e 00  WA  0   0  4
  [23] .bss              NOBITS          000b2788 0a2786 0022c0 00  WA  0   0  8
  [24] .ARM.attributes   ARM_ATTRIBUTES  00000000 0a2786 000034 00      0   0  1
  [25] .shstrtab         STRTAB          00000000 0a27ba 0000da 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  y (purecode), p (processor specific)

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x0a1800 0x000a9800 0x000a9800 0x00008 0x00008 R   0x4
  PHDR           0x000034 0x00008034 0x00008034 0x00100 0x00100 R E 0x4
  INTERP         0x000134 0x00008134 0x00008134 0x00013 0x00013 R   0x1
      [Requesting program interpreter: /lib/ld-linux.so.3]
  LOAD           0x000000 0x00008000 0x00008000 0xa180c 0xa180c R E 0x8000
  LOAD           0x0a2000 0x000b2000 0x000b2000 0x00786 0x02a48 RW  0x8000
  DYNAMIC        0x0a200c 0x000b200c 0x000b200c 0x000e8 0x000e8 RW  0x4
  NOTE           0x000148 0x00008148 0x00008148 0x00020 0x00020 R   0x4
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4

 Section to Segment mapping:
  Segment Sections...
   00     .ARM.exidx
   01
   02     .interp
   03     .interp .note.ABI-tag .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt .init .plt .text .fini .rodata .ARM.exidx .eh_frame
   04     .init_array .fini_array .jcr .dynamic .got .data .bss
   05     .dynamic
   06     .note.ABI-tag
   07
[0] [08:19:32] ~/r/gw-test HEAD > /usr/arm-none-eabi/bin/readelf --headers cat
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0xc914
  Start of program headers:          52 (bytes into file)
  Start of section headers:          665748 (bytes into file)
  Flags:                             0x5000002, Version5 EABI, <unknown>
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         8
  Size of section headers:           40 (bytes)
  Number of section headers:         26
  Section header string table index: 25

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .interp           PROGBITS        00008134 000134 000013 00   A  0   0  1
  [ 2] .note.ABI-tag     NOTE            00008148 000148 000020 00   A  0   0  4
  [ 3] .hash             HASH            00008168 000168 000990 04   A  4   0  4
  [ 4] .dynsym           DYNSYM          00008af8 000af8 0015b0 10   A  5   1  4
  [ 5] .dynstr           STRTAB          0000a0a8 0020a8 000b33 00   A  0   0  1
  [ 6] .gnu.version      VERSYM          0000abdc 002bdc 0002b6 02   A  4   0  2
  [ 7] .gnu.version_r    VERNEED         0000ae94 002e94 000020 00   A  5   1  4
  [ 8] .rel.dyn          REL             0000aeb4 002eb4 000050 08   A  4   0  4
  [ 9] .rel.plt          REL             0000af04 002f04 000a60 08   A  4  11  4
  [10] .init             PROGBITS        0000b964 003964 00000c 00  AX  0   0  4
  [11] .plt              PROGBITS        0000b970 003970 000fa4 04  AX  0   0  4
  [12] .text             PROGBITS        0000c914 004914 07dcd8 00  AX  0   0  4
  [13] .fini             PROGBITS        0008a5ec 0825ec 000008 00  AX  0   0  4
  [14] .rodata           PROGBITS        0008a5f8 0825f8 01f208 00   A  0   0  8
  [15] .ARM.exidx        ARM_EXIDX       000a9800 0a1800 000008 00  AL 12   0  4
  [16] .eh_frame         PROGBITS        000a9808 0a1808 000004 00   A  0   0  4
  [17] .init_array       INIT_ARRAY      000b2000 0a2000 000004 00  WA  0   0  4
  [18] .fini_array       FINI_ARRAY      000b2004 0a2004 000004 00  WA  0   0  4
  [19] .jcr              PROGBITS        000b2008 0a2008 000004 00  WA  0   0  4
  [20] .dynamic          DYNAMIC         000b200c 0a200c 0000e8 08  WA  5   0  4
  [21] .got              PROGBITS        000b20f4 0a20f4 000544 04  WA  0   0  4
  [22] .data             PROGBITS        000b2638 0a2638 00014e 00  WA  0   0  4
  [23] .bss              NOBITS          000b2788 0a2786 0022c0 00  WA  0   0  8
  [24] .ARM.attributes   ARM_ATTRIBUTES  00000000 0a2786 000034 00      0   0  1
  [25] .shstrtab         STRTAB          00000000 0a27ba 0000da 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  y (purecode), p (processor specific)

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x0a1800 0x000a9800 0x000a9800 0x00008 0x00008 R   0x4
  PHDR           0x000034 0x00008034 0x00008034 0x00100 0x00100 R E 0x4
  INTERP         0x000134 0x00008134 0x00008134 0x00013 0x00013 R   0x1
      [Requesting program interpreter: /lib/ld-linux.so.3]
  LOAD           0x000000 0x00008000 0x00008000 0xa180c 0xa180c R E 0x8000
  LOAD           0x0a2000 0x000b2000 0x000b2000 0x00786 0x02a48 RW  0x8000
  DYNAMIC        0x0a200c 0x000b200c 0x000b200c 0x000e8 0x000e8 RW  0x4
  NOTE           0x000148 0x00008148 0x00008148 0x00020 0x00020 R   0x4
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4

 Section to Segment mapping:
  Segment Sections...
   00     .ARM.exidx
   01
   02     .interp
   03     .interp .note.ABI-tag .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt .init .plt .text .fini .rodata .ARM.exidx .eh_frame
   04     .init_array .fini_array .jcr .dynamic .got .data .bss
   05     .dynamic
   06     .note.ABI-tag
   07

Solution:

I made this Dockerfile:

FROM buildpack-deps:xenial

RUN dpkg --add-architecture i386 && \
    apt-get update && \
    apt-get install -y --no-install-recommends \
        apt-utils build-essential sudo libssl-dev \
        subversion wget cpio python unzip rsync bc libncurses-dev \
        libc6:i386 libncurses5:i386 libstdc++6:i386 ca-certificates \
        ant ant-optional bison curl flex fusesmb

COPY ./dp-mgw-toolchain_1.6_i386.deb ./
RUN dpkg -i dp-mgw-toolchain_1.6_i386.deb

RUN curl https://sh.rustup.rs -sSf | bash -s -- -y

RUN echo '. ~/.cargo/env' >> ~/.bashrc

RUN . ~/.cargo/env && \
    rustup target add armv5te-unknown-linux-musleabi

ENV PATH=/opt/develco-products/buildroot-toolchain-multigateway-1.6/host/usr/bin:$PATH
ENV RUSTFLAGS="-C target-cpu=arm926ej-s"

RUN cargo build --target=armv5te-unknown-linux-musleabi --release

The deb file is actually the toolchain provided by the manufacturer. Now I'm using the correct linker and it runs properly on the board.

like image 747
Cecile Avatar asked Mar 31 '19 06:03

Cecile


1 Answers

My first bet are incompatible ABI. Your binary:

0x5000200, Version5 EABI, soft-float ABI

Binaries from your board:

0x5000002, Version5 EABI, <unknown>

Next bet, your binary has many sections missing, so it looks like linker issue. Your QEMU environment isn't identical to that used on your target, but I'd expect your QEMU runs bare metal binary. These cannot run under Linux as would not have direct access to hardware & peripherals.

You need to setup your linker to link ELF compatible with environment on your board. I don't have experience with Freescale but looking at any project that cross compiles for this board should lead you to proper linker invocation.

like image 58
Michał Fita Avatar answered Nov 08 '22 00:11

Michał Fita