What makes executables compiled with gcc on ARM64 larger than on x86_64?

Question

It has been my observation that compiling the same code using gcc 12.2.0 on both Raspberry Pi OS Bookworm aarch64 and Debian Bookworm x86_64, always results in a much smaller executable on the latter. In both cases the same command-line is used:

gcc foo.c

It doesn't make a difference if the program is a simple "Hello World" type or something much larger and more complex -- the binary for x86_64 is markedly leaner than its counterpart for the Pi. To give an example, the following results in an executable file of only 15952 bytes on x86_64, compared with 70432 bytes on the Pi:

#include <stdio.h>

int main(int argc, char *argv[])
{
    printf("Hello world.\n");
    return 0;
}

The same comparison holds true for a larger project I've compiled for both platforms, with one binary coming in at 21968 bytes vs. 72528 bytes. Any ideas as to why this is so?

I was expecting the executable files to be roughly equal in size on both architectures.

Answer 1

It's an illusion

The ARM64 binary is not that much larger in any meaningful sense. The apparent file size is greater, but it's a sparse file, and the amount of actual disk space used is quite similar between the two systems.

On x86-64:

$ ls -l a.out 
-rwxrwxr-x 1 nate nate 15960 Sep 11 21:46 a.out
$ size a.out 
   text    data     bss     dec     hex filename
   1386     600       8    1994     7ca a.out
$ du -h a.out 
16K     a.out

ARM64:

$ ls -l a.out 
-rwxr-xr-x 1 nate nate 69808 Sep 11 21:40 a.out
$ size a.out 
   text    data     bss     dec     hex filename
   1602     640       8    2250     8ca a.out
$ du -h a.out 
16K a.out

So both files actually consume roughly 16KiB of disk space, and the amount of code and data is fairly similar (ARM64 about 15% larger).

---

So why does the ARM64 binary appear bigger?

The binary contains both code and static data. These must be loaded into memory on separate pages, because they need to have different permissions (read-only and executable for code, read-write and not executable for data). The binary is mapped into memory by a mechanism similar to mmap(2), and it likewise requires that regions to be mapped are at file offsets that are multiples of the page size.

On x86, the page size is always 4 KiB. But for ARM64, the hardware supports page sizes of either 4 KiB, 16 KiB, or 64 KiB. Linux can support any of these sizes, but the choice must be made when the kernel is configured and built, and then the entire system will use that page size.

So, if we want to support all ARM64 page size choices, since the data section must be at a file offset that's a multiple of the page size, it follows that its offset in the file must be at least 65536. (Offset 0 is occupied by the code section.) Hence the minimum (apparent) file size is a bit over 65536.

As the space between the end of the code section and the start of the next page (at offset 65536) is not actually used for anything, it can be made sparse. No disk blocks are allocated for this region of the file, and it reads (and is mapped into memory) as zeros.

Of course, the effect is magnified in this case since the binary is otherwise so small. For a much larger program, you would probably not have noticed the difference between a binary that's 4190283 bytes, and one that's 4253639 bytes.

---

If you like, you can tell the linker to use a smaller page size, via the option -z max-page-size, and the resulting binary will be "smaller" - though again, mainly just in apparent size. But it will no longer work on systems with a larger page size. (At least I assume not - I have not tested it.)

$ gcc -Wl,-z,max-page-size=16384 -o hello-16k.bin foo.c 
$ gcc -Wl,-z,max-page-size=4096 -o hello-4k.bin foo.c 
$ ls -l *.bin
-rwxr-xr-x 1 nate nate 20656 Sep 11 22:20 hello-16k.bin
-rwxr-xr-x 1 nate nate  8368 Sep 11 22:20 hello-4k.bin

I believe Raspberry Pi OS configures their stock kernel with 16 KiB pages. So hello-16k.bin should run on your system, but would not run if copied to a system using 64 KiB pages. And hello-4k.bin probably won't run on your system, but it works fine on mine, which is configured with 4 KiB pages. And note, hello-4k.bin is actually a lot smaller than the x86-64 binary.

Answer 2

The x86_64 "hello world" binary is smaller because the AMD64 ABI and Debian's toolchain generate leaner dynamic ELF executables. On Raspberry Pi OS (aarch64), the ABI uses larger relocations, bigger unwind info, and the toolchain links in bulkier startup/runtime objects by default. That overhead dominates for small programs, which is why your ARM64 executables look bloated compared to x86_64.