How are memory addresses placed in an binary files?

Question

Im having trouble understanding how the sections in an elf file are loaded into memory and how the addresses are chosen? Embedded systems usually assign specific addresses to the code but where is that put?

Basically, how and when are addresses put in the sections and how is that loaded in both an OS and with ROM or RAM in an embedded system.

Answer 1

A specific operating system has a specific set of rules or possibly multiple sets of rules for where a compatible program can be loaded. The toolchain including default linker script (think gcc hello.c -o hello) made for that platform conforms to these rules.

So for example I decide to create an operating system, for a platform that has an MMU. Because it has an MMU I can create the operating system such that every program sees the same (virtual) address space. So I can decide that for applications on my operating system, the memory space starts at 0x00000000 but the entry point must be 0x00001000. The binary file format supported is a Motorola s-record let's say.

So take a simple program with a simple linker script

MEMORY
{
    ram : ORIGIN = 0x1000, LENGTH = 0x10000
}
SECTIONS
{
    .text : { *(.text*) } > ram
}

The disassembly of my simple program

00001000 <_start>:
    1000:   e3a0d902    mov sp, #32768  ; 0x8000
    1004:   eb000001    bl  1010 <main>
    1008:   e3a00000    mov r0, #0
    100c:   ef000000    svc 0x00000000

00001010 <main>:
    1010:   e3a00000    mov r0, #0
    1014:   e12fff1e    bx  lr

And the "binary" file happens to be human readable:

S00F00006E6F746D61696E2E737265631F
S3150000100002D9A0E3010000EB0000A0E3000000EF1E
S30D000010100000A0E31EFF2FE122
S70500001000EA

and you may or may not notice that the address is indeed in the binary describing where things go.

Being an operating system based program that is loaded into ram we don't have to play too many games with memory, we can assume one flat all ram (read/write) so if there were .data, .bss, etc it can all be packed in there.

For a real operating system it is desirable to have the binary include additional information perhaps the size of the program. So you can google around the various common file formats and see how this is done, either a simple up front I need this much, or one to many sections individually defined. And yes, again the "binary" is more than just opcodes and data, I assume you understand that.

The toolchain I used outputs elf formatted files by default but objcopy can be used to create a number of different formats one of which is a raw memory image (which does not contain any address/location information) many/most of the rest contain the machine code and data as well as labels for the debugger/disassembler or addresses for where chunks of this data wants to live in the memory space, etc.

Now when you say embedded and use the words ROM and RAM I assume you mean bare-metal like a microcontroller for example, but even if you mean booting an x86 or full sized ARM or whatever the same things apply. In the case of an MCU the chip designers have perhaps per the rules of the processor or their own choice have determined the rules for the memory space. Just like an operating system will dictate the rules. We are cheating a bit, as a lot of the tools we use today (gnu based) are not really designed for bare-metal but because a generic compiler is a generic compiler and more important a toolchain lends itself for this kind of portability, we can use such tools. Ideally using a cross compiler meaning the output machine code is not necessarily meant to run on the computer generating that output machine code. The major difference that matters is that we want to control the linking and libraries, don't link in host operating system based libraries and let us control or for this toolchain have a default linker script that targets our MCU. So lets say I have an ARM7TDMI based MCU, and the chip designers say I need the binary such that the ROM starts at address 0x00000000 and is of some size, and RAM starts at 0x40000000 and is of some size. Being an ARM7 the processor starts execution by fetching the instruction at address 0x00000000 and the chip designers have mapped that 0x00000000 to the ROM.

So now my simple program

unsigned int xyz;
int notmain ( void )
{
    xyz=5;
    return(0);
}

linked like this

MEMORY
{
    bob : ORIGIN = 0x00000000, LENGTH = 0x1000
    ted : ORIGIN = 0x40000000, LENGTH = 0x1000
}
SECTIONS
{
    .text : { *(.text*) } > bob
    .bss : { *(.bss*) } > ted
}

gives a disassembly of this

Disassembly of section .text:

00000000 <_start>:
   0:   e3a0d101    mov sp, #1073741824 ; 0x40000000
   4:   e38dda01    orr sp, sp, #4096   ; 0x1000
   8:   eb000000    bl  10 <notmain>
   c:   eafffffe    b   c <_start+0xc>

00000010 <notmain>:
  10:   e3a02005    mov r2, #5
  14:   e59f3008    ldr r3, [pc, #8]    ; 24 <notmain+0x14>
  18:   e3a00000    mov r0, #0
  1c:   e5832000    str r2, [r3]
  20:   e12fff1e    bx  lr
  24:   40000000    andmi   r0, r0, r0

Disassembly of section .bss:

40000000 <xyz>:
40000000:   00000000    andeq   r0, r0, r0

And that would be a perfectly valid program, doesn't do much interesting, but still a perfectly valid program.

First and foremost if you leave out _start, the toolchain gives a warning but still functions just fine. (hmm, actually didn't warn that time, interesting).

arm-none-eabi-as --warn --fatal-warnings vectors.s -o vectors.o
arm-none-eabi-gcc -Wall -Werror -O2 -nostdlib -nostartfiles -ffreestanding -c notmain.c -o notmain.o
arm-none-eabi-ld vectors.o notmain.o -T memmap -o notmain.elf
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary notmain.bin

And now you have the loading issue. Each MCU is different as to how you load it what tools are available and/or you make your own tools. Ihex and srec were popular for prom programmers where you had say a separate rom next to your processor and/or the through hole mcu would get plugged into the prom programmer. raw binary images work too but can quickly get large as will show in a second. As written above there is .bss but no .data so

ls -al notmain.bin
-rwxr-xr-x 1 user user 40 Oct 21 22:05 notmain.bin

40 bytes. But if I do this for demonstration purposes, even though it wont work correctly:

unsigned int xyz=5;
int notmain ( void )
{
    return(0);
}

with

MEMORY
{
    bob : ORIGIN = 0x00000000, LENGTH = 0x1000
    ted : ORIGIN = 0x40000000, LENGTH = 0x1000
}
SECTIONS
{
    .text : { *(.text*) } > bob
    .bss : { *(.bss*) } > ted
    .data : { *(.data*) } > ted
}

gives

Disassembly of section .text:

00000000 <notmain-0x10>:
   0:   e3a0d101    mov sp, #1073741824 ; 0x40000000
   4:   e38dda01    orr sp, sp, #4096   ; 0x1000
   8:   eb000000    bl  10 <notmain>
   c:   eafffffe    b   c <notmain-0x4>

00000010 <notmain>:
  10:   e3a00000    mov r0, #0
  14:   e12fff1e    bx  lr

Disassembly of section .data:

40000000 <xyz>:
40000000:   00000005    andeq   r0, r0, r5

and

-rwxr-xr-x  1 user user 1073741828 Oct 21 22:08 notmain.bin

OUCH! 0x40000004 bytes, which was expected, I asked for a memory image I defined stuff at one address (machine code) and a few bytes at another (0x40000000) so the raw memory image has to be that whole range.

hexdump notmain.bin 
0000000 d101 e3a0 da01 e38d 0000 eb00 fffe eaff
0000010 0000 e3a0 ff1e e12f 0000 0000 0000 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
*
40000000 0005 0000                              
40000004

Instead one would just use the elf file the toolchain generates or maybe an ihex or srecord.

S00F00006E6F746D61696E2E737265631F
S3150000000001D1A0E301DA8DE3000000EBFEFFFFEA79
S30D000000100000A0E31EFF2FE132
S3094000000005000000B1
S70500000000FA

all the info I need but not a huge file for so few bytes.

Not a hard and fast rule but moving data around is easier today (than a floppy from one computer to another with the prom programmer on it). And particularly i if you have a bundled IDE that vendor likely uses the toolchains default format, but even if not elf and other similar formats are supported and you don't have to go the route of a raw binary or an ihex or srec. But it still depends on the tool that takes the "binary" and programs it into the ROM(/FLASH) on the MCU.

Now I cheated to demonstrate the large file problem above, instead you have to do more work when it is not a ram only system. If you feel the need to have .data or desire to have .bss zeroed then you need to write or use a more complicated linker script that helps you out with the locations and boundaries. And that linker script is married to the bootstrap that uses linker generated information to perform those tasks. Basically a copy of .data needs to be preserved in non-volatile memory (ROM/FLASH) but it cant live there at runtime .data is read/write so ideally/typically you use the linker scripts language/magic to state that the .data read/write space is blah, and the flash space is boo at this address and this size so the bootstrap can copy from flash at that address for that amount of data to ram. And for .bss the linker script generates variables that we save into flash that tell the bootstrap to zero ram from this address to this address.

So operating system defines the memory space, the linker script matches that if you want the program to work. The system designers or chip designers determine the address space for something embedded and the linker script matches that. The bootstrap is married to the linker script for that build and target.

---

Edit

toolchain basics...

mov sp,#0x40000000
orr sp,sp,#0x1000
bl notmain
b .


unsigned int xyz;
int notmain ( void )
{
    xyz=5;
    return(0);
}

MEMORY
{
    bob : ORIGIN = 0x1000, LENGTH = 0x1000
    ted : ORIGIN = 0x2000, LENGTH = 0x1000
}
SECTIONS
{
    .text : { *(.text*) } > bob
    .bss : { *(.bss*) } > ted
}

My bootstrap, main program and linker script

arm-none-eabi-as --warn --fatal-warnings vectors.s -o vectors.o
arm-none-eabi-gcc -Wall -Werror -O2 -nostdlib -nostartfiles -ffreestanding -save-temps -c notmain.c -o notmain.o
arm-none-eabi-ld vectors.o notmain.o -T memmap -o notmain.elf
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary notmain.bin

Some folks will argue and is sometimes true that compiles don't generate assembly any more. Still the sane way to do it and you will find it more often than not, as in this case...

The bootstrap makes an object which we can disassemble.

00000000 <.text>:
   0:   e3a0d101    mov sp, #1073741824 ; 0x40000000
   4:   e38dda01    orr sp, sp, #4096   ; 0x1000
   8:   ebfffffe    bl  0 <notmain>
   c:   eafffffe    b   c <.text+0xc>

It's not "linked" so the address this disassembler uses is zero based, and you can see the call to notmain is incomplete, not yet linked.

the compiler generated assembly for the C code

    .cpu arm7tdmi
    .fpu softvfp
    .eabi_attribute 20, 1
    .eabi_attribute 21, 1
    .eabi_attribute 23, 3
    .eabi_attribute 24, 1
    .eabi_attribute 25, 1
    .eabi_attribute 26, 1
    .eabi_attribute 30, 2
    .eabi_attribute 34, 0
    .eabi_attribute 18, 4
    .file   "notmain.c"
    .text
    .align  2
    .global notmain
    .type   notmain, %function
notmain:
    @ Function supports interworking.
    @ args = 0, pretend = 0, frame = 0
    @ frame_needed = 0, uses_anonymous_args = 0
    @ link register save eliminated.
    mov r2, #5
    ldr r3, .L2
    mov r0, #0
    str r2, [r3]
    bx  lr
.L3:
    .align  2
.L2:
    .word   xyz
    .size   notmain, .-notmain
    .comm   xyz,4,4
    .ident  "GCC: (15:4.9.3+svn231177-1) 4.9.3 20150529 (prerelease)"

that gets assembled into an object which we can also disassemble.

Disassembly of section .text:

00000000 <notmain>:
   0:   e3a02005    mov r2, #5
   4:   e59f3008    ldr r3, [pc, #8]    ; 14 <notmain+0x14>
   8:   e3a00000    mov r0, #0
   c:   e5832000    str r2, [r3]
  10:   e12fff1e    bx  lr
  14:   00000000    andeq   r0, r0, r0

Now not shown but that object also contains information for the global variable xyz and its size.

The linkers job is perhaps part of your confusion. It links the objects together such that the result will be sane or will work on the final destination (bare-metal or operating system).

Disassembly of section .text:

00001000 <notmain-0x10>:
    1000:   e3a0d101    mov sp, #1073741824 ; 0x40000000
    1004:   e38dda01    orr sp, sp, #4096   ; 0x1000
    1008:   eb000000    bl  1010 <notmain>
    100c:   eafffffe    b   100c <notmain-0x4>

00001010 <notmain>:
    1010:   e3a02005    mov r2, #5
    1014:   e59f3008    ldr r3, [pc, #8]    ; 1024 <notmain+0x14>
    1018:   e3a00000    mov r0, #0
    101c:   e5832000    str r2, [r3]
    1020:   e12fff1e    bx  lr
    1024:   00002000    andeq   r2, r0, r0

Disassembly of section .bss:

00002000 <xyz>:
    2000:   00000000    andeq   r0, r0, r0

I made this linker script so that you can see both .data and .bss moving around. The linker has filled in all of the .text into the 0x1000 address space and has patched in the call to notmain() as well as how to reach xyz. It has also allocated/defined the space for the xyz variable in the 0x2000 address space.

And then to your next question or confusion. It is very much up to the tools that load the system, be it the operating system loading a program into memory to be run, or programming the flash of an MCU or programming the ram of some other embedded system (like a mouse for example which you might not know some of them the firmware is downloaded from the operating system and not all of it burned into a flash /lib/firmware or other locations).