Understand U-Boot memory footprint

Question

I don't understand what is happening in RAM when loading U-Boot. I'm working on a Xilinx Zynq ZC702 evaluation kit and I'm trying to load a Linux kernel on it using U-Boot. So I used the Xilinx tool Vivado and the SDK to generate a BOOT.bin file that is written on an SD card step by step:

• Create an HW project using Vivado,
• Generate FSBL and FPGA bitstream using the SDK,
• Create a boot image containing the FSBL + bitstream + U-Boot (I downloaded the U-Boot sources from the xilinx Git repository).

In short, I followed all the steps described on Xilinx User Guide.

But now, before loading the kernel, I wanted to understand what was happening but I can't. According to the documentation, U-Boot, if loaded from flash, copies itself into RAM, and executes itself from there, but where ?

I searched on the web and I found that the address where U-Boot extracts itself is defined in include/configs/zynq-common.h by CONFIG_SYS_TEXT_BASE, which appear to be 0x400_0000.

But on an other website I saw that we can print the relocation address in debug mode, so I modified the file common/board_r.c and overwrote the function "initr_announce" to print the "gd->relocaddr" field. This time it appear that U-Boot uses the offset address 0x3FF3_7000.

When I use U-Boot command "md" to inspect the memory I see that the two offsets are used, and I see a sort of magic number "be00_00ea" at both locations :

Xilinx First Stage Boot Loader 
Release 2014.4  Feb  8 2016-14:53:56
Devcfg driver initialized 
Silicon Version 3.1
Boot mode is SD
SD: rc= 0
SD Init Done 
Flash Base Address: 0xE0100000
Reboot status register: 0x60400000
Multiboot Register: 0x0000C000
Image Start Address: 0x00000000
Partition Header Offset:0x00000C80
Partition Count: 3
Partition Number: 1
Header Dump
Image Word Len: 0x000F6EC0
Data Word Len: 0x000F6EC0
Partition Word Len:0x000F6EC0
Load Addr: 0x00000000
Exec Addr: 0x00000000
Partition Start: 0x000065D0
Partition Attr: 0x00000020
Partition Checksum Offset: 0x00000000
Section Count: 0x00000001
Checksum: 0xFFD14B7E
Bitstream
In FsblHookBeforeBitstreamDload function 
PCAP:StatusReg = 0x40000A30
PCAP:device ready
PCAP:Clear done
Level Shifter Value = 0xA 
Devcfg Status register = 0x40000A30 
PCAP:Fabric is Initialized done
PCAP register dump:
PCAP CTRL 0xF8007000: 0x4C00E07F
PCAP LOCK 0xF8007004: 0x0000001A
PCAP CONFIG 0xF8007008: 0x00000508
PCAP ISR 0xF800700C: 0x0802000B
PCAP IMR 0xF8007010: 0xFFFFFFFF
PCAP STATUS 0xF8007014: 0x00000A30
PCAP DMA SRC ADDR 0xF8007018: 0x00100001
PCAP DMA DEST ADDR 0xF800701C: 0xFFFFFFFF
PCAP DMA SRC LEN 0xF8007020: 0x000F6EC0
PCAP DMA DEST LEN 0xF8007024: 0x000F6EC0
PCAP ROM SHADOW CTRL 0xF8007028: 0xFFFFFFFF
PCAP MBOOT 0xF800702C: 0x0000C000
PCAP SW ID 0xF8007030: 0x00000000
PCAP UNLOCK 0xF8007034: 0x757BDF0D
PCAP MCTRL 0xF8007080: 0x30800100

DMA Done ! 

FPGA Done ! 
In FsblHookAfterBitstreamDload function 
Partition Number: 2
Header Dump
Image Word Len: 0x0001BA12
Data Word Len: 0x0001BA12
Partition Word Len:0x0001BA12
Load Addr: 0x04000000
Exec Addr: 0x04000000
Partition Start: 0x000FD490
Partition Attr: 0x00000010
Partition Checksum Offset: 0x00000000
Section Count: 0x00000001
Checksum: 0xF7EAFAC8
Application
Handoff Address: 0x04000000
In FsblHookBeforeHandoff function 
SUCCESSFUL_HANDOFF
FSBL Status = 0x1


U-Boot 2015.07 (Feb 11 2016 - 10:24:28 +0100)

Model: Zynq ZC702 Development Board
I2C:   ready
DRAM:  ECC disabled 1 GiB
MMC:   zynq_sdhci: 0
SF: Detected N25Q128A with page size 256 Bytes, erase size 64 KiB, total 16 MiB
In:    serial
Out:   serial
Err:   serial
Model: Zynq ZC702 Development Board
Net:   Gem.e000b000
Hit any key to stop autoboot:  0 
zynq-uboot> md 0x4000000
04000000: ea0000be e59ff014 e59ff014 e59ff014    ................
04000010: e59ff014 e59ff014 e59ff014 e59ff014    ................
04000020: 04000060 040000c0 04000120 04000180    `....... .......
04000030: 040001e0 04000240 040002a0 deadbeef    ....@...........
04000040: 0badc0de e320f000 e320f000 e320f000    ...... ... ... .
04000050: e320f000 e320f000 e320f000 e320f000    .. ... ... ... .
04000060: e51fd028 e58de000 e14fe000 e58de004    (.........O.....
04000070: e3a0d013 e169f00d e1a0e00f e1b0f00e    ......i.........
04000080: e24dd048 e88d1fff e51f2050 e892000c    H.M.....P ......
04000090: e28d0048 e28d5034 e1a0100e e885000f    H...4P..........
040000a0: e1a0000d eb0005dc e320f000 e320f000    .......... ... .
040000b0: e320f000 e320f000 e320f000 e320f000    .. ... ... ... .
040000c0: e51fd088 e58de000 e14fe000 e58de004    ..........O.....
040000d0: e3a0d013 e169f00d e1a0e00f e1b0f00e    ......i.........
040000e0: e24dd048 e88d1fff e51f20b0 e892000c    H.M...... ......
040000f0: e28d0048 e28d5034 e1a0100e e885000f    H...4P..........
zynq-uboot> md 0x3ff37000
3ff37000: ea0000be e59ff014 e59ff014 e59ff014    ................
3ff37010: e59ff014 e59ff014 e59ff014 e59ff014    ................
3ff37020: 3ff37060 3ff370c0 3ff37120 3ff37180    `p.?.p.? q.?.q.?
3ff37030: 3ff371e0 3ff37240 3ff372a0 deadbeef    .q.?@r.?.r.?....
3ff37040: 3f312628 e320f000 e320f000 e320f000    (&1?.. ... ... .
3ff37050: e320f000 e320f000 e320f000 e320f000    .. ... ... ... .
3ff37060: e51fd028 e58de000 e14fe000 e58de004    (.........O.....
3ff37070: e3a0d013 e169f00d e1a0e00f e1b0f00e    ......i.........
3ff37080: e24dd048 e88d1fff e51f2050 e892000c    H.M.....P ......
3ff37090: e28d0048 e28d5034 e1a0100e e885000f    H...4P..........
3ff370a0: e1a0000d eb0005dc e320f000 e320f000    .......... ... .
3ff370b0: e320f000 e320f000 e320f000 e320f000    .. ... ... ... .
3ff370c0: e51fd088 e58de000 e14fe000 e58de004    ..........O.....
3ff370d0: e3a0d013 e169f00d e1a0e00f e1b0f00e    ......i.........
3ff370e0: e24dd048 e88d1fff e51f20b0 e892000c    H.M...... ......
3ff370f0: e28d0048 e28d5034 e1a0100e e885000f    H...4P..........
zynq-uboot> 

Why U-Boot need those two offsets ? And what is the real memory footprint of U-Boot ? And more generally where can I put my kernel to be sure that it will not overwrite something ?

Answer 1

u-boot relocation on ARM architecture

Here is the whole sequence for two-stage boot process:

1. ROM-code reads SPL (from MLO file on SD card) to CONFIG_SPL_TEXT_BASE address. This address is usually in SRAM which doesn't need to be initialized to be functional (as opposed to RAM). ROM-code jumps to SPL code.
2. SPL configures RAM, then reads u-boot (from u-boot.img file on SD card) to CONFIG_SYS_TEXT_BASE RAM address (which is usually in the begin of RAM) and run it
3. u-boot relocates itself to gd->relocaddr RAM address (which is usually in the end of RAM) and jumps to relocated code
4. Now we are ready to boot kernel

For single-stage boot you don't have SPL, usually only u-boot.bin file is used. In that case you only have steps 3 and 4.

There are two cases regarding relocation (as described in doc/README.arm-relocation):

1. CONFIG_SYS_TEXT_BASE != gd->relocaddr: relocation will be performed
2. CONFIG_SYS_TEXT_BASE == gd->relocaddr: relocation will not be performed

In your case you see that relocation was performed (as CONFIG_SYS_TEXT_BASE != gd->relocaddr).

So, addressing your question:

Why U-Boot need those two offsets ?

The reason behind that relocation is described in u-boot ARM relocation task:

...we can measure the actual size of memory present on the board, and then relocate U-Boot to the very end of the RAM, leaving nearly the whole RAM usable as one big contiguous area for "applications" like loading Linux kernel, ramdisk, etc.

Indeed, if you look to code you can see that gd->relocaddr is the end of RAM minus monitor code (U-Boot) size:

gd->relocaddr = gd->ram_top;
...
gd->relocaddr -= gd->mon_len;

Also some extra memory reserving can be performed. For example, on my platform (TI DRA7XX EVM), I can see next functions beeing called:

setup_dest_addr()
reserve_round_4k()
reserve_mmu()
reserve_uboot()

The actual relocation is done just after board_init_f() call.

arch/arm/lib/crt0.S:

bl board_init_f
...
b relocate_code

arch/arm/lib/relocate.S:

ENTRY(relocate_code)

Now it's easy to answer your next question:

what is the real memory footprint of U-Boot ?

Before relocation U-Boot resides at CONFIG_SYS_TEXT_BASE. After relocation U-Boot resides at gs->relocaddr.

Regarding your last question:

where can I put my kernel to be sure that it will not overwrite something?

Since U-Boot was relocated to the end of RAM you can theoretically use any RAM address to put kernel to. But take a look at CONFIG_EXTRA_ENV_SETTINGS definition in include/configs/zynq-common.h:

"sdboot=if mmcinfo; then " \
        "run uenvboot; " \
        "echo Copying Linux from SD to RAM... && " \
        "load mmc 0 ${kernel_load_address} ${kernel_image} && " \
        "load mmc 0 ${devicetree_load_address} ${devicetree_image} && " \
        "load mmc 0 ${ramdisk_load_address} ${ramdisk_image} && " \
        "bootm ${kernel_load_address} ${ramdisk_load_address} ${devicetree_load_address}; " \
    "fi\0" \

From there you can see that you should load kernel to ${kernel_load_address}, which is 0x2080000:

"kernel_load_address=0x2080000\0" \

See rest of that definition for other constants.

bdinfo command

You can find bdinfo command useful: relocation address along with other useful information can be found using bdinfo command from U-Boot shell. For example for DRA7XX EVM:

=> bdinfo

DRAM bank   = 0x00000000
-> start    = 0x80000000
-> size     = 0x60000000
TLB addr    = 0xDFFF0000
relocaddr   = 0xDFF5D000
reloc off   = 0x5F75D000
irq_sp      = 0xDEF3CEE0
sp start    = 0xDEF3CED0

From here you can see that:

• RAM starts at 0x80000000
• RAM size is 0x60000000
• ...so RAM end (gd->ram_top) is 0x80000000 + 0x60000000 = 0xE0000000
• relocation address is 0xDFF5D000
• reserved memory for relocation is 0xE0000000 - 0xDFF5D000 = 652 KB
• monitor (U-Boot) size is approximately TLB addr - relocaddr = 0xDFFF0000 - 0xDFF5D000 = 588 KB

See also:

[1] u-boot : Relocation

[2] what is the use of SPL (secondary program loader)

[3] commit that adds ARM relocation support to u-boot