Debugging early kernel startup code in QEMU

Question

I have some code (a mix of assembly and C) compiled into an ELF binary, that some firmware/bootloader code in QEMU loads at a particular physical address. The ELF is linked using virtual addresses; however, the code is meant to be run with the MMU turned off, which means that it needs to be position-independent.

The problem is that the PC is set to the entry point's physical address, which makes sense, but since all symbols are referenced using virtual addresses, GDB does not know where the entry point is.

I can still set a breakpoint on a function that will be called when the MMU is turned on (and hence the PC will be dealing with virtual addresses at that point), but this is not good enough to debug early code, including single-stepping it.

I've been able to debug this code by linking it using physical addresses, but clearly this will create problems as soon as I turn on the MMU.

Does anyone know what I'm missing here?

Answer 1

I don't think your issue is particular to QEMU (or even ARM). I believe there are at least three ways to handle this,

1. Just use assembler
2. Use symbol-file
3. Overlays

Assembler

I had this problem, but would just switch to examine assembler. It is fairly easy especially if you objdump -S the MMU/normal address version to match the assembler. You can pipe the objdump output to a file and remove non-relocated code if you want. However, most editors will be able to handle the large file.

Looking at the assembler may be very helpful as often some sort of relocation constant or non-PIC type reference is not correct. You need both code/data to be correct and your PIC code may call some gcc library routine linked some place you didn't account for. So although this is primitive, it does have some advantages.

Symbol-file

Another way is to use what you have with symbol-file. Use the physical link with symbol-file during boot and then symbol-file of the MMU/normal link after the MMU switch. Obviously load the MMU version.

Overlays

I think you could handle this with overlays although you are actually doing an inverse. VMA is the MMU address and LMA is the physical boot address. Mark it as mapped initially and then you can update with _ovly_debug_event() when you turn on the MMU.

The overlays take code, but it is the most convenient. I would attempt that if you already have a 'debug' build targets in your software development system and conditionalized that code. However if you use actual overlays in ld, it may actually solve some of your issues and you might find this the most robust. The overlay resource impact is pretty minimal, but it exists.

---

Reference

• How to run code from RAM on ARM - similar issues to MMU/non-MMU.
• Relocation in ARM assembler - code for getting an offset (to adjust pointers).

The references are coding issue as to why you might want to debug this.