ATmega328P hex file, reverse engineering and debugging

Question

I have a compiled hex file for the ATmega328P microcontroller, but no source code. I need to find a way to debug this in the AVR simulator.

What I already did:

• Installed Microchip Studio, which has an AVR simulator and debugger. But it can't load hex files. It expects only source or ASM files.
• Disassembled hex file with IDA or AVRDisassembler tool, but again Microchip Studio does not accept this disassembly format.

I read about the avr8js emulator project, but I have no idea how to plug any debugger into this simulator.

Could you please advise me on tools on Windows/Linux/MacOSX platforms for do that?

Answer 1

You could use avr-gdb.

GDB will accept a hex file, but obviously debugging functionality will be limited due to the lack of debug symbols:

avr-gdb firmware.hex

To debug the program in a simulator, you can use simavr, which exposes a GDB RSP debug server, to which you can connect via GDB.

simavr -g --mcu atmega328p --freq 1000000 firmware.hex

Then, to connect to simavr's GDB server, in GDB:

(gdb) target remote :1234

At this point, you can debug the program in GDB.

To disassemble program memory in GDB, you can use the x command:

(gdb) x/10bfi 0x100

The 10b is the number of bytes, the fi instructs GDB to disassemble the memory, and the 0x100 is the start address. So the above command will disassemble 10 bytes from address 0x100 in program memory.

I recommend using a fairly recent version of avr-gdb. Version 10.1 or newer. The one shipped by Microchip is severely outdated.

If you have an AVR-compatible debug tool, such as an Atmel-ICE or PICKit4, you can use a tool like Bloom to debug your program. Bloom will connect to your debug tool and expose an interface to the target via a GDB server, much like simavr. But Bloom is only available on Linux.

Disclosure: I am the author of Bloom.