Strange segmentation fault in assembly

Question

i have written a hexdump utility in assembly language using nasm compiler and ld linker.The programe is supposed to dump the hex values for any input file.However it segfaults at one particular procedure "LoadBuff".The function of loadbuff is to read input to a buffer of 16 bytes.the code is

LoadBuff:
    push ebx                 
    push edx
    push eax

    mov eax,3       ;sys_read call      
    mov ebx,0               ;read from standard input
    mov ecx,Buff            ;pass the buffer adress
    mov edx,BuffLen         ;pass the number of bytes to be read at a time
    int 80h                 ;call the linux kernel
    mov ebp,eax
    ;cmp eax,0              ;number of characters read is returned in eax
    ;jz exit                ;if zero character is returned i.e end of iinput file      
                            ;jump to exit

    xor ecx,ecx 
    pop eax
    pop edx
    pop ebx
    ret

If the lines

;cmp eax,0                   
;jz exit                      

are uncommented the code runs fine,without any seg fault.However,when i comment it and include these lines in the caller so as to do the same comparision in the caller and not here,this procedure seg faults.

gdb backtrace gives

#0  0x00000000 in ?? ()

Any idea why is it happening like that?

Answer 1

You're using NASM but you're not specifying if you're using Intel-style syntax or AT&T-style syntax. Looking at your sample code, however, I'm guessing it's Intel-style.

In Intel-style syntax, operations like mov work like this:

mov <destination>, <source>

They try, in other words, to mimic the "destination = source" style of thinking. In AT&T syntax it goes the other way:

mov <source>, <destination>

They view it, in other words, as if you're reading "move the source to the destination".

Now look at this line of code:

mov ebp, eax

If you're using Intel-style syntax (and I think you are because AT&T-style syntax would be mov %ebp, %eax), you're moving the contents of register eax into ebp. ebp is traditionally used as a "base pointer" ... note that word "pointer" there ... and is often used exactly as that. When you get your 0 in eax, you're overwriting the existing base pointer with a null pointer. Wacky hijinks ensue.

---

That's not the only problem, however. A further problem is this:

jz exit

I don't see an exit label anywhere in the code you've posted, so you're jumping SOMEWHERE (otherwise the assembler would whine) outside of your procedure. In the process you're passing over your stack cleanup code, leaving your stack in an unknown state. You've basically pushed the contents of three registers onto the stack and left them there where they're not expected by other routines.

The problem is that you're jumping past your cleanup code. At the beginning of your procedure you're pushing ebx, edx and eax. At the end of your procedure you correctly pop them in reverse order (eax, edx and ebx). This leaves the stack in the same state on exit that it had on entry and the code that relies on this is set to run as expected.

That jz jumps past that point, however, so wherever you're going there are three values on the stack that should not be there. You have to jump to your cleanup code, not past it.

The general rule is always pop what you push within your procedure. There are (a vanishingly small number of) exceptions to this rule, but they don't occur often enough to distract you with it right now.