implementing memset in assembly x86

Question

I'm trying to implement memset in assembly x86. both with a copy by byte and by word, so I'd get two functions: kmemset and kmemsetw that I expose to my C code this way:

extern uint8_t* kmemset(uint8_t* dest, uint8_t val, uint8_t size);  
extern uint16_t* kmemsetw(uint16_t* dest, uint16_t val, uint8_t size);  

problem is when I test it, I get a segmentation fault. I tried to debug with gdb but it doesn't seem to be able to step into the asm code. I would be happy if anyone could comment on the code. (kmemset is pretty similar so I didn't include it).

.intel_syntax noprefix  
.section .text  
.align 16  
.global kmemsetw  
.type kmemsetw, @function  

kmemsetw:  
    push ebp  
    mov ebp, esp  
    push edi  
    push ecx  
    push ebx  
    xor eax, eax  
    mov ebx, [ebp+4]  
    mov ax, [ebp+8]  
    mov ecx, [ebp+12]  
    mov edi, ebx  
    rep stosw  
    mov eax, edi  
    pop ebx  
    pop ecx  
    pop edi  
    pop ebp  
    ret  

Answer 1

Your not using ebx in your procedure, why save it? It does not need to be saved. ecx is a volatile register, you do not need to save it.

As gnometorule, mentioned, your parameters in your proc are off.

Another biggie is that you do not restore the stack pointer at the end. Sure you pop ebp, but where do you mov esp, ebp??? You mov ebp, esp at the beginning, but never reverse it.

If you look at memset, it returns the pointer passed to the proc. So, this is wrong: mov eax, edi it should be: mov eax, [ebp + 8]. rep stos? increments the pointer in edi so if you return edi, the returned pointer is wrong.

But why even set up a stack frame for this tiny proc? Just use esp, since we need to save edi to the stack, the params in esp will be offset the same as if we set up a stack frame:

kmemset:      
    push    edi             ; proc uses edi, so save it.

    mov     ecx, [esp + 16] ; size_t num
    mov     al, [esp + 12]  ; int value 
    mov     edi, [esp + 8]  ; void * ptr
    rep     stosb 

    mov     eax, [esp + 8]  ; return pointer
    pop     edi             ; restore edi
    ret                     ; let caller adjust stack

using stosw will be a bit different.

SomeProc:
    push    ebp
    mov     ebp, esp
    push    edi

    ; params are at:
     ;~ ebp + 8
     ;~ ebp + 12
     ;~ ebp + 16
     etc...
    ; ...
    ; ...
    ; ...

    pop     edi
    ; the following 2 lines
    ; can be replaced with
    ; leave
    mov     esp, ebp
    pop     ebp
    ret