Why do i get `** stack smashing detected **` error?

Question

I am trying to link this assembly function with this c++ code:

.text 

.global _Z7scambiaRiS_

_Z7scambiaRiS_:
    pushq %rbp
    mov %rsp, %rbp

    mov (%rdi), %rax
    mov (%rsi), %rcx
    mov %rcx, (%rdi)
    mov %rax, (%rsi)

    leave
    ret

.data

#include <iostream>

void extern scambia(int &a, int &b);

int main()
{ 
    int a, b;
    std::cin>>a>>b;
    std::cout<<"a: "<<a<<" b: "<<b<<"\n";
    
    scambia(a, b);

    std::cout<<"a: "<<a<<" b: "<<b<<"\n";
    
    return 0;
};

Assembly is a intel x86-64 version.

The program works, but then it stops with segmentation fault, here is the output, given as input 3 4:

3 4 
a: 3 b: 4
a: 4 b: 3
*** stack smashing detected ***: terminated
Aborted (core dumped)

I don't really understand how i managed to compromise the stack, and even debugging with gdb i couldn't find out what went wrong

Answer 1

scambia is declared as taking references to int, which is 4 bytes on this platform. However

    mov (%rdi), %rax
    mov (%rsi), %rcx
    mov %rcx, (%rdi)
    mov %rax, (%rsi)

is doing 8-byte loads and stores, because rax and rcx are 64-bit registers. So each of them writes 4 bytes beyond the int that you passed in, which means that other stack space is overwritten, triggering the stack smashing detector.

To use the assembly function as is, it should be declared in C++ as taking references to int64_t or some other 64-bit type, and the types of a,b adjusted accordingly. (long int is 64 bits on many x86-64 platforms, but not all).

To keep the declaration as swapping two ints, change the assembly code to use 32-bit registers for the data being moved to and from memory:

    mov (%rdi), %eax
    mov (%rsi), %ecx
    mov %ecx, (%rdi)
    mov %eax, (%rsi)

Note that rdi and rsi should stay as is, since they are pointers, which are 64 bits.