How to tell gcc to not align function parameters on the stack?

Question

I am trying to decompile an executable for the 68000 processor into C code, replacing the original subroutines with C functions one by one.

The problem I faced is that I don't know how to make gcc use the calling convention that matches the one used in the original program. I need the parameters on the stack to be packed, not aligned.

Let's say we have the following function

int fun(char arg1, short arg2, int arg3) {
    return arg1 + arg2 + arg3;
}

If we compile it with

gcc -m68000 -Os -fomit-frame-pointer -S source.c

we get the following output

fun:
    move.b 7(%sp),%d0
    ext.w %d0
    move.w 10(%sp),%a0
    lea (%a0,%d0.w),%a0
    move.l %a0,%d0
    add.l 12(%sp),%d0
    rts

As we can see, the compiler assumed that parameters have addresses 7(%sp), 10(%sp) and 12(%sp):

but to work with the original program they need to have addresses 4(%sp), 5(%sp) and 7(%sp):

One possible solution is to write the function in the following way (the processor is big-endian):

int fun(int bytes4to7, int bytes8to11) {
    char arg1 = bytes4to7>>24;
    short arg2 = (bytes4to7>>8)&0xffff;
    int arg3 = ((bytes4to7&0xff)<<24) | (bytes8to11>>8);
    return arg1 + arg2 + arg3;
}

However, the code looks messy, and I was wondering: is there a way to both keep the code clean and achieve the desired result?

---

UPD: I made a mistake. The offsets I'm looking for are actually 5(%sp), 6(%sp) and 8(%sp) (the char-s should be aligned with the short-s, but the short-s and the int-s are still packed):

Hopefully, this doesn't change the essence of the question.

---

UPD 2: It turns out that the 68000 C Compiler by Sierra Systems gives the described offsets (as in UPD, with 2-byte alignment).

However, the question is about tweaking calling conventions in gcc (or perhaps another modern compiler).

Answer 1

Here's a way with a packed struct. I compiled it on an x86 with -m32 and got the desired offsets in the disassembly, so I think it should still work for an mc68000:

typedef struct {
    char arg1;
    short arg2;
    int arg3;
} __attribute__((__packed__)) fun_t;

int
fun(fun_t fun)
{

    return fun.arg1 + fun.arg2 + fun.arg3;
}

---

But, I think there's probably a still cleaner way. It would require knowing more about the other code that generates such a calling sequence. Do you have the source code for it?

Does the other code have to remain in asm? With the source, you could adjust the offsets in the asm code to be compatible with modern C ABI calling conventions.

I've been programming in C since 1981 and spent years doing mc68000 C and assembler code (for apps, kernel, device drivers), so I'm somewhat familiar with the problem space.

Answer 2

It's not a gcc 'fault', it is 68k architecture that requires stack to be always aligned on 2 bytes. So there is simply no way to break 2-byte alignment on the hardware stack.

but to work with the original program they need to have addresses 4(%sp), 5(%sp) and 7(%sp):

Accessing word or long values off the ODD memory address will immediately trigger alignment exception on 68000.