Does this abuse of function declarations invoke undefined behavior?

Question

Consider the following program:

int main()
{
    int exit();
    ((void(*)())exit)(0);
}

As you can see, exit is declared with the wrong return type, but is never called with the incorrect function type. Is this program's behavior well-defined?

Answer 1

MSVC has no problem with this program, but gcc does (at least gcc 4.6.1). It issues the following warnings:

test.c: In function 'main':
test.c:3:9: warning: conflicting types for built-in function 'exit' [enabled by default]
test.c:4:22: warning: function called through a non-compatible type [enabled by default]
test.c:4:22: note: if this code is reached, the program will abort

And, as promised, it does crash when run. The crash is no accident of an incorrect calling convention or something - gcc actually generates an undefined instruction with the opcode 0x0b0f to explicitly force a crash (gdb disassembles it as ud2 - I haven't looked up what that the CPU manual might say about the opcode):

main:
.LFB0:
    .cfi_startproc
    push    ebp
    .cfi_def_cfa_offset 8
    .cfi_offset 5, -8
    mov ebp, esp
    .cfi_def_cfa_register 5
        .value  0x0b0f
    .cfi_endproc

I'm reluctant to say that gcc is wrong in doing this because I'm sure the people who write that compiler know a lot more about C than I do. But here's how I read what the standard says about it; I'm sure someone will point out what I'm missing:

C99 says this about conversions of function pointers (6.3.2.3/8 "Pointers"):

A pointer to a function of one type may be converted to a pointer to a function of another type and back again; the result shall compare equal to the original pointer. If a converted pointer is used to call a function whose type is not compatible with the pointed-to type, the behavior is undefined.

In an expression, the identifier exit evaluates to a function pointer.

The sub-expression ((void(*)())exit) converts the function pointer that exit evaluates to into a function pointer of the type void (*)(). Then a function call is made through that pointer, passing the int argument 0.

The standard library contains a function named exit that has the following prototype:

void exit(int status);

The standard also says (7.1.4/2 "Use of library functions"):

Provided that a library function can be declared without reference to any type defined in a header, it is also permissible to declare the function and use it without including its associated header.

Your program doesn't include the header containing that prototype, but the function call made through the converted pointer uses the 'declaration' provided in the cast. The declaration in the cast isn't a prototype declaration, so we need to determine if the function type of exit as defined by the standard library and the function type of the converted function pointer in your program are compatible. The standard says (6.7.5.3/15 "Function declarators (including prototypes)")

For two function types to be compatible, both shall specify compatible return types. ... If one type has a parameter type list and the other type is specified by a function declarator that is not part of a function definition and that contains an empty identifier list, the parameter list shall not have an ellipsis terminator and the type of each parameter shall be compatible with the type that results from the application of the default argument promotions

It seems to me that the converted function pointer has a compatible function type - the return type is the same (void) and the type of the single parameter is int after the default argument promotions. So it appears to me that there's no undefined behavior here.

---

Update: After a little more thought on this, it might be reasonable to interpret 7.1.4/2 to mean that a 'self-declared' library function name must be declared correctly (though not necessarily with a prototype, but with a correct return type). Especially since the standard also says that "All identifiers with external linkage in any of the following subclauses ... are always reserved for use as identifiers with external linkage" (7.1.3).

So I think a reasonable argument can be made that the program has undefined behavior.