C ABI: is changing a void function to return an int a breaking change?

Question

Is there any non-exotic architecture/OS/compiler where the change of:

void func(void *, int, int)

to:

int func(void *, int, int)

would break the ABI? (i.e. a program compiled for "void" version of a shared library would break when run with the "int" version)

Answer 1

Several comments have said no, and we can strengthen that with a little reasoning. In the most familiar ABIs, a function with declaration void func(void *, int, int) is permitted to use the register in which an int result would be returned as a scratch register; it need not save and restore it. A function with declaration int func(void *, int, int) is required to use the register in which an int result would be returned. In other respects, these declarations are identical. Therefore, the machine code of any implementation of an int func(void *, int, int) is also machine code that satisfies void func(void *, int, int).

Put another way, the caller has no way of distinguishing machine code that deliberately returns a result in the return register from code that happens to leave some scratch calculation in that register.

Note that this reasoning requires that the called function be hidden behind an ABI; it relies on the binary behavior specified by the ABI. If the source of the called function is visible when the called function is compiled (or other parts of the implementation that might affect how the call is implemented), then optimization could result in behaviors that bypass the ABI (e.g., observing that the called routine does not use the return register and therefore using it in the caller to hold some value expected to remain unchanged over the call).

Since you say this is for a shared library, you are likely safe: The shared library is compiled separately, and its source is not available to its callers. However, you should consider side channels. For example, perhaps the shared library is part of a set of libraries that share sources and contain cross-library calls. In this case, somebody might have an old version of a shared library that was compiled with a view to the void source and a new version of the shared library that contains the int source. (And even this requires unusual source code arrangements or fancy compilers that integrate information across multiple compilation units.)