C preprocessor, recursive macros

Question

Why does M(0) and N(0) have different results?

#define CAT_I(a, b) a ## b
#define CAT(a, b) CAT_I(a, b)

#define M_0 CAT(x, y)
#define M_1 whatever_else
#define M(a) CAT(M_, a)
M(0);       //  expands to CAT(x, y)

#define N_0() CAT(x, y)
#define N_1() whatever_else
#define N(a) CAT(N_, a)()
N(0);       //  expands to xy

Answer 1

In fact, it depends on your interpretation of the language standard. For example, under mcpp, a preprocessor implementation that strictly conforms to the text of the language standard, the second yields CAT(x, y); as well [extra newlines have been removed from the result]:

C:\dev>mcpp -W0 stubby.cpp
#line 1 "C:/dev/stubby.cpp"
        CAT(x, y) ;
        CAT(x, y) ;
C:\dev>

There is a known inconsistency in the C++ language specification (the same inconsistency is present in the C specification, though I don't know where the defect list is for C). The specification states that the final CAT(x, y) should not be macro-replaced. The intent may have been that it should be macro-replaced.

To quote the linked defect report:

Back in the 1980's it was understood by several WG14 people that there were tiny differences between the "non-replacement" verbiage and the attempts to produce pseudo-code.

The committee's decision was that no realistic programs "in the wild" would venture into this area, and trying to reduce the uncertainties is not worth the risk of changing conformance status of implementations or programs.

---

So, why do we get different behavior for M(0) than for N(0) with most common preprocessor implementations? In the replacement of M, the second invocation of CAT consists entirely of tokens resulting from the first invocation of CAT:

M(0) 
CAT(M_, 0)
CAT_I(M_, 0)
M_0
CAT(x, y)

If M_0 was instead defined to be replaced by CAT(M, 0), replacement would recurse infinitely. The preprocessor specification explicitly prohibits this "strictly recursive" replacement by stopping macro replacement, so CAT(x, y) is not macro replaced.

However, in the replacement of N, the second invocation of CAT consists only partially of tokens resulting from the first invocation of CAT:

N(0)
CAT(N_, 0)       ()
CAT_I(N_, 0)     ()
N_0              ()
CAT(x, y)
CAT_I(x, y)
xy

Here the second invocation of CAT is formed partially from tokens resulting from the first invocation of CAT and partially from other tokens, namely the () from the replacement list of N. The replacement is not strictly recursive and thus when the second invocation of CAT is replaced, it cannot yield infinite recursion.