Why do I need double layer of indirection for macros?

Question

At: C++ FAQ - Miscellaneous technical issues - [39.6] What should be done with macros that need to paste two tokens together?

Could someone explain to me why? All I read is trust me, but I simply can't just trust on something because someone said so.

I tried the approach and I can't find any bugs appearing:

#define mymacro(a) int a ## __LINE__
mymacro(prefix) = 5;
mymacro(__LINE__) = 5;
int test = prefix__LINE__*__LINE____LINE__; // fine

So why do I need to do it like this instead (quote from the webpage):

However you need a double layer of indirection when you use ##. Basically you need to create a special macro for "token pasting" such as:

 #define NAME2(a,b)         NAME2_HIDDEN(a,b)
 #define NAME2_HIDDEN(a,b)  a ## b 

Trust me on this — you really need to do this! (And please nobody write me saying it sometimes works without the second layer of indirection. Try concatenating a symbol with __ LINE__ and see what happens then.)

Edit: Could someone also explain why he uses NAME2_HIDDEN before it's declared below? It seems more logical to define NAME2_HIDDEN macro before I use it. Is it some sort of trick here?

Answer 1

The relevant part of the C spec:

6.10.3.1 Argument substitution

After the arguments for the invocation of a function-like macro have been identified, argument substitution takes place. A parameter in the replacement list, unless preceded by a # or ## preprocessing token or followed by a ## preprocessing token (see below), is replaced by the corresponding argument after all macros contained therein have been expanded. Before being substituted, each argument’s preprocessing tokens are completely macro replaced as if they formed the rest of the preprocessing file; no other preprocessing tokens are available.

The key part that determines whether you want the double indirection or not is the second sentence and the exception in it -- if the parameter is involved in a # or ## operation (such as the params in mymacro and NAME2_HIDDEN), then any other macros in the argument are NOT expanded prior to doing the # or ##. If, on the other hand, there's no # or ## IMMEDIATELY in the macro body (as with NAME2), then other macros in the parameters ARE expanded.

So it comes down to what you want -- sometimes you want all macros expanded FIRST, and then do the # or ## (in which case you want the double layer indirection) and sometime you DO NOT want the macros expanded first (in which case you CAN'T HAVE double layer macros, you need to do it directly.)

Answer 2

__LINE__ is a special macro that is supposed to resolve to the current line number. When you do a token paste with __LINE__ directly, however, it doesn't get a chance to resolve, so you end up with the token prefix__LINE__ instead of, say, prefix23, like you would probably be expecting if you would write this code in the wild.

Answer 3

Chris Dodd has an excellent explanation for the first part of your question. As for the second part, about the definition sequence, the short version is that #define directives by themselves are not evaluated at all; they are only evaluated and expanded when the symbol is found elsewhere in the file. For example:

#define A a  //adds A->a to the symbol table
#define B b  //adds B->b to the symbol table

int A;

#undef A     //removes A->a from the symbol table
#define A B  //adds A->B to the symbol table

int A;

The first int A; becomes int a; because that is how A is defined at that point in the file. The second int A; becomes int b; after two expansions. It is first expanded to int B; because A is defined as B at that point in the file. The preprocessor then recognizes that B is a macro when it checks the symbol table. B is then expanded to b.

The only thing that matters is the definition of the symbol at the point of expansion, regardless of where the definition is.

Answer 4

The most non-technical answer, which I gathered from all links here, and link of links ;) is that, a single layer indirection macro(x) #x stringifies the inputted macro's name, but by using double layers, it will stringify the inputted macro's value.

#define valueOfPi 3
#define macroHlp(x) #x
#define macro(x) macroHlp(x)  
#define myVarOneLayer "Apprx. value of pi = " macroHlp(valueOfPi)
#define myVarTwoLayers "Apprx. value of pi = " macro(valueOfPi)

printf(myVarOneLayer); // out: Apprx. value of pi = valueOfPi 
printf(myVarOTwoLayers); // out: Apprx. value of pi = 3

What happens at printf(myVarOneLayer)

printf(myVarOneLayer) is expanded to printf("Apprx. value of pi = " macroHlp(valueOfPi))

macroHlp(valueOfPi) tries to stringify the input, the input itself is not evaluated. It's only purpose in life is to take an input and stringify. So it expands to "valueOfPi"

So, what happens at printf(myVarTwoLayers)

printf(myVarTwoLayers) is expanded to printf("Apprx. value of pi = " macro(valueOfPi)

macro(valueOfPi) has no stringification operation, i.e. there is no #x in it's expansion, but there is an x, so it has to evaluate x and input the value to macroHlp for stringification. It expands to macroHlp(3) which in turn will stringify the number 3, since it is using #x