Why does stringizing an euro sign within a string literal using UTF8 not produce an UCN?

Question

The spec says that at phase 1 of compilation

Any source file character not in the basic source character set (2.3) is replaced by the universal-character-name that designates that character.

And at phase 4 it says

Preprocessing directives are executed, macro invocations are expanded

At phase 5, we have

Each source character set member in a character literal or a string literal, as well as each escape sequence and universal-character-name in a character literal or a non-raw string literal, is converted to the corresponding member of the execution character set

For the # operator, we have

a \ character is inserted before each " and \ character of a character literal or string literal (including the delimiting " characters).

Hence I conducted the following test

#define GET_UCN(X) #X
GET_UCN("€")

With an input character set of UTF-8 (matching my file's encoding), I expected the following preprocessing result of the #X operation: "\"\\u20AC\"". GCC, Clang and boost.wave don't transform the € into a UCN and instead yield "\"€\"". I feel like I'm missing something. Can you please explain?

Answer 1

It's simply a bug. §2.1/1 says about Phase 1,

(An implementation may use any internal encoding, so long as an actual extended character encountered in the source file, and the same extended character expressed in the source file as a universal-character-name (i.e. using the \uXXXX notation), are handled equivalently.)

This is not a note or footnote. C++0x adds an exception for raw string literals, which might solve your problem at hand if you have one.

This program clearly demonstrates the malfunction:

#include <iostream>

#define GET_UCN(X) L ## #X

int main() {
std::wcout << GET_UCN("€") << '\n' << GET_UCN("\u20AC") << '\n';
}

http://ideone.com/lb9jc

Because both strings are wide, the first is required to be corrupted into several characters if the compiler fails to interpret the input multibyte sequence. In your given example, total lack of support for UTF-8 could cause the compiler to slavishly echo the sequence right through.