How is `>>>` lexed in C++0x?

Question

>>> is lexed as >> >. But what happens if the first > closes a template argument list, should the result be equivalent to > > > or > >>?

It does matter in the following code:

template<class T> struct X { };

void operator >>(const X<int>&, int) { }

int main() {
    *new X<int>>> 1;
}

Answer 1

The text of the FDIS says

Similarly, the first non-nested >> is treated as two consecutive but distinct > tokens

It cannot unlex tokens and relex. So this will be a > > >. Note that the input to a C++ implementation is first lexed into preprocessing tokens, and then those tokens are converted into C++ tokens. So first your input are the C++ tokens >> >, then the C++ parser changes these to > > >.

Each preprocessing token is converted into a token. (2.7). The resulting tokens are syntactically and semantically analyzed and translated as a translation unit. [ Note: The process of analyzing and translating the tokens may occasionally result in one token being replaced by a sequence of other tokens (14.2). — end note ]

There's no chance you could merge those two trailing > > tokens.

Answer 2

In that particular piece of code, my understanding is that it will be > >>. The parser is greedy and will try to bundle as much as possible into each single token, when the first > is encountered, the context rule will dictate that it is a full token and that it should not try to parse more, but once it is outside of the template arguments' context it will parse the rest following the general rules, as if it was X<int> >>, or

typedef X<int> X_int;
X_int >> 1;