ANTLR: mismatched input

Question

I couldn't understand a bug in my grammar. The file, Bug.g4, is:

grammar Bug;

text: TEXT;

WORD: ('a'..'z' | 'A'..'Z')+ ;
TEXT: ('a'..'z' | 'A'..'Z')+ ;

NEWLINE: [\n\r] -> skip ;

After running antlr4 and javac, I run

grun Bug text -tree
aa
line 1:0 mismatched input 'aa' expecting TEXT
(text aa)

But if I instead use text: WORD in the grammar, things are okay. What's wrong?

Answer 1

When two lexer rules each match the same string of text, and no other lexer rule matches a longer string of text, ANTLR assigns the token type according to the rule which appeared first in the grammar. In your case, a TEXT token can never be produced by the lexer rule because the WORD rule will always match the same text and the WORD rule appears before the TEXT rule in the grammar. If you were to reverse the order of these rules in the grammar, you would start to see TEXT tokens but you would never see a WORD token.