Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ANTLR3 lexer precedence

I want to create a token from '..' in the ANTLR3 lexer which will be used to string together expressions like

a..b     // [1]
c .. x   // [2]
1..2     // [3] 
3 .. 4   // [4]

So, I have added,

DOTDOTSEP : '..' 
          ;

The problem is that I already have a rule:

FLOAT : INT (('.' INT (('e'|'E') INT)? 'f'?) | (('e'|'E') INT)? ('f'))
      ;

And in example [3] above 1..2 is getting matched as a FLOAT (I'm not sure why since following the first . is another . not an INT, but it is).

I wonder if there is a way to change the precedence of the lexer rules so DOTDOTSEP gets matched first and then FLOAT.

Looking here it seems I'm losing out to, "The rule having the greatest count is the winner.", but wonder if there is a way around it.

P.S. INT is defined as below ...

fragment DIGIT
    : '0'..'9'
    ;

INT : DIGIT+
    ;

Edit. A little further testing makes me think it's not quite as simple as it getting directly matched to the FLOAT rule. (I was going to change the question but since I've got answers now, I won't.) The problem (I believe) still lies in the lexer rules precedence so the question still remains the same.

like image 439
tjm Avatar asked Oct 15 '10 12:10

tjm


1 Answers

Did you look at http://sds.sourceforge.net/src/antlr/doc/lexer.html ?

A possible solution is to define the following:

fragment
INT : DIGIT+
    ;

fragment
RANGE : INT DOTDOTSEP INT
      ;

fragment
FLOAT : INT (('.' INT (('e'|'E') INT)? 'f'?) | (('e'|'E') INT)? ('f'))
      ;

NUMBER
    : (INT '.') => FLOAT       { $type=FLOAT; }
    | (INT DOTDOTSEP) => RANGE { $type=RANGE; }
    | INT                      { $type=INT; }
    ;
like image 74
Marc Avatar answered Oct 12 '22 15:10

Marc