[antlr-interest] How to get the last token type just recognized in the lexer?

Gavin Lambert antlr at mirality.co.nz
Fri Nov 21 12:45:21 PST 2008


At 02:30 22/11/2008, chain one wrote:
>INT
>:    (DIGIT)+
>;
>
>FLOAT
>:    '.' (DIGIT)+ (('e' | 'E') ('+' | '-')? (DIGIT)+)?
>|    '.' ('e' | 'E') ('+' | '-')? (DIGIT)+
>      ;
>
>DIGIT
>:'0'..'9'
>;
[...]
>So it seems that the only solution is to predict the last token 
>just recognized, if the last token is a INT, then FLOAT rule 
>should be followed.
>
>so I need to know what the last token is.
>calling function LA(-1) could do this, however LA(-1) is only 
>available in parser not in lexer

You're asking the wrong question.  Instead of trying to work out 
INTs followed by FLOATs, simply recognise it as a FLOAT in the 
first place:

protected DIGIT: '0'..'9';

protected EXPONENT: ('e' | 'E') ('+' | '-')? (DIGIT)+;

FLOAT
   : '.' (DIGIT)+ (EXPONENT)?
   ;

INT
   : (DIGIT)+
     ( FLOAT { $setType(FLOAT); }
     | EXPONENT { $setType(FLOAT); }
     )?
   ;

(".e16" by itself shouldn't be recognised as a float anyway.)



More information about the antlr-interest mailing list