[antlr-interest] Lexing problem I cannot resolve

[Carter Cheng]

How would you declare the FLOAT rule in this case? 


--- On Sun, 8/3/08, Gavin Lambert <antlr at mirality.co.nz> wrote:

> From: Gavin Lambert <antlr at mirality.co.nz>
> Subject: Re: [antlr-interest] Lexing problem I cannot resolve
> To: carter_cheng at yahoo.com, antlr-interest at antlr.org
> Date: Sunday, August 3, 2008, 3:50 AM
> At 22:16 3/08/2008, Carter Cheng wrote:
>  >1..2
>  >
>  >Which the lexer seems to like to lex as two FLOATS as
> oppose to 
> as
>  >INT RANGE INT. In the language in question FLOAT FLOAT
> is 
> illegal
>  >but obviously the lexer cannot know that. Is there a
> way to 
> resolve
>  >this in ANTLR cleanly?
> 
> Presumably it's splitting it up into
> FLOAT["1."] FLOAT[".2"]?
> 
> For starters, you could declare the former one to be an
> illegal 
> FLOAT -- after all it's a bit odd to have a trailing
> period with 
> no following digits.
> 
> But whether you choose to make that illegal or not (and you
> don't 
> *have* to), you'll need to modify the FLOAT rule to
> look ahead, 
> see two periods, and exit without matching either.
> 
> Something along these lines ought to do the trick:
> 
> fragment DIGIT: '0'..'9';
> RANGE: '..';
> INT
>    : DIGIT+
>      ( ('..') => /* RANGE; ignore */
>      | '.' DIGIT* { $type = FLOAT; }
>      )?
>    | ('.' DIGIT) => '.' DIGIT+ { $type =
> FLOAT; }
>    ;
> 
> (If you want to make "1." illegal, then changing
> DIGIT* to DIGIT+ 
> on the sixth line ought to do the trick.)
> 
> You *might* need to merge the RANGE rule into the INT rule
> as 
> well, but I think the above will work ok as is.