[antlr-interest] Emit two Tokens in one Lexer Rule

[Jim Idle]

> -----Original Message-----
> From: Gavin Lambert [mailto:antlr at mirality.co.nz]
> Sent: Tuesday, February 12, 2008 11:26 AM
> To: Alexander Gängel; antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Emit two Tokens in one Lexer Rule
> 
> At 02:16 13/02/2008, Alexander Gängel wrote:
>  >
>  >My problem it that I have to decide if I have
> an Float like  1.1 or
>  >an Int DOT Identifiert lik 5.max
>  >My workaround would be to emit just the first an restart lexing
>  >from the last correct charterer. So I would Just emit 5 as Int
>  >and reparse from the Dot.
> 
> Which is easy enough with a syntactic predicate:
> 
> fragment FLOAT: NUMBER DOT NUMBER;
> INT
>    : NUMBER
>      ((DOT NUMBER) => DOT NUMBER { $type = FLOAT; })?
>    ;

And even better if you dont repdicate the entire number set ;-)

((DOT DIGIT) => ....

> fragment NUMBER: DIGIT+;
> fragment DIGIT: '0'..'9';
>