[antlr-interest] Lexing problem I cannot resolve

[Carter Cheng]

Thanks for the replies both. I have been trying this and it still does not seem to lex properly. Am I still doing something wrong? This is with ANTLR 3.0.1

Carter.

DOTDOT	: '..'
	;


INT	

	: Digit+ ( ('..') =>
        | '.' Digit* { $type = FLOAT; } )?

	| ( '0' ('x'|'X') ) => '0' ('x'|'X') (Digit|'a'..'f'|'A'..'F')+
        | ('.' Digit+) => '.' Digit+ { $type = FLOAT; }
	;

--- On Sun, 8/3/08, Carter Cheng <carter_cheng at yahoo.com> wrote:

> From: Carter Cheng <carter_cheng at yahoo.com>
> Subject: Re: [antlr-interest] Lexing problem I cannot resolve
> To: antlr-interest at antlr.org
> Date: Sunday, August 3, 2008, 12:28 PM
> How would you declare the FLOAT rule in this case? 
> 
> 
> --- On Sun, 8/3/08, Gavin Lambert
> <antlr at mirality.co.nz> wrote:
> 
> > From: Gavin Lambert <antlr at mirality.co.nz>
> > Subject: Re: [antlr-interest] Lexing problem I cannot
> resolve
> > To: carter_cheng at yahoo.com, antlr-interest at antlr.org
> > Date: Sunday, August 3, 2008, 3:50 AM
> > At 22:16 3/08/2008, Carter Cheng wrote:
> >  >1..2
> >  >
> >  >Which the lexer seems to like to lex as two
> FLOATS as
> > oppose to 
> > as
> >  >INT RANGE INT. In the language in question FLOAT
> FLOAT
> > is 
> > illegal
> >  >but obviously the lexer cannot know that. Is
> there a
> > way to 
> > resolve
> >  >this in ANTLR cleanly?
> > 
> > Presumably it's splitting it up into
> > FLOAT["1."] FLOAT[".2"]?
> > 
> > For starters, you could declare the former one to be
> an
> > illegal 
> > FLOAT -- after all it's a bit odd to have a
> trailing
> > period with 
> > no following digits.
> > 
> > But whether you choose to make that illegal or not
> (and you
> > don't 
> > *have* to), you'll need to modify the FLOAT rule
> to
> > look ahead, 
> > see two periods, and exit without matching either.
> > 
> > Something along these lines ought to do the trick:
> > 
> > fragment DIGIT: '0'..'9';
> > RANGE: '..';
> > INT
> >    : DIGIT+
> >      ( ('..') => /* RANGE; ignore */
> >      | '.' DIGIT* { $type = FLOAT; }
> >      )?
> >    | ('.' DIGIT) => '.' DIGIT+ {
> $type =
> > FLOAT; }
> >    ;
> > 
> > (If you want to make "1." illegal, then
> changing
> > DIGIT* to DIGIT+ 
> > on the sixth line ought to do the trick.)
> > 
> > You *might* need to merge the RANGE rule into the INT
> rule
> > as 
> > well, but I think the above will work ok as is.