[antlr-interest] [Antlr 3] lexer problem

[Sven Efftinge]

Alex,

>
> For this to happen lexer should know what the parser expects.

I think it has nothing to do with the parser.

> And I do not
> think that is how lexer works. It knows 0 about parser. The parser  
> knows 0
> about lexer it only cares about token stream. In my understanding  
> tracking
> back can happen only in either one of them. But not across their  
> boundaries
> (unless you do it yourself of course)

Yes, that's right but it doesn't relate to my problem.
I'm talking about the lexer only.

>
> So in your case you could disambiguate your lexer by using  
> syntactic predicates

The grammar results in something like the following lexer rules which  
are not ambigous, are they?

DOT 	:	'.';

FOO 	: 	'foo';

FLOAT 	:	'0'..'9'+ (DOT '0'..'9'+)?;


> I guess. I have never used  them on lexers but I guess if it works
> similarly to how they
> work in parsers, it should look something like this:
>
> INTORFLOAT:
>  ('0'..'9'+ '.' '0'..'9') => FLOAT
> | INT;
>
> FLOAT  : '0'..'9'+ ('.' '0'..'9'+)?;
> INT       : '0'..'9'+;

I've tried syntactic predicates without success (same behaviour).
Anyway I would like to understand why it behaves that way (if it is  
no bug)

thanks,
Sven