[antlr-interest] Re: more lexical determinism

[Sinan]

howardckatz wrote:
> 
> This has been an interesting exercise. I can see that this particular
> problem -- where two tokens consist of closely overlapping character
> sets -- is one that antlr doesn't handle that well. I can see one
> other approach that might work -- sticking some string-parsing Java
> code of my own either into the parser grammar or maybe in a
> downstream TokenStream. Time to play I guess ...
> 

yacc/lex won't either.

what you should do really is 
assume a could have '_' and b can't.

 then you really have something like

 rule : (a | b) COLON b;

so in lexer you say

B : ( LETTER | DIGIT | '_' { set type to A})+;


in parser

rule: (A | B) COLON B;

or pushing into other rules

rule : id COLON word ;

id : A | B;
word : B;

Sinan

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/