[antlr-interest] Re: more lexical determinism

Sinan sinan.karasu at boeing.com
Thu Dec 6 10:04:01 PST 2001

howardckatz wrote:
> This has been an interesting exercise. I can see that this particular
> problem -- where two tokens consist of closely overlapping character
> sets -- is one that antlr doesn't handle that well. I can see one
> other approach that might work -- sticking some string-parsing Java
> code of my own either into the parser grammar or maybe in a
> downstream TokenStream. Time to play I guess ...

yacc/lex won't either.

what you should do really is 
assume a could have '_' and b can't.

 then you really have something like

 rule : (a | b) COLON b;

so in lexer you say

B : ( LETTER | DIGIT | '_' { set type to A})+;

in parser

rule: (A | B) COLON B;

or pushing into other rules

rule : id COLON word ;

id : A | B;
word : B;



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 

More information about the antlr-interest mailing list