[antlr-interest] Can antlr v3 lex star | tristar properly?

[Guntis Ozols]

The real life example is actually:

DCOLON                :        '::';
NS_TEST                :        NCName ':*';
PrefixedName        :        NCName ':' NCName;
NCName                :        ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z'
| '.' | '-' | '_' |
'0'..'9')*;

> As Johannes pointed out, you can do manual left-factoring - but this somehow
> defies the reason to use ANTLR ...

And it even does not work...

> A "somewhat less intrusive" way, which avoid the left-factoring, is the
> following:
>
> stars        :        (STAR | TRISTAR)* EOF;
> STAR : '*' ;
> TRISTAR :
>    {input.LA(1)=='*' && input.LA(2)=='*' && input.LA(3)=='*'}? =>
>    STAR STAR STAR;
>
> {...}? is a semantic predicate; and the => forces checking the predicate as
> with syntactic predicates.
>
> This way, you could keep any actions inside your grammar - at times a
> nuisance with left-factoring and a primary reason to use some tool to
> disambiguate input ...
>
> Regards
> Harald
>
> -------- Original-Nachricht --------
> > Datum: Wed, 21 Nov 2007 12:54:31 +0200
> > Von: Guntis Ozols <guntiso at latnet.lv>
> > An: antlr-interest at antlr.org
> > Betreff: [antlr-interest] Can antlr v3 lex star | tristar properly?
>
> > Is there a way to lex this simple grammar (I am using ANTLRWorks 1.1.4):
> >
> > stars        :        (STAR | TRISTAR)* EOF;
> > STAR : '*' ;
> > TRISTAR : STAR STAR STAR;
> >
> > For '**' I keep getting: mismatched character '<EOF>' expecting '*'.
> >
> > I am out of options. Below is what I've tried:
> > //tokens { STAR; TRISTAR;}
> > //options { backtrack = true; }
> > //options { k = 3; }
> > //ST        :        ('***') => '***' {type = TRISTAR;}
> > //        |        '*' {type = STAR;};
> > //STAR options {greedy=false;}: '*' ;
> > //TRISTAR : '***';
> > //TRISTAR : ('***')=>'***';
> > //TRISTAR options { greedy=true; } : '***';
> > //TRISTAR options { greedy=true; } : ('***')=>'***';
> > //TRISTAR options {greedy=false;} : '***';
> >
> > Guntis