[antlr-interest] Matching tokens only at certain places

[Terence Parr]

On Jun 19, 2006, at 7:13 AM, Emond Papegaaij wrote:
> I've tried to replace
>   METHOD_SIG_ACTION: (~';')+ ;
> with
>   METHOD_SIG_ACTION: {sig}?=> (~';')+ ;
> and set 'sig' to 'true' when the token is valid. However the  
> predicate does
> not have the desired effect. In stead of disabling the token, it  
> disables all
> paths the token could match. This is what the resulting s0 DFA  
> state in the
> generated code looks like:

Hi Emond,

Sending feedback from the parser to the lexer is generally not a good  
idea.  In v3, for example, all tokens are consumed and built before  
the parser even starts...this is only the default of course.  Hence,  
your pred will not work.  Try having the lexer set "sig" when it sees  
a construct of interest.

> public DFA.State transition(IntStream input) throws  
> RecognitionException {
>   int LA3_0 = input.LA(1);
>   if ( LA3_0=='{' && (sig)) {return s1;}
>   if ( LA3_0=='}' && (sig)) {return s2;}
>   if ( LA3_0=='i' && (sig)) {return s3;}
>   if ( LA3_0==';' ) {return s4;}
>   if ( (..)||(..)||(..) && (sig)) {return s5;}
>   if ( (..)||(..)||LA3_0==' ' && (sig)) {return s6;}
>   if ( (..)||..||(..)||(..)||(..)||(..)||..||(..) && (sig)) {return  
> s7;}
>   NoViableAltException nvae =
>     new NoViableAltException("", 3, 0, input);
>   throw nvae;
> }
>
> It is clear that this disables all paths except "LA3_0==';'" when  
> 'sig' is
> false. As a result the lexer will only except ';' tokens as long as  
> 'sig' is
> false. Am I using the {..}?=> predicates incorrectly?

Well, not sure about your intention, but you have told antlr to only  
turn on that METHOD_SIG_ACTION rule when sig is true.  When input is  
';' that rule will not match no matter what, right?  It's ~';'.  That  
means another rule will match ';'.

Ter