[antlr-interest] Confused about backtracking

[Jim Idle]

Try turning backtrack mode off and see what ANTLR says about your
production. This will give you the clue that you need. You probably need
to add an explicit predicate. Clearly, you rules below are ambiguous on
the '.' as it is a separator. So it thinks you are parsing ID '.' as a
statement.

The problem is that backtrack hides ALL problems, not just ambiguities
that are inherent in the grammar, so you need to turn it off just to see
what ANTLR thinks the issue is, and also learn to use the ANTLRWorks
debugger.

In general then, you do not want an error from ANTLR when you turn off
backtrack mode unless there is absolutely no way to avoid it.

In this case, you make need to hand code a predicate that goes here:

expr : ID ({here}?=> ('.' expr) )*;

Or change around how you look for separators. Or just make the assumption
that the dot is a composite ID:

expr : ID ( ('.')=>'.' expr )*;

Or perhaps you have to explicitly tell the grammar to follow the whole
expr (which will get very slow):

expr : ID ( ('.' expr)=>'.' expr )*;

Or perhaps you can have a lexer rule like this:

fragment DOTTEDID :;
ID : ('A'..'Z')+ ( {input.LA(1)=='.'}?=> { $type = DOTTEDID; } | ) ;

expr : DOTTEDID ('.' ID)?
     | ID
     ;

In other words, you are going to have to hack it to get it to work. The
problem is not with ANTLR, but with the terrible language you are trying
to parse, which I think you are aware of. You are going to have to do
strange things to get it to work.

This grammar is never going to be able to give you good syntax error
messages, as it will retreat to the highest point in the rule chain to
give an error, so you will have to hope that the input is syntactically
correct (and shoot the original author of the syntax, so they cannot
repeat this ;) ).

Finally, don't forget to add memorize if you are using backtrack.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of franck102
> Sent: Monday, November 28, 2011 6:18 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Confused about backtracking
>
>
> Christian wrote
> >
> > Hi,
> >
> > what is the error/exception message?
> >
> > Regards,
> > Christian
> >
> >
>
> MissingTokenException at the second '=', after parsing a=b.c as an
> expression. The tail recursion on expr is causing it it seems, but
> that's a real issue for me... here is a slightly modified version with
> the recursion made explicit that has the same problem:
>
> program
>         : statement* EOF
>         ;
>
> statement
>         : ID '=' expr
>         | sep
>         ;
>
> expr : ID ( '.' expr )*;
>
> sep : ';' | '.';
>
> --
> View this message in context:
> http://antlr.1301665.n2.nabble.com/Confused-about-backtracking-
> tp7033712p7038881.html
> Sent from the ANTLR mailing list archive at Nabble.com.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address