[antlr-interest] Weird (to me) grammar problem

[David-Sarah Hopwood]

Evan Driscoll wrote:
> I just started work on a grammar to read well, context free grammars,
> and am running into a problem. (I'm probably just doing something dumb.)
> I've attached my grammar.
> 
> The ARROW token (used between the left and right sides of a production)
> should recognize either ':' or '->', but the AntlrWorks interpreter only
> accepts '->'. If I try to parse the input 'a -> b;', I get the proper
> result. If I try to parse 'a : b;', it gives a MismatchedTokenException.
> (I am pretty sure I saw the same behavior using the debug option, but I
> don't have the JDK on this computer and can't confirm it.)
> 
> The rules in question are:
> 
>   COLON : ':'; // used in multiple places
> 
>   ARROW	
>       : '->'
>       | COLON
>       ;
> 
>   production
>       : SYMBOL ARROW disjunction SEMICOLON
>       ;

Since COLON and ARROW are lexer rules and COLON appears first, ':' will
always match COLON and never ARROW. It can be fixed by changing ARROW
to a parser rule:

COLON  : ':';
RARROW : '->';

arrow
    : RARROW
    | COLON
    ;

production
    : SYMBOL arrow disjunction SEMICOLON
    ;

(It's not the use of string literals vs token rules that is significant
here; just whether arrow is a lexer or parser rule. This seems to be
one of the most common mistakes made by people new to ANTLR.)

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20091202/537ab578/attachment.bin