[antlr-interest] Fragments in parser rules - generate error?

[Kevin J. Cummings]

On 06/02/2010 12:45 PM, Ken Williams wrote:
> Hi,
> 
> I have the following toy grammar:
> 
> -------------------------
> date    :    DIGIT+ SLASH DIGIT+ SLASH DIGIT+ ;
> 
> SLASH    :    '/' ;
> DIGITS    :    DIGIT+ ;
> fragment DIGIT    : '0'..'9' ;
> -------------------------
> 
> (No options{} declarations etc.)
> 
> I know I shouldn't be using fragments in this way, but it was an accident -
> 'DIGIT' used to be a lexer rule, but then I changed it into a fragment,
> added DIGITS as a new lexer rule, and forgot to remove DIGIT+ from the
> parser rule.
> 
> The problem I encountered was that even though the grammar check succeeded
> and the lexer/parser classes were successfully generated, the 'date' rule
> doesn't match the intended input.

How can it?  When the lexer runs (before the parser, and without any
knowledge of the parse state) when it finds DIGIT+, it creates a DIGITS
token.  This is your problem, not the fragment DIGIT.  What happens when
you change your date rule to:

date : DIGITS SLASH DIGITS SLASH DIGITS
     ;

?

Does it work then?

> It seems like the right solution would be for Antlr to generate an error
> when a fragment is used in a parser rule.
> 
> Or is there a reason I'm not thinking of to allow them?
> 
> Thanks.

-- 
Kevin J. Cummings
kjchome at rcn.com
cummings at kjchome.homeip.net
cummings at kjc386.framingham.ma.us
Registered Linux User #1232 (http://counter.li.org)