[antlr-interest] Fragments in parser rules - generate error?

[Jim Idle]

Ken was commenting on the fact that a fragment was mentioned in the parser grammar but no error was given out by ANTLR about this; there should be as fragment tokens cannot be sent to the parser. Hopefully v4 will address that one :-)

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Kevin J. Cummings
> Sent: Wednesday, June 02, 2010 11:49 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Fragments in parser rules - generate
> error?
> 
> On 06/02/2010 12:45 PM, Ken Williams wrote:
> > Hi,
> >
> > I have the following toy grammar:
> >
> > -------------------------
> > date    :    DIGIT+ SLASH DIGIT+ SLASH DIGIT+ ;
> >
> > SLASH    :    '/' ;
> > DIGITS    :    DIGIT+ ;
> > fragment DIGIT    : '0'..'9' ;
> > -------------------------
> >
> > (No options{} declarations etc.)
> >
> > I know I shouldn't be using fragments in this way, but it was an
> accident -
> > 'DIGIT' used to be a lexer rule, but then I changed it into a
> fragment,
> > added DIGITS as a new lexer rule, and forgot to remove DIGIT+ from
> the
> > parser rule.
> >
> > The problem I encountered was that even though the grammar check
> succeeded
> > and the lexer/parser classes were successfully generated, the 'date'
> rule
> > doesn't match the intended input.
> 
> How can it?  When the lexer runs (before the parser, and without any
> knowledge of the parse state) when it finds DIGIT+, it creates a DIGITS
> token.  This is your problem, not the fragment DIGIT.  What happens
> when
> you change your date rule to:
> 
> date : DIGITS SLASH DIGITS SLASH DIGITS
>      ;
> 
> ?
> 
> Does it work then?
> 
> > It seems like the right solution would be for Antlr to generate an
> error
> > when a fragment is used in a parser rule.
> >
> > Or is there a reason I'm not thinking of to allow them?
> >
> > Thanks.
> 
> --
> Kevin J. Cummings
> kjchome at rcn.com
> cummings at kjchome.homeip.net
> cummings at kjc386.framingham.ma.us
> Registered Linux User #1232 (http://counter.li.org)
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address