[antlr-interest] Lexer question

[Jim Idle]

Because that allows ID    : as well as ID: so you have to check that the
start position of the ':' is consistent with the end position of ID. I
prefer this approach when an error message such as "A colon for a name
must follow immediately after the name without any whitespace or something
worded better than this" will make more sense than "Syntax error, expected
NAME, found ID"

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of John B. Brodie
> Sent: Tuesday, August 23, 2011 2:58 PM
> To: Scott Smith
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Lexer question
>
> try
>
> rule1: IDENTIFIER ':' expression ;
>
> that is why bother differentiating the two tokens?
>
> On Tue, 2011-08-23 at 21:19 +0000, Scott Smith wrote:
> > I have a parser that is doing pretty much what I want.  However, I
> want to do the following.
> >
> > I have a definition for an IDENTIFIER
> >
> > IDENTIFIER: LETTER (LETTER | NUMBER)                // LETTER and
> NUMBER mean the usual thing
> >
> > Now in some of my rules, I'm looking for an IDENTIFIER and in one of
> my rules I look for a NAME.  NAME has exactly the same definition as
> IDENTIFIER (starts with a letter followed by alphanumerics).  However,
> you can tell by the token after whether it was a NAME or an IDENTIFIER.
> To be more explicit, a NAME is ALWAYS followed by a colon.  An
> IDENTIFIER can be followed by a number of things, but NEVER by a colon.
> >
> > So, I have rules that looksomething like:
> >
> > rule1:
> >                 NAME ':' expression
> >                 ;
> >
> > rule2:
> >                 IDENTIFIER '+' expression
> >       |       IDENTIFIER  '-' expression
> >      ;
> >
> > I don't seem to be able to make this work.  Can someone suggest a
> solution?  Do I have to turn on backtracking to make this work?
> >
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address