[antlr-interest] Lexer question

[Scott Smith]

That's what I have now.  It does work.  I was just hoping to differentiate it in my tree grammar so I wanted "NAME" to have a different token.

-----Original Message-----
From: John B. Brodie [mailto:jbb at acm.org] 
Sent: Tuesday, August 23, 2011 3:58 PM
To: Scott Smith
Cc: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Lexer question

try

rule1: IDENTIFIER ':' expression ;

that is why bother differentiating the two tokens?

On Tue, 2011-08-23 at 21:19 +0000, Scott Smith wrote:
> I have a parser that is doing pretty much what I want.  However, I want to do the following.
> 
> I have a definition for an IDENTIFIER
> 
> IDENTIFIER: LETTER (LETTER | NUMBER)                // LETTER and NUMBER mean the usual thing
> 
> Now in some of my rules, I'm looking for an IDENTIFIER and in one of my rules I look for a NAME.  NAME has exactly the same definition as IDENTIFIER (starts with a letter followed by alphanumerics).  However, you can tell by the token after whether it was a NAME or an IDENTIFIER.  To be more explicit, a NAME is ALWAYS followed by a colon.  An IDENTIFIER can be followed by a number of things, but NEVER by a colon.
> 
> So, I have rules that looksomething like:
> 
> rule1:
>                 NAME ':' expression
>                 ;
> 
> rule2:
>                 IDENTIFIER '+' expression
>       |       IDENTIFIER  '-' expression
>      ;
> 
> I don't seem to be able to make this work.  Can someone suggest a solution?  Do I have to turn on backtracking to make this work?
>