[antlr-interest] Parsing keyword vs symbol

[Peter Kronenberg]

Bryan,
   thanks for your response.  I'm trying to understand the best way to
use the tool.  It's not always straight forward.

If I understand you correctly, I need two different tokens: one, defined
in the lexer, to represent the symbol (e.g., EQ1: '=') and another,
defined in the parser, to represent the keyword (tokens: EQ2="EQ".  And
then, in the parser, I would need to test for all posibilities, e.g.
  relationalExpr: term (EQ1 | EQ2 | LT1 | LT2 | GT1 | GT2 ...) term

Is this correct? 

Peter Kronenberg
Software Engineer
(703) 885-1222
pkronenberg at technicacorp.com

-----Original Message-----
From: Bryan Ewbank [mailto:ewbank at gmail.com] 
Sent: Wednesday, April 06, 2005 4:00 PM
To: Peter Kronenberg
Subject: Re: [antlr-interest] RE: antlr-interest Digest, Vol 5, Issue 11

It /is/ a lexer rule.  Since "eq" looks like an /identifier/, it's
easiest to assume it's an identier and then correct that assumption
with a lookup table, as previously suggested.

You can certainly do what you like with the tools, but the facility
provided for keyword recognition (things like "for", "while",
"integer", etc) is exactly the facility that fits your needs here for
the keyword "eq":
   IDENT options {testLiterals=true;} ...
   // i.e., report the appropriate literal
   // if any is found (e.g., "eq" => EQ, "for" => FOR),
   // otherwise report that an IDENT was found

At another level, consider that error messages from the parser may
actually care whether it was "eq" or "=" - imagine the user's
confusion if you reported an error at a nonexistent token.

Another alternative, if you'd rather, is to run it thru cpp with:
   #define eq =
   #define ne !=
   ...

Hope this helps,
- Bryan

On Apr 6, 2005 1:27 PM, Peter Kronenberg <PKronenberg at technicacorp.com>
wrote:
> Well, I think it makes sense for it to be a lexer rule, since the
parser
> shouldn't care or need to know the exact for of the token, either EQ
or
> '='.

The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Technica Corporation does not represent this e-mail to be free from any virus, fault or defect and it is therefore the responsibility of the recipient to first scan it for viruses, faults and defects. To reply to our e-mail administrator directly, please send an e-mail to postmaster at technicacorp.com. Thank you.