[antlr-interest] Is '!' operator working with ANTLR 3.1.2

[Jim Idle]

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Colin Macdonald
> Sent: Thursday, November 18, 2010 9:03 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Is '!' operator working with ANTLR 3.1.2
> 
> OK, checked the reference last night (page 98) and ! is definitely
> parser only
> 
> Jim's got a good point about white space, so you need to go back to
> your original lexer rule and hack the text.
> 
> The easiest option would be to always trim out the "EXT." by calling
> something like
> 
>         $EXTERNAL_CALL.text.substring(4)
> 
> I don't suggest messing around with start & stop.  You don't have
> access to the Token in the lexer actions, 

Yes, you DO have access to the token - it isn't created until the rule
exits, so you set the things that will be used by the lexer.


besides it would be fragile
> unless you extend CommonToken to track if you've made the change
> already.

No, you don't need to do that, you change it in the lexer actions and then
never touch it again. From the top of my head (so check it):

K : 'CCCC'
     { $start 		= getCharIndex(); }
     'DDD' 'A'*
     etc


However, it is probably just as easy to use a substring when you reference
the token text, which you will generally only do once anyway.

Jim