[antlr-interest] Possible serious bug using setText in Antlr3

[Robert Hill]

Lol, this is partly related to my problem.
I altered the lexer to strip off some characters, - using settext - and
that's why I'd altered the BLOCKCODE/ LINECODE rules to just make a template
from the token.. LOL, didn't realise that wasn't working either!

I still cant figure out why my LINECODE rule won't re-write the template
though :(


> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Terence Parr
> Sent: 15 November 2006 16:56
> To: ANTLR Interest
> Subject: Re: [antlr-interest] Possible serious bug using setText in Antlr3
> 
> 
> On Nov 14, 2006, at 10:16 PM, shmuel siegel wrote:
> 
> > The wiki page, "how do I strip quotes", recommends using setText to
> > change the text of the token to be generated. For example
> >
> > ESC : '\\' 'n' {setText("\n");};
> >
> > As far as I can tell, this sets an instance variable of the lexer
> > to the specified text. Emit then adds this text to generated
> > tokens. getText will then preferentially return this text rather
> > than build a string from the character stream. So far, so good. But
> > nothing changes the instance variable back to null. So all future
> > tokens will have the same text.
> >
> > setting text=null in nextToken seems to work for me.
> 
> Yes, you are right.  I have that fix already in 3.0b5 coming out
> today :)
> 
> Ter