[antlr-interest] Modifying token text - is it possible?

[Shmuel Siegel]

I'm not sure if I did it this way because of some beta version didn't
handle the "$" but in my grammar I write {setText(buffer.toString());}.

Your $x example won't work because the string for the token is taken
from the input stream (the characters between start of token and end of
token). Changes to intermediate results are irrelevant. You must
override the final token string.

-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Steve Bennett
Sent: Tuesday, November 20, 2007 2:30 PM
To: antlr-interest at antlr.org
Subject: [antlr-interest] Modifying token text - is it possible?

According to
http://www.cs.usfca.edu/~parrt/course/652/lectures/antlr-lexers.html
it ought to be possible to modify a token's text two ways:

FOO: '<'! ... ; // '<' does not become part of the token

FOO: '<' ...{ $setText (...); // $setText manually overrides the matched
text.

Neither of these constructs seems to work for me...were they perhaps
taken out or changed to something else? I see there is a setText
method on Lexer, but can't make it work: does FOO: x=TOKEN {$x...};
notation actually work?

Steve