[antlr-interest] [v3] not including text in token. Still possible?

[Terence Parr]

On Feb 5, 2006, at 7:54 PM, Kay Roepke wrote:

> Hi!
>
> Am I mistaken, have I missed anything, or am I plain stupid? (Quite  
> a possibility...;))
>
> In v2 one could suffix a literal or tokenref with a '!' and keep  
> that (token's) text from being included in the token, like this:

Hi!  I haven't figured out to make it do that for v3 yet.  I don't  
create strings for a token (just indexes into the char buffer) so  
it's hard to do modifications.  I definitely need it though.

> Also, is there a way to get back the behavior of EA7 when it comes  
> to printing the tokens of a CommonTokenStream? It used
> to show a lot of extra information about the tokens. A first glance  
> at CommonTokenStream.java didn't reveal the secret to me :(

I think it's toDebugString or something...

Ter