[antlr-interest] use channel=HIDDEN now

[Kay Roepke]

Hi Michiel!

On Oct 17, 2006, at 8:48 AM, Michiel Vermandel wrote:

> (how) can I get an AST that contains all nodes on any channel (so  
> also the
> nodes on the hidden channel)?

There are two ways to achieve that.

1) CommonTokenStream has a channelOverrideMap. It maps tokenTypes to  
channel numbers. E.g.:

Say you have whitespace tokens (type WS) on channel=HIDDEN (99 in the  
current implementation).
If you do

	CommonTokenStream tokenStream = new CommonTokenStream(lexer);
	tokenStream.channelOverrideMap.put(new Integer(YourLexerClass.WS),  
new Integer(Token.DEFAULT_CHANNEL));

anything that reads from this tokenStream will receive WS tokens on  
the default channel even though they
really are on channel HIDDEN. The tokens themselves are not altered.

2) in a subclass of CommonTokenStream override skipOffTokenChannels 
(int) and skipOffTokenChannelsReverse(int) to always return
their argument. They return the next (or previous) index on the token  
stream that contains a token to be considered.
This is more of a hack, of course, but this way you don't have to  
manually specify every single tokentype in the channelOverrideMap.

Be sure to do either one before you actually use that tokenStream.  
There is no way to "reset" the stream making it read the tokens again.
Once they are buffered up, changes to channelOverrideMap aren't going  
to effect anything.
You could of course write you own TokenStream subclass that does it  
differently. In cases you might not even want to buffer up the stream  
beforehand, as it is done now (I know someone asked this a couple of  
weeks ago...can't remember who it was).

HTH,

-k