[antlr-interest] multi-channel token stream

[Michael Bedward]

Hi folks,

I'd like my parser to be able to tune in to more than one token
channel with the choice of channel(s) being set when constructing the
TokenStream. The DAR book doesn't have an example of doing this,
unless I've missed it. The wiki refers to being able to merge channels
but I couldn't find an example of doing this - unless it just meant
using BufferedTokenStream instead of CommonTokenStream ?

My solution has been to create a simple class,
MultiChannelTokenStream, adapted from CommonTokenStream. Instead of
holding the index of a single channel it maintains a list of "active
channels" which is used by the various stream access and positioning
methods.  The source is here:

http://code.google.com/p/jai-tools/source/browse/trunk/jiffle/src/main/java/jaitools/jiffle/parser/MultiChannelTokenStream.java

I still have a lurking doubt that I'm re-inventing the wheel here, ie.
that I must be missing something obvious in the ANTLR API (or perhaps
more fundamentally in lexer / parser design). If that's the case I'd
appreciate any tips. On the other hand, if the class is of any use to
others, please feel free to grab it.

Michael