[antlr-interest] multi-channel token stream

Michael Bedward michael.bedward at gmail.com
Mon May 30 19:30:47 PDT 2011


Hello,

This message is just to update the URL to the MultiChannelTokenStream
class mentioned in the message below from some months ago. The current
URL is:

http://jiffle.googlecode.com/svn/trunk/language/src/main/java/jaitools/jiffle/parser/MultiChannelTokenStream.java

Please feel free to do whatever you like with it, although I'd
appreciate hearing about any bugs or ways of improving it.
Alternatively, if it's entirely useless and there are much better ways
of handling multiple channels I'd like to hear about that too :)

Michael


On 10 February 2011 10:59, Michael Bedward <michael.bedward at gmail.com> wrote:
> Hi folks,
>
> I'd like my parser to be able to tune in to more than one token
> channel with the choice of channel(s) being set when constructing the
> TokenStream. The DAR book doesn't have an example of doing this,
> unless I've missed it. The wiki refers to being able to merge channels
> but I couldn't find an example of doing this - unless it just meant
> using BufferedTokenStream instead of CommonTokenStream ?
>
> My solution has been to create a simple class,
> MultiChannelTokenStream, adapted from CommonTokenStream. Instead of
> holding the index of a single channel it maintains a list of "active
> channels" which is used by the various stream access and positioning
> methods.  The source is here:
>
> http://code.google.com/p/jai-tools/source/browse/trunk/jiffle/src/main/java/jaitools/jiffle/parser/MultiChannelTokenStream.java
>
> I still have a lurking doubt that I'm re-inventing the wheel here, ie.
> that I must be missing something obvious in the ANTLR API (or perhaps
> more fundamentally in lexer / parser design). If that's the case I'd
> appreciate any tips. On the other hand, if the class is of any use to
> others, please feel free to grab it.
>
> Michael
>


More information about the antlr-interest mailing list