[antlr-interest] Is it safe to overwrite the Lexer's text?

Diehl, Matthew J matthew.j.diehl at intel.com
Tue Aug 28 07:37:45 PDT 2007


> I think the current channel handling is something of a work in
> progress. Currently the channel support is rather basic.
> CommonTokenStream only allows a parser to read from one channel and
> there doesn't appear to be any direct way to access off-channel
> tokens.

Yeah, it's not too 'user-friendly' I guess, lol.  This is what I did.  I
stored all of my comments (I needed different things on different
channels) on channel 4.  I sent this function/class my 'input', then:

if (input == null) return;
TokenStream tok = input.getTokenStream();
int n = tok.size();
Token t;
allComments = new HashMap<Integer,String>();//my class private variable
//then you just run through the entire document and store each comment
in the map with it's token number
for (int i = 0; i < n; i++){
  t = (token)tok.get(i);
  if ( t.getChannel()==4)
    allComments.put(I,t.getText());
}

Then once it's all stored the way I would re-associate it with my code
was by the token number.  If I was parsing a particular subtree, I would
collect all of the comments up until the first token of the subtree, and
then print them out before the subtree prints out.  (well, that's an
over-simplified description of how I associate it, but I'm sure you have
some clever ideas.)

Good luck, hope this helped,

Matt


More information about the antlr-interest mailing list