[antlr-interest] Is it safe to overwrite the Lexer's text?

Bjoern Doebel doebel at tudos.org
Tue Aug 28 05:11:31 PDT 2007


Hi,

I have a language consisting of source code and comments. I have a lexer
and parser that create an AST of it. I would like to use the created syntax
tree as a starting point for several tools. Some of them will need access
to the comments, some of them won't. I cannot just ignore the comments, but
I also don't want to store all of them inside the AST. Instead, inside the
lexer I recognize comments, copy them (current value of self.text) into a
separate table and replace the comment's text by overwriting self.text with
an index referencing the comment's table entry.

This works for now and my question is: Is this safe or does overwriting
self.text have any side effects that I did not realize up to now and that
might lead to problems later on?

Regards,
Bjoern


More information about the antlr-interest mailing list