[antlr-interest] Remembering Skipped Tokens

[Johannes Luber]

jason heddings wrote:
> Hello again-
>  
> If I can get this one, I think I'm up and running...
>  
> I have set up a token in my lexer for matching whitespace, and set the
> type to Token.SKIP to make the parser easier to write.  However, in
> order to check proper syle, I need to keep whitespace information in my
> resulting AST.  Is there an easy way to do this, so I don't have to put
> (WS)* in every parser rule I write?
>  
> I'm enjoying my new challenge (creating a flexible style checker).  I
> hope to be able to share my results sometime, and maybe you all can
> provide some suggestions for improvements.
>  
> --jah

Hello,

if you are are using ANTLR 3, then the following quote from the PDF
seems to be describing your situation:

"One of the most difficult lexing issues to deal with is the paradox
that the parser must ignore whitespace and comments, but at the same
time provide access to those tokens for use during translation. To solve
this problem, ANTLR allows each token object to exist on different
channels, sort of like different radio frequencies. The parser can
“tune” to any single channel and, hence, it ignores any off-channel tokens."

But I haven't got to the part where such hidden tokens are read, so I
can't help you further.

Best regards,
Johannes Luber