[antlr-interest] Collecting chars in the lexer

[Ric Klaren]

On 10/23/05, Jim Crafton <jim.crafton at gmail.com> wrote:
> I have the following rule in my lexer
>
> Pragma
>         :       ('#' "pragma" (~('\r' | '\n'))* EndOfLine)
>                 {_ttype = ANTLR_USE_NAMESPACE(antlr)Token::SKIP; newline();}
>         ;
>
> As I understand it, this means the lexer will skip everything till the
> first newline char. How would I collect this text anyways, for
> parsing/analysis done by some other method?

The preserveWhitespace example might be interesting to look at. It
details how to make skipped whitespace available at later stages, the
same could also be applied to this.

Another approach would be to create a tokenstream filter it depends a
little on what you want to do at later stages.

Cheers,

Ric