[antlr-interest] Discarding characters in lexer

[Andrey Timoshenkov]

Thank you for the reply, now it is clear what I have to do.

Gavin Lambert wrote:
> At 04:02 28/03/2008, Andrey Timoshenkov wrote:
> >I have a possibly quick question about the lexical analyzer. In
> >version 2 there was a '!' operator that allowed to discard
> >characters from token text like this:
> >
> >STRING: '"'! ('"' '"'! | ~('"') )* '"'! ;
> >
> >It seems that it has gone in version 3. Is it possible to achieve
> >the same result in version 3?
>
> In v3, the ! operator only works in the parser (and affects AST 
> generation).  You can't use it in the lexer any more.
>
> To achieve the result you want now, you'll need to call $setText at 
> the end of the rule (which means you'll have to rescan the string 
> outside of ANTLR to convert escape sequences and the like).  Bit of a 
> hassle, I agree, but that's the only option for now.
>
>