[antlr-interest] Using ! operator in lexer rules

Wincent Colaiuta win at wincent.com
Wed Apr 18 23:56:04 PDT 2007


This[1] wonderful ANTLR 2 tutorial shows the use of a ! operator in  
lexer rules to omit characters from a token's text value:

CHARLIT : '\''! . '\''! ; // enclosing quotes omitted

I've tried this in ANTLR 3 and the operator seems to have no effect  
at all. The new Definitive ANTLR Reference shows many examples using  
the operator, but only in the context of parser rules which output  
AST nodes.

As an example, I have an escaped backslash token. I want it to  
recognize the \\ sequence in the input stream and emit a token whose  
text value is just \. The only way I've found to do this is to  
manually invoke setText from within an action:

ESCAPED_BACKSLASH : BACKSLASH BACKSLASH { setText("\\"); };

The simpler alternative doesn't work:

ESCAPED_BACKSLASH : BACKSLASH! BACKSLASH;

Likewise for literals:

ESCAPED_BACKSLASH : '\\'! '\\';

So is it correct that the ! operator no longer works in lexer rules  
in ANTLR 3, or am I missing something here?

Cheers,
Wincent

[1] <http://javadude.com/articles/antlrtut/>



More information about the antlr-interest mailing list