[antlr-interest] Using ! operator in lexer rules
Wincent Colaiuta
win at wincent.com
Wed Apr 18 23:56:04 PDT 2007
This[1] wonderful ANTLR 2 tutorial shows the use of a ! operator in
lexer rules to omit characters from a token's text value:
CHARLIT : '\''! . '\''! ; // enclosing quotes omitted
I've tried this in ANTLR 3 and the operator seems to have no effect
at all. The new Definitive ANTLR Reference shows many examples using
the operator, but only in the context of parser rules which output
AST nodes.
As an example, I have an escaped backslash token. I want it to
recognize the \\ sequence in the input stream and emit a token whose
text value is just \. The only way I've found to do this is to
manually invoke setText from within an action:
ESCAPED_BACKSLASH : BACKSLASH BACKSLASH { setText("\\"); };
The simpler alternative doesn't work:
ESCAPED_BACKSLASH : BACKSLASH! BACKSLASH;
Likewise for literals:
ESCAPED_BACKSLASH : '\\'! '\\';
So is it correct that the ! operator no longer works in lexer rules
in ANTLR 3, or am I missing something here?
Cheers,
Wincent
[1] <http://javadude.com/articles/antlrtut/>
More information about the antlr-interest
mailing list