[antlr-interest] @after in lexer rule

Hartmuth Gutsche hgutsche at storm.ca
Mon Sep 8 20:32:28 PDT 2008


I have a lexer grammar (not written by me) that worked fine in ANTLR 3.0,
but shows problems with ANTLR 3.1. I meanwhile boiled it down to
assignments of the type $channel=HIDDEN used in @after actions in the lexer
rules. An example is:


T_EOS
@after {
// if the previous token was a T_EOS, then the one we're
// processing now is whitespace, so throw it away.
// also, if the previous token is null it means we have a
// blank line or a semicolon at the start of the file and
// we need to ignore it.
if(prevToken == null ||
(prevToken != null && prevToken.getType() == T_EOS)) {
$channel=HIDDEN;
}
if(includeLine) {
$channel=HIDDEN;
// Part of include file handling..
includeFile();
includeLine = false;
}
// Make sure we clear the flag saying we're in a format-stmt
inFormat = false;
}
: ';'
| ('\r')? ('\n')
;

What happens now is that for mT_EOS() in the generated lexer the following
statements are generated:
...

state.type = _type;
state.channel = _channel;

// if the previous token was a T_EOS, then the one we're
// processing now is whitespace, so throw it away.
// also, if the previous token is null it means we have a
// blank line or a semicolon at the start of the file and
// we need to ignore it.
if(prevToken == null ||
(prevToken != null && prevToken.getType() == T_EOS)) {
_channel=HIDDEN;
}
if(includeLine) {
_channel=HIDDEN;
// Part of include file handling..
includeFile();
includeLine = false;
}
// Make sure we clear the flag saying we're in a format-stmt
inFormat = false;
}
finally {
}
}
// $ANTLR end "T_EOS"

I.e. the state class doesn't get modified with the values assigned by the
@after action.

Because I use ANTLR only occasionally my question here is if I have to
change the grammar ( I am aware of changes with respect to $channel) or if
that is a bug?


Thanks!

Hartmuth




More information about the antlr-interest mailing list