[antlr-interest] skipOffChannelTokens in CommonTokenStream
Martin Probst
mail at martin-probst.com
Wed Apr 2 03:08:28 PDT 2008
Hi,
I'm running into a bug where the parser will see and analyze tokens
that are on a hidden channel, thus failing with noviablealt
exceptions. I think the culprit is this code in
CommonTokenStream.LT(int), around line 250.
int i = p;
int n = 1;
// find k good tokens
while ( n<k ) {
// skip off-channel tokens
i = skipOffTokenChannels(i+1); // leave p on valid token
n++;
}
Where k is the desired lookahead, p is the current position in the
token stream and i is the position of the actually asked token. If k
== 1 the while loop will never execute. So if the token at k (at 1) is
a $channel == HIDDEN token, it will still be returned, even if it
should be ignored.
I think changing the condition to "while ( n<=k )" and changing
"skipOffTokenChannels(i+1)" to "skipOffTokenChannels(i)" should fix
the issue.
Regards,
Martin
More information about the antlr-interest
mailing list