[antlr-interest] Matching tokens on hidden channels
Zenzike
zenzike at gmail.com
Thu Jan 22 01:45:35 PST 2009
I am currently trying to match tokens on hidden channels -- this is
sometimes needed when a token is hidden, since generally, these tokens
should not be considered by the parser, but on occasion they should.
I saw a brief mention of "hidden token matching" in a wiki document
that Ter wrote [1], where the syntax:
.TOKEN
matches a particular hidden token TOKEN, and this is indicated by the
"." that precedes TOKEN. It seems that this functionality is not in
ANTLR 3.1.1 yet -- are there any plans to include it?
In order to try and solve this for what I have now, I've tried to use
semantic predicates along these lines:
rule0 : PREV ({NL()}?=> NEXT) ;
where PREV and NEXT are tokens that I want to separate with a hidden
NL token, and NL() is defined as such:
boolean NL() {
int i = -1;
while (input.LA(i) != Token.EOF && input.LT(-1).getChannel() == HIDDEN) {
if (input.LA(i) == NL)
return true;
i--;
}
return false;
}
This works fine if NEXT is a token, but when it's a rule, I get an
error. For example, where I have:
rule1 : PREV ({NL()}?=> rule2);
I get the following error if I compile with the -debug flag (no error
otherwise):
ANTLR Parser Generator Version 3.1.1
error(10): internal error: AztexParser.g :
java.util.NoSuchElementException: no such attribute: description in
template context [outputFile parser genericParser(...) cyclicDFA
if(dfa.specialStateSTs)_subtemplate anonymous cyclicDFAState
cyclicDFAEdge notPredicate evalPredicate(...)]
org.antlr.stringtemplate.StringTemplate.checkNullAttributeAgainstFormalArguments(StringTemplate.java:1276)
org.antlr.stringtemplate.StringTemplate.get(StringTemplate.java:800)
Is this a bug in my code, or is it a problem with debug mode? Is there
another way of achieving what I'm looking for?
Any help would be greatly appreciated.
zenzike
[1] http://www.antlr.org/wiki/display/~admin/2007/06/28/Token+stream+rewriting+with+rewrite+rules
More information about the antlr-interest
mailing list