Hi James. ack! i think reset() should call
skipOffTokenChannels()
Can you override reset to call super.reset() then skipOffTokenChannels() and see if that works? If so, i can fix for 3.4
Ter
On Jul 5, 2011, at 5:30 AM, James Reid wrote:
> Hi all,
>
> I think I found a bug but I want to be sure. I have a parser grammar that
> I run two passes on. Here are short versions of the rules.
>
>
>
> firstpass
> : (collect_matches
> | collect_labels
> | .)*
> ;
>
>
> script
> : header? matches* EOF
> ;
>
>
> When I run the code I use a CommonTokenStream and do the following
>
> CommonTokenStream tokens = new CommonTokenStream(lex);
> MyParser parser = new MyParser(tokens);
>
> parser.firstpass();
>
> tokens.reset();
>
> parser.script();
>
>
> The problem comes when I do the tokens.reset(). If the very first token is
> a comment (i.e. on the hidden channel) it is returned in parser.script() and
> throws a NoViableAlternativeException because nothing in my grammar is
> expecting a comment. To get around this I do the following...
>
> //Reset the tokens back to the begining
> tokens.reset();
>
> //For some reason after the tokens have been buffered up if a
> hidden
> //token is the first token it is returned instead getting the
> //first non-hidden token. This little hack works around that.
> if (tokens.LT(1).getChannel() == Token.HIDDEN_CHANNEL){
> tokens.consume();
> }
>
> //now we can build the AST
> r=parser.script();
>
> If I consume the hidden token then the token stream points at the next
> on-channel token like it is supposed to do. Is this a bug or am I doing
> things wrong?
>
> Thanks,
> James
>
> PS: This is with Antlr 3.3
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address