[antlr-interest] Think I found a bug.

James Reid james1018 at gmail.com
Wed Jul 6 16:15:59 PDT 2011


This worked...

@Override
    public void reset() {
        super.reset();
        p = skipOffTokenChannels(0);
    }

Glad I could contribute in some small way.

James

On Wed, Jul 6, 2011 at 1:57 PM, Terence Parr <parrt at cs.usfca.edu> wrote:

> Hi James. ack! i think reset() should call
>
> skipOffTokenChannels()
>
> Can you override reset to call super.reset() then skipOffTokenChannels()
> and see if that works?  If so, i can fix for 3.4
> Ter
> On Jul 5, 2011, at 5:30 AM, James Reid wrote:
>
> > Hi all,
> >
> >  I think I found a bug but I want to be sure.  I have a parser grammar
> that
> > I run two passes on.  Here are short versions of the rules.
> >
> >
> >
> > firstpass
> >  :  (collect_matches
> >  |  collect_labels
> >  |  .)*
> >  ;
> >
> >
> > script
> >  :  header? matches* EOF
> >  ;
> >
> >
> > When I run the code I use a CommonTokenStream and do the following
> >
> > CommonTokenStream tokens = new CommonTokenStream(lex);
> > MyParser parser = new MyParser(tokens);
> >
> > parser.firstpass();
> >
> > tokens.reset();
> >
> > parser.script();
> >
> >
> > The problem comes when I do the tokens.reset().  If the very first token
> is
> > a comment (i.e. on the hidden channel) it is returned in parser.script()
> and
> > throws a NoViableAlternativeException because nothing in my grammar is
> > expecting a comment.  To get around this I do the following...
> >
> >           //Reset the tokens back to the begining
> >            tokens.reset();
> >
> >            //For some reason after the tokens have been buffered up if a
> > hidden
> >            //token is the first token it is returned instead getting the
> >            //first non-hidden token.  This little hack works around that.
> >            if (tokens.LT(1).getChannel() == Token.HIDDEN_CHANNEL){
> >                tokens.consume();
> >            }
> >
> >            //now we can build the AST
> >            r=parser.script();
> >
> > If I consume the hidden token then the token stream points at the next
> > on-channel token like it is supposed to do.  Is this a bug or am I doing
> > things wrong?
> >
> > Thanks,
> > James
> >
> > PS:  This is with Antlr 3.3
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>


More information about the antlr-interest mailing list