[antlr-interest] Think I found a bug.

Terence Parr parrt at cs.usfca.edu
Wed Jul 6 16:51:43 PDT 2011


tanx. fixed for 3.4
Ter
On Jul 6, 2011, at 4:15 PM, James Reid wrote:

> This worked...
> 
> @Override
>     public void reset() {
>         super.reset();
>         p = skipOffTokenChannels(0);
>     }
> 
> Glad I could contribute in some small way.
> 
> James
> 
> On Wed, Jul 6, 2011 at 1:57 PM, Terence Parr <parrt at cs.usfca.edu> wrote:
> Hi James. ack! i think reset() should call
> 
> skipOffTokenChannels()
> 
> Can you override reset to call super.reset() then skipOffTokenChannels() and see if that works?  If so, i can fix for 3.4
> Ter
> On Jul 5, 2011, at 5:30 AM, James Reid wrote:
> 
> > Hi all,
> >
> >  I think I found a bug but I want to be sure.  I have a parser grammar that
> > I run two passes on.  Here are short versions of the rules.
> >
> >
> >
> > firstpass
> >  :  (collect_matches
> >  |  collect_labels
> >  |  .)*
> >  ;
> >
> >
> > script
> >  :  header? matches* EOF
> >  ;
> >
> >
> > When I run the code I use a CommonTokenStream and do the following
> >
> > CommonTokenStream tokens = new CommonTokenStream(lex);
> > MyParser parser = new MyParser(tokens);
> >
> > parser.firstpass();
> >
> > tokens.reset();
> >
> > parser.script();
> >
> >
> > The problem comes when I do the tokens.reset().  If the very first token is
> > a comment (i.e. on the hidden channel) it is returned in parser.script() and
> > throws a NoViableAlternativeException because nothing in my grammar is
> > expecting a comment.  To get around this I do the following...
> >
> >           //Reset the tokens back to the begining
> >            tokens.reset();
> >
> >            //For some reason after the tokens have been buffered up if a
> > hidden
> >            //token is the first token it is returned instead getting the
> >            //first non-hidden token.  This little hack works around that.
> >            if (tokens.LT(1).getChannel() == Token.HIDDEN_CHANNEL){
> >                tokens.consume();
> >            }
> >
> >            //now we can build the AST
> >            r=parser.script();
> >
> > If I consume the hidden token then the token stream points at the next
> > on-channel token like it is supposed to do.  Is this a bug or am I doing
> > things wrong?
> >
> > Thanks,
> > James
> >
> > PS:  This is with Antlr 3.3
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 
> 



More information about the antlr-interest mailing list