[antlr-interest] How to put back a token read by lexer

Matthew Ford Matthew.Ford at forward.com.au
Thu Oct 25 16:28:35 PDT 2001


You want to be careful about pushing back tokens because of Antlr's
lookahead.
Use the lookahead instead.
matthew
----- Original Message -----
From: "bob mcwhirter" <bob at werken.com>
To: <antlr-interest at yahoogroups.com>
Sent: Friday, October 26, 2001 9:11 AM
Subject: Re: [antlr-interest] How to put back a token read by lexer


>
> Alternatively, couldn't you just use look-ahead to do whatever
> it is you need to do, to avoid having to actually pop the token
> in the first place?
>
> Or are you inserting artificial tokens into the stream based
> upon things you've seen up to that point?
>
> -bob
>
> On Thu, 25 Oct 2001, Sinan wrote:
>
> > cela_vei at yahoo.com wrote:
> > >
> > > I want to put back the most recently read token back into the
> > > input stream. I searched through the archive and found that people
> > > have asked this question multiple times but without an answer.
> > >
> > > Please Help
> > > Cela Vei
> > >
> > >
> > >
> > > Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
> >
> >
> >
> > Do something like the following to implement your own push....
> >
> > public class MyTokenStreamSelector extends TokenStreamSelector {
> >
> >
> >         public MyTokenStreamSelector() {
> >                 super();
> > ...create some kind of stack.....
> >
> >         }
> >         public void pushToken(Token token) {
> >                 ....push it somewhere;
> >         }
> >         public Token nextToken() throws TokenStreamException {
> >  if anything pushed, pop an return it
> >
> >                 else
> >                         try {
> >                 Token tok = super.nextToken();
> >
> >                    return tok;
> >                         }
> >                         catch (TokenStreamRetryException r) {
> > // whatever.....
> >                         }
> >                 }
> >         }
> > }
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
> >
> >
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list