[antlr-interest] How to put back a token read by lexer
Matthew Ford
Matthew.Ford at forward.com.au
Thu Oct 25 16:28:35 PDT 2001
You want to be careful about pushing back tokens because of Antlr's
lookahead.
Use the lookahead instead.
matthew
----- Original Message -----
From: "bob mcwhirter" <bob at werken.com>
To: <antlr-interest at yahoogroups.com>
Sent: Friday, October 26, 2001 9:11 AM
Subject: Re: [antlr-interest] How to put back a token read by lexer
>
> Alternatively, couldn't you just use look-ahead to do whatever
> it is you need to do, to avoid having to actually pop the token
> in the first place?
>
> Or are you inserting artificial tokens into the stream based
> upon things you've seen up to that point?
>
> -bob
>
> On Thu, 25 Oct 2001, Sinan wrote:
>
> > cela_vei at yahoo.com wrote:
> > >
> > > I want to put back the most recently read token back into the
> > > input stream. I searched through the archive and found that people
> > > have asked this question multiple times but without an answer.
> > >
> > > Please Help
> > > Cela Vei
> > >
> > >
> > >
> > > Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
> >
> >
> >
> > Do something like the following to implement your own push....
> >
> > public class MyTokenStreamSelector extends TokenStreamSelector {
> >
> >
> > public MyTokenStreamSelector() {
> > super();
> > ...create some kind of stack.....
> >
> > }
> > public void pushToken(Token token) {
> > ....push it somewhere;
> > }
> > public Token nextToken() throws TokenStreamException {
> > if anything pushed, pop an return it
> >
> > else
> > try {
> > Token tok = super.nextToken();
> >
> > return tok;
> > }
> > catch (TokenStreamRetryException r) {
> > // whatever.....
> > }
> > }
> > }
> > }
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
> >
> >
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list