[antlr-interest] Rewind one token?

[Ingo Maier]

On Mon, 2004-08-16 at 04:54, Ric Klaren wrote:
> On Mon, Aug 16, 2004 at 09:03:42AM -0400, Ingo Maier wrote:
> > Well, my question seemed to be misleading. I don't want to know where
> > mark/rewind is used (a call hierarchy check tells me that, I use eclipse
> > :)). I want to know, if there is a possibility to
> > rewind/backtrack/whatever an already read token? 
> > 
> > I use two lexers on the same stream with a multiplexer. First lexer
> > starts. When it passes a specific token (sequence) to the parser, lexers
> > will be switched. But the tokens already read by the first lexer and
> > indicating the switch will be needed by the second one.
> > 
> > Any ideas how to do that?
> 
> Call mark at the start of the switch sequence then switch to the other
> lexer and rewind directly at the start of the second lexer (e.g. pass the
> mark id and call rewind in the init action of the startrule)?
> 
> I'm not sure wether you do but.. it sounds like you switch lexers from the
> parser. In general this does not work (especially if there's some
> predicates in the parser (mark/rewind again)) (just checking)

Well, sorry, I think I was on the wrong level. The second lexer should
get a chance to "rescan" the last tokens, read by the first lexer. It
might get totally different tokens from the same input sequence. This
have to be done on the level of a CharBuffer or InputBuffer, probably.
So, I want to put back the last tokens, character by character.

Just to clarify, I want to do the following:

1) give parser1/lexer1 a try
2) when they fail, put back all tokens since failure, can be several due
to lookahead etc
3) give parser2/lexer2 a try, beginning from the point of failure, as if
nothing happened before

I hope this makes it clearer,
Ingo



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/