[antlr-interest] Mark/rewind documentation

[Ric Klaren]

On Tue, Aug 31, 2004 at 11:58:32AM +0200, Xavier Décoret wrote:
> In a recent thread about rewinding a token, the mark/rewind mechanism
> was mentionned. I am looking in the manual and I cannot find that much
> info about this mechanism. There is just an example of mark/rewind calls
> just before section "Scanning binary files" in
> http://www.antlr.org/doc/lexer.html#Lexical_Analysis_with_ANTLR.
>
> Where can I find more info?

See the javadocs in antlr/InputBuffer.java.

> What I want to do is multiplexing with rewinding of a token. When lexer0
> finds a @, it switch the selector to another lexer (lexer1, which shares
> the same input) *and* rewind the @ because it must be used by lexer1.

Note that when you use mark/rewind on extensive chunks of input that your
parser will probably be a lot more memory intensive. When you call mark()
all the input from then on will be safed in a circular queue. When you
rewind to that mark point you'll start off again at the point of the mark
call.

I'm not 100% sure wether it's actually needed but it is probably a good
idea to make sure that you match mark calls with either commit or rewind
calls (if you 'lose' commit/rewind calls you probably will leak memory (the
queue will never be reset)).

As an alternative you can probably restructure the lexers so that the
mark/rewind won't be necessary.

Cheers,

Ric
--
-----+++++*****************************************************+++++++++-------
    ---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893755  ----
-----+++++*****************************************************+++++++++-------
     "Evil will always triumph, because Good is dumb." ---  Spaceballs



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/