[antlr-interest] Mark/rewind documentation
Ric Klaren
klaren at cs.utwente.nl
Tue Aug 31 04:32:46 PDT 2004
On Tue, Aug 31, 2004 at 11:58:32AM +0200, Xavier Décoret wrote:
> In a recent thread about rewinding a token, the mark/rewind mechanism
> was mentionned. I am looking in the manual and I cannot find that much
> info about this mechanism. There is just an example of mark/rewind calls
> just before section "Scanning binary files" in
> http://www.antlr.org/doc/lexer.html#Lexical_Analysis_with_ANTLR.
>
> Where can I find more info?
See the javadocs in antlr/InputBuffer.java.
> What I want to do is multiplexing with rewinding of a token. When lexer0
> finds a @, it switch the selector to another lexer (lexer1, which shares
> the same input) *and* rewind the @ because it must be used by lexer1.
Note that when you use mark/rewind on extensive chunks of input that your
parser will probably be a lot more memory intensive. When you call mark()
all the input from then on will be safed in a circular queue. When you
rewind to that mark point you'll start off again at the point of the mark
call.
I'm not 100% sure wether it's actually needed but it is probably a good
idea to make sure that you match mark calls with either commit or rewind
calls (if you 'lose' commit/rewind calls you probably will leak memory (the
queue will never be reset)).
As an alternative you can probably restructure the lexers so that the
mark/rewind won't be necessary.
Cheers,
Ric
--
-----+++++*****************************************************+++++++++-------
---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893755 ----
-----+++++*****************************************************+++++++++-------
"Evil will always triumph, because Good is dumb." --- Spaceballs
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list