[antlr-interest] Re: Reloading from previous parse?

Bill Canfield canfield at computer.org
Mon Mar 6 08:53:57 PST 2006

> Message: 6
> Date: Fri, 3 Mar 2006 23:37:59 -0500
> From: "Bryan Ewbank" <ewbank at gmail.com>
> Subject: [antlr-interest] Reloading from previous parse?
> Has anyone worked on, or have pointers to, technology for AST
> persistence?  I'm mainly curious about preserving/recovering symbol
> table information and file/line information from the original parse -
> the AST itself seems straightforward (by comparison  :-) 


After getting really bad performance with the boost::serialization
library, I just switched to a strategy where I save the tokens from my
antlr Lexer, then have a TokenStream that feeds them to the parser when
I want to reload them.

The TokenStream that saves the tokens injects fake tokens when the
file/line number changes.  The TokenStream that reads them filters those
out and adjusts the location information accordingly.

By the way, I'm not knocking the boost stuff, I think it just turns out
that if you try to write out the contents of your ASTs, it's sort of an
uncompressed version of the text they were parsed from.  So you might as
well save the original text and reparse.

Another alternative that comes to mind is using mmap(), but I haven't
tried that myself.


More information about the antlr-interest mailing list