[antlr-interest] ANTLR running out of memory while parsing huge files

Nick Vlassopoulos nvlassopoulos at gmail.com
Tue Apr 21 06:42:13 PDT 2009


Hi Andreas,

Thanks for your fast reply!
So it should be something like a "line parser" that's instatiated for each
line of the BODY section!

Thanks again!
Nikos

On Tue, Apr 21, 2009 at 2:22 PM, Andreas Meyer
<andreas.meyer at smartshift.de>wrote:

> I do not remember something built-in, so I guess the easiest solution
> would be to create a lexer for the HEADER/DECL/BODY/line/END tokens, and
> from within the lexer, instantiate/call a new lexer/parser.
>
> Best,
> Andreas Meyer
>
> smartShift
> smart e*lliance GmbH
> Willy-Brandt Platz 6
> 68161 Mannheim
> Germany
>
> T +49 (621) 400 676-13
> F +49 621 400 67606
>
> Geschäftsführer:  Stefan Hetges
> Amtsgericht Hamburg, HRB 83484
> Ust.-ID-Nr.: DE 813489791
>
>
>
> Nick Vlassopoulos schrieb:
> > Hi,
> >
> > I am fairly new to ANTLR and I have come accross a problem.
> > I have written a simple grammar to parse huge data files (several
> > gigabytes each)
> > and antlr seems to crash by running out of memory (I am using "C" as
> > the target language).
> >
> > The data files have the general format:
> > HEADER
> >  DECL
> > BODY
> >  <several millions of lines here>
> > END
> >
> > What seems to be the problem is that antlr tries to parse the whole
> > data file
> > at once. Is there a way to "force" parsing line by line? (at least for
> > the "BODY" part?)
> >
> >
> > Thank you very much in advance for your time!
> >
> > Nikos
> >
> >
> >
> >
> > ------------------------------------------------------------------------
> >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090421/803753b8/attachment.html 


More information about the antlr-interest mailing list