[antlr-interest] C target memory usage

Jim Idle jimi at temporal-wave.com
Mon Dec 26 09:33:17 PST 2011


Are you sure that this is not because you are using $text? Anyway, over
the next few days I am going to make some changes to the runtime for speed
and memory use, that will probably help everyone.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Richard Gildea
> Sent: Thursday, December 22, 2011 8:00 PM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] C target memory usage
>
> Hi,
>
> We have been successfully using antlr in the form of the C target for
> some time, however we have recently noticed that the memory consumption
> can be quite large - up to 150 times the size of the input file. Is
> this factor of
> ~150 to be expected, or does it indicate that we may be doing something
> wrong? For the vast majority of possible inputs this does not cause a
> problem, however some input files can be as large as 0.5 Gb, giving a
> peak memory usage of 75 Gb - not exactly feasible on most machines!
>
> Does anyone have any examples of using a custom lexer that provides a
> token buffer rather than storing all tokens in memory?
>
> Cheers,
>
> Richard
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address


More information about the antlr-interest mailing list