[antlr-interest] C target memory usage

Jim Idle jimi at temporal-wave.com
Sat Dec 31 14:32:17 PST 2011


If I get the time, then to 3.4

Jim

On Dec 31, 2011, at 5:53, Richard Gildea <rgildea at gmail.com> wrote:

> Hi,
> 
> Thanks for you reply. No, we are not using the $text since I had read that it could lead to increased memory usage. Are the improvements that you mention targeted at ANTLR 4 or ANTLR 3.x?
> 
> Cheers,
> 
> Richard
> 
> On 26 December 2011 09:33, Jim Idle <jimi at temporal-wave.com> wrote:
> Are you sure that this is not because you are using $text? Anyway, over
> the next few days I am going to make some changes to the runtime for speed
> and memory use, that will probably help everyone.
> 
> Jim
> 
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Richard Gildea
> > Sent: Thursday, December 22, 2011 8:00 PM
> > To: antlr-interest at antlr.org
> > Subject: [antlr-interest] C target memory usage
> >
> > Hi,
> >
> > We have been successfully using antlr in the form of the C target for
> > some time, however we have recently noticed that the memory consumption
> > can be quite large - up to 150 times the size of the input file. Is
> > this factor of
> > ~150 to be expected, or does it indicate that we may be doing something
> > wrong? For the vast majority of possible inputs this does not cause a
> > problem, however some input files can be as large as 0.5 Gb, giving a
> > peak memory usage of 75 Gb - not exactly feasible on most machines!
> >
> > Does anyone have any examples of using a custom lexer that provides a
> > token buffer rather than storing all tokens in memory?
> >
> > Cheers,
> >
> > Richard
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-address
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 


More information about the antlr-interest mailing list