[antlr-interest] Big grammar => static initializer/method size is exceeding the 65535 bytes limit

Loring Craymer lgcraymer at yahoo.com
Wed Nov 4 11:13:29 PST 2009


If DFA size grew linearly with the size of the grammar, Jim would be correct.  The evidence I have seen is that they grow nonlinearly, so partitioning a grammar will not always work, and it is always best if a tool circumvents mysterious "explosions" during development.  Alex's solution is both elegant and easy to  apply, and the net cost is a few more .class files.

--Loring




----- Original Message ----
> From: Jim Idle <jimi at temporal-wave.com>
> To: Antlr interest <antlr-interest at antlr.org>
> Sent: Wed, November 4, 2009 8:10:14 AM
> Subject: Re: [antlr-interest] Big grammar => static initializer/method size is exceeding the 65535 bytes limit
> 
> Guys - you are asking for the wrong problem to be fixed (at least of the three 
> of you, at least two will be ;-). Try the new -X options, then look at splitting 
> your grammar into multiple import grammars, then start taking out huge 
> predicates such as (expression)=> or generally (rule)=>. You can stop anywhere 
> along that path if you do not feel that optimizing the grammar is something 
> worth your while and the first and/or second options make the DFA table size 
> issue go away.
> 
> There are cases where big DFAs become inevitable, and then you should definitely 
> look at the import capability, which will prevent a single parser class being 
> used for everything and allow you to manage what goes in which class at the 
> grammar level.
> 
> Jim
> 
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Renee Luo
> > Sent: Wednesday, November 04, 2009 6:45 AM
> > To: Antlr interest
> > Subject: Re: [antlr-interest] Big grammar => static initializer/method
> > size is exceeding the 65535 bytes limit
> > 
> > Yes, we are trying to migrate our ANTLR2 grammar to ANTLR2, we are also
> > facing this problem. If the static initialize code will be separated
> > from parser_class, That's will be great for us.
> > 
> > Renee
> > 
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Andreas Meyer
> > Sent: Wednesday, November 04, 2009 8:32 AM
> > To: Antlr interest
> > Subject: Re: [antlr-interest] Big grammar => static initializer/method
> > size is exceeding the 65535 bytes limit
> > 
> > Back in the days when we tried to migrate our ANTLR2 grammar to ANTLR3,
> > we also experienced this problem, due to lots of static initializer
> > code
> > in the _parser_ class. Our solution was to apply some perl-skript
> > magic,
> > but if Alex Marin now proposes a built-in solution, that is only good
> > for ANTLR.
> > 
> > Andreas
> > 
> > Jim Idle schrieb:
> > > I think that the issue is more likely something to do with your lexer
> > specification. You should not need to worry about having lots of
> > keywords, so one of the other rules must be causing the huge expansion.
> > For instance I have problems with the complete lexer for TSQL, which
> > has more keywords than you can shake a stick at.
> > >
> > > Did you ever post your complete lexer spec? I was out of the country
> > when you first started this thread.
> > >
> > > Jim
> > >
> > 
> > 
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-address
> > 
> > This email and its attachments may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views
> > or opinions expressed are solely those of the author and do not
> > necessarily represent those of ImexSystems Inc.
> > If you are not the intended recipient of this email and its
> > attachments, you must take no action based upon them, nor must you copy
> > or show them to anyone.
> > Please contact the sender if you believe you have received this email
> > in error.
> > 
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-address
> 
> 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: 
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address



      


More information about the antlr-interest mailing list