[antlr-interest] Big grammar => static initializer/method size is exceeding the 65535 bytes limit

[Jim Idle]

Yes - what I am saying is that you are getting such huge DFA tables generated because your rules are not well organized and it is taking a huge amount of analysis and so on to work out how to put it together for you. 

You can avoid table driven DFAs by using the new switches for code generation in ANTLR 3.2. These will influence code generation to use big switch() statements, which should result in less byte code than the DFA tables (though I don't know how big switch statements can be in Java).


Can increase this to use if statements more often....

  -Xmaxinlinedfastates m  max DFA states before table used rather than inlining      [10]


Increase this to say 3000...
  -Xmaxswitchcaselabels m don't generate switch() statements for dfas bigger  than m [300]

And reduce this to say 1....
  -Xminswitchalts m       don't generate switch() statements for dfas smaller than m [3]

To generate switches rather than tables.


But really, rejigging your parser will be a much better thing to do overall.

Jim


> -----Original Message-----
> From: Alex Marin [mailto:alex.marin at amiq.ro]
> Sent: Wednesday, November 04, 2009 5:27 AM
> To: Jim Idle
> Cc: Antlr interest; Etools
> Subject: Re: [antlr-interest] Big grammar => static initializer/method
> size is exceeding the 65535 bytes limit
> Importance: Low
> 
> Hello again,
> 
> The issue appeared when generating a parser grammar, it (mostly) had to
> do
> with the static initializer code for prediction DFAs. We modified the
> StringTemplate to enclose such declarations:
> 
> static final String DFA1_eotS =
> "\12\uffff";
> static final String DFA1_eofS =
> "\1\1\11\uffff";
> static final String DFA1_minS =
> "\1\5\11\uffff";
> static final String DFA1_maxS =
> "\1\23\11\uffff";
> static final String DFA1_acceptS =
> "\1\uffff\1\2\1\1\7\uffff";
> static final String DFA1_specialS =
> "\12\uffff}>";
> static final String[] DFA1_transitionS = {
> "\1\2\6\uffff\5\2\1\uffff\2\2",
> "",
> "",
> "",
> "",
> "",
> "",
> "",
> "",
> ""
> };
> 
> 
> in static classes. Happily, this fixed the problem.
> 
> Best regards,
> Alex Marin
> 
> On Tue, 03 Nov 2009 12:28:19 -0800, "Jim Idle" <jimi at temporal-wave.com>
> wrote:
> > I think that the issue is more likely something to do with your lexer
> > specification. You should not need to worry about having lots of
> keywords,
> > so one of the other rules must be causing the huge expansion. For
> instance
> > I have problems with the complete lexer for TSQL, which has more
> keywords
> > than you can shake a stick at.
> >
> > Did you ever post your complete lexer spec? I was out of the country
> when
> > you first started this thread.
> >
> > Jim
> >
> >> -----Original Message-----
> >> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> >> bounces at antlr.org] On Behalf Of Alex Marin
> >> Sent: Tuesday, November 03, 2009 10:43 AM
> >> To: Antlr interest
> >> Cc: Etools
> >> Subject: [antlr-interest] Big grammar => static initializer/method
> size
> >> is exceeding the 65535 bytes limit
> >>
> >> Hello,
> >>
> >> I've managed to find an automated solution for one of the previously
> >> reported errors, namely for:
> >>
> >> The code for the static initializer is exceeding the 65535 bytes
> limit
> >>
> >> The "fix" consists of modifying the StringTemplate that generates
> the
> >> parser code so that static initializers are automatically included
> in
> >> speparate classes (and properly referenced). By using static
> classes,
> >> all of the code is contained in a single file, so the change is
> >> transparent from a user's point of view.
> >>
> >> You can find the modified code in the attached file.
> >>
> >> Best regards,
> >> Alex Marin
> >>
> >> PS: In order to use the fix you have to replace the Java.stg from
> >> org.antlr.codegen.templates.Java with this one and recompile antlr.
> >
> >
> >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> address