[antlr-interest] "code too large" / can't split AST output into smaller files

Fri Oct 8 12:34:25 PDT 2010

Is this a parser or a lexer rule? Do you have backtracking turned on? Do you
have a rule that has too many alts within the same rule? You should find
that right before the method, there is a comment that indicate which piece
of the grammar generated the code. There are a lot of reasons why this might
be happening but we can't really help without the grammar file to reference.
Or at least the  comments before the method and the method name. See where
the method is called from as that will also help. Open the generated code in
Netbeans.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Stephen Winnall
> Sent: Friday, October 08, 2010 11:57 AM
> To: Edson Tirelli
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] "code too large" / can't split AST output
into
> smaller files
> 
> So, I've split my grammar up, which caused a number of knock-on errors,
but
> now Antlr does its thing again without complaining.
> 
> However, even after moving some parser rules into an imported file, I
still
> get the "code too large". So I suspect Edson's "obvious" statement
indicates
> the source of the problem. The error report is:
> 
> 	/Users/steve/Work/Applications/Xxxx/target/generated-
> sources/antlr3/org/vimia/xxxx/antlr/Root.java:[4701,19] code too large
> 
> Closer observation reveals that the error points at a method which starts
at
> line 4701 and ends on line 11091. I can well imagine that that may be over
64K
> :-)
> 
> But I can see which rule it belongs to, so I can try and refactor the
rule.
> 
> Thanks for all your help.
> 
> Steve
> 
> On 8 Oct 2010, at 18:22, Edson Tirelli wrote:
> 
> >   Also, I might be stating the obvious, but I lost some time on this
> > recently: usually the "code too large" refers to a method that is over
> > the 64k limit, and not due to the size of the whole class. If that is
> > your case, it does not help to split the grammar, as the offending
> > method will sill be there.
> >
> >   In my case I had to rewrite the offending grammar rule to reduce the
> > size of the generated method. Once I did that, I was able to go back
> > to a non-composite grammar, that has much better tooling support.
> >
> >   Hope it helps,
> >     Edson
> >
> > 2010/10/8 Jim Idle <jimi at temporal-wave.com>:
> >> I don't know why you are seeing a difference other than you are not
> >> compiling what you think you are in Works, or a different version of
> >> ABTLR is in use between the two systems. Anyway, you are correct to
> >> use Maven, and should be using version 3.2.1 at the moment.
> >>
> >> Importing of tree walking and tree generating grammars works fine,
> >> just remember that the order of the statements is a bit fragile in
> >> 3.2 and so you have to get the statements in the correct order. Your
> >> grammar files should be in the antlr3/com/my/package/r.g order and
> >> your includes in the antlr3/imports directory. Then your importing
> grammar should be:
> >>
> >> parser grammar x;
> >> options {}
> >> import i1, i2, i3, i4, etc;
> >> tokens {}
> >> @parser::...
> >>
> >> rule1 : ....
> >>
> >>
> >> The grammars you import are:
> >>
> >> parser grammar myimport;
> >>
> >> rulenm: ...;
> >>
> >>
> >> Without you including any specific errors or your grammar files, it
> >> is difficult to see what else you are doing wrong, if anything.
> >>
> >> Jim
> >>
> >>> -----Original Message-----
> >>> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> >>> bounces at antlr.org] On Behalf Of Stephen Winnall
> >>> Sent: Friday, October 08, 2010 6:45 AM
> >>> To: antlr-interest at antlr.org
> >>> Subject: [antlr-interest] "code too large" / can't split AST output
> >>> into
> >> smaller
> >>> files
> >>>
> >>> I'm using Antlr 3.2 with Java 1.6.0_20 under Mac OS X 10.6.4.
> >>>
> >>> I have a composite grammar which generates AST trees. I can run and
> >>> debug the grammar from AntlrWorks 1.4 without any problem.
> >>>
> >>> However, my grammar is part of a larger project, which I am
> >>> developing
> >> with
> >>> NetBeans 6.9.1 and Maven, using the antlr3-maven-plugin. When I try
> >>> to build my project, the grammar file gets translated into Java
> >>> alright, but
> >> the
> >>> resultant Java parser file causes a "code too large" error.
> >>>
> >>> I don't understand why AntlrWorks can compile the grammar but
> >>> NetBeans cannot. Can anyone enlighten me? The generated parser file
> >>> is 13756 lines long and the "code too large" error under NetBeans
> >>> appears on line 7204,
> >> so
> >>> there seems to be a fundamental difference in what NetBeans/Maven
> >>> and AntlrWorks are doing.
> >>>
> >>> I have tried to split the grammar into smaller bits to work round
> >>> this
> >> problem.
> >>> I took some parser rules and put them into a different parser
> >>> grammar, which I imported into the main one. However, nearly all my
> >>> parser rules include "-> ^( ... )" to generate AST stuff. Antlr
> >>> complains when it sees
> >> a "^("
> >>> in the imported file. I have included "output=AST;" in the options
> >>> for the imported file.
> >>>
> >>> None of the examples of composite grammars that I have been able to
> >>> find in the net deal with AST output from the imported file. Is
> >>> there some
> >> trick I
> >>> should be aware of?
> >>>
> >>> Regards
> >>> Steve
> >>> --
> >>> This message has been scanned for viruses and dangerous content by
> >>> MailScanner, and is believed to be clean.
> >>>
> >>>
> >>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >>> Unsubscribe:
> >>> http://www.antlr.org/mailman/options/antlr-interest/your-
> >>> email-address
> >>
> >>
> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >> Unsubscribe:
> >> http://www.antlr.org/mailman/options/antlr-interest/your-email-addres
> >> s
> >>
> >
> >
> >
> > --
> >   Edson Tirelli
> >   JBoss Drools Core Development
> >   JBoss by Red Hat @ www.jboss.com
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >
> > --
> > This message has been scanned for viruses and dangerous content by
> > MailScanner, and is believed to be clean.
> >
> 
> 
> --
> This message has been scanned for viruses and dangerous content by
> MailScanner, and is believed to be clean.
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address