[antlr-interest] [Antlr + ST] Huge output string - OutOfMemoryError
Jim Idle
jimi at temporal-wave.com
Mon Oct 19 23:13:33 PDT 2009
Then you either need to avoid the toString() and use a writer, or see if there is something wrong with your template that is causing infinite recursion etc.
Jim
From: Hieu Phung [mailto:phungngochieu at gmail.com]
Sent: Tuesday, October 20, 2009 11:10 AM
To: Jim Idle
Cc: antlr-interest at antlr.org
Subject: Re: [antlr-interest] [Antlr + ST] Huge output string - OutOfMemoryError
Hi Jim,
Thank you for your suggestion. Actually the result I got was with JVM setting -Xms1024m -Xmx1024m. And it's the maximum memory size I can set for my application server. The gc setting was also configured...
Regards,
Helen
On Sun, Oct 18, 2009 at 12:05 PM, Jim Idle <jimi at temporal-wave.com> wrote:
Run Java with - Xmx1500M
Or some other option that increases available memory size. However this could point to something that can be better organized in terms of recursion or whatever.
You may also consider running Java with a better garbage collector such as:
-Xincgc
JIm
From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Hieu Phung
Sent: Friday, October 16, 2009 4:03 PM
To: antlr-interest at antlr.org
Subject: [antlr-interest] [Antlr + ST] Huge output string - OutOfMemoryError
Hi all,
I am writing a program using Antlr and StringTemplate to translate an input file to an XML file. Input is small but output is huge: 720KB input file will generate about 100MB output file. When my input reached 7608KB, I got OOM error
time taken in second: 18 - file size in KB: 4804
time taken in second: 20 - file size in KB: 5205
time taken in second: 20 - file size in KB: 5605
time taken in second: 21 - file size in KB: 6006
time taken in second: 23 - file size in KB: 6406
time taken in second: 25 - file size in KB: 6806
time taken in second: 31 - file size in KB: 7207
at size 7608KB:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)
at java.lang.StringBuffer.append(StringBuffer.java:320)
at java.io.StringWriter.write(StringWriter.java:60)
at org.antlr.stringtemplate.AutoIndentWriter.write(AutoIndentWriter.java:172)
at org.antlr.stringtemplate.language.ASTExpr.writePOJO(ASTExpr.java:702)
at org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:683)
at org.antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java:660)
at org.antlr.stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java:86)
at org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149)
at org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705)
at org.antlr.stringtemplate.language.ASTExpr.writeTemplate(ASTExpr.java:750)
at org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:680)
at org.antlr.stringtemplate.language.ASTExpr.writeIterableValue(ASTExpr.java:792)
at org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:682)
at org.antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java:660)
at org.antlr.stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java:86)
at org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149)
at org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705)
at org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java:1670)
at org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java:1661)
at FFMParser.file(FFMParser.java:127)
at CSVTest.main(CSVTest.java:33)
Is there any suggestion so that program can generate 1GB file?
1) Having multiple templates, each template generates small string, then write the small strings to file.
My input has format: ((Part1) (Part2)+ (Part3)). For each part2, there is an xml element in the output. I can have 3 templates here. In Antlr action, I will write each part to file when parsing is done for the part.
2) Any template engine that could write to file directly, instead of returning a string...? (This might not be proper to discussed here :D)
I hope problem is stated clearly. Please let me know if you need more information. Any suggestion is much appreciated. Thank you.
Regards,
Helen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091020/d3e6f336/attachment.html
More information about the antlr-interest
mailing list