[antlr-interest] [Antlr + ST] Huge output string - OutOfMemoryError
Terence Parr
parrt at cs.usfca.edu
Sat Oct 17 10:24:43 PDT 2009
An important note about using ST: don't translate toString and then
emit. Send in a Writer to write(). No need to buffer in memory.
CC'ing the other list.
Ter
On Oct 17, 2009, at 9:05 PM, Jim Idle wrote:
> Run Java with - Xmx1500M
>
> Or some other option that increases available memory size. However
> this could point to something that can be better organized in terms
> of recursion or whatever.
>
> You may also consider running Java with a better garbage collector
> such as:
>
> -Xincgc
>
> JIm
>
>
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org
> ] On Behalf Of Hieu Phung
> Sent: Friday, October 16, 2009 4:03 PM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] [Antlr + ST] Huge output string -
> OutOfMemoryError
>
> Hi all,
>
> I am writing a program using Antlr and StringTemplate to translate
> an input file to an XML file. Input is small but output is huge:
> 720KB input file will generate about 100MB output file. When my
> input reached 7608KB, I got OOM error
>
> time taken in second: 18 - file size in KB: 4804
> time taken in second: 20 - file size in KB: 5205
> time taken in second: 20 - file size in KB: 5605
> time taken in second: 21 - file size in KB: 6006
> time taken in second: 23 - file size in KB: 6406
> time taken in second: 25 - file size in KB: 6806
> time taken in second: 31 - file size in KB: 7207
>
> at size 7608KB:
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2882)
> at
> java
> .lang
> .AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
> at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)
> at java.lang.StringBuffer.append(StringBuffer.java:320)
> at java.io.StringWriter.write(StringWriter.java:60)
> at
> org
> .antlr.stringtemplate.AutoIndentWriter.write(AutoIndentWriter.java:
> 172)
> at
> org.antlr.stringtemplate.language.ASTExpr.writePOJO(ASTExpr.java:702)
> at
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:683)
> at
> org
> .antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java:
> 660)
> at
> org
> .antlr
> .stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java:
> 86)
> at
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149)
> at
> org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705)
> at
> org.antlr.stringtemplate.language.ASTExpr.writeTemplate(ASTExpr.java:
> 750)
> at
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:680)
> at
> org
> .antlr
> .stringtemplate.language.ASTExpr.writeIterableValue(ASTExpr.java:792)
> at
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:682)
> at
> org
> .antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java:
> 660)
> at
> org
> .antlr
> .stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java:
> 86)
> at
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149)
> at
> org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705)
> at
> org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java:
> 1670)
> at
> org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java:
> 1661)
> at FFMParser.file(FFMParser.java:127)
> at CSVTest.main(CSVTest.java:33)
>
> Is there any suggestion so that program can generate 1GB file?
>
> 1) Having multiple templates, each template generates small
> string, then write the small strings to file.
> My input has format: ((Part1) (Part2)+ (Part3)). For each part2,
> there is an xml element in the output. I can have 3 templates here.
> In Antlr action, I will write each part to file when parsing is done
> for the part.
>
> 2) Any template engine that could write to file directly,
> instead of returning a string...? (This might not be proper to
> discussed here :D)
>
> I hope problem is stated clearly. Please let me know if you need
> more information. Any suggestion is much appreciated. Thank you.
> Regards,
> Helen
>
>
>
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091017/d0ed042b/attachment.html
More information about the antlr-interest
mailing list