[antlr-interest] [Antlr + ST] Huge output string - OutOfMemoryError

[Terence Parr]

An important note about using ST: don't translate toString and then  
emit. Send in a Writer to write().  No need to buffer in memory.   
CC'ing the other list.
Ter

On Oct 17, 2009, at 9:05 PM, Jim Idle wrote:

> Run Java with   - Xmx1500M
>
> Or some other option that increases available memory size. However  
> this could point to something that can be better organized in terms  
> of recursion or whatever.
>
> You may also consider running Java with a better garbage collector  
> such as:
>
> -Xincgc
>
> JIm
>
>
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org 
> ] On Behalf Of Hieu Phung
> Sent: Friday, October 16, 2009 4:03 PM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] [Antlr + ST] Huge output string -  
> OutOfMemoryError
>
> Hi all,
>
> I am writing a program using Antlr and StringTemplate to translate  
> an input file to an XML file. Input is small but output is huge:  
> 720KB input file will generate about 100MB output file. When my  
> input reached 7608KB, I got OOM error
>
> time taken in second: 18 - file size in KB: 4804
> time taken in second: 20 - file size in KB: 5205
> time taken in second: 20 - file size in KB: 5605
> time taken in second: 21 - file size in KB: 6006
> time taken in second: 23 - file size in KB: 6406
> time taken in second: 25 - file size in KB: 6806
> time taken in second: 31 - file size in KB: 7207
>
> at size 7608KB:
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>         at java.util.Arrays.copyOf(Arrays.java:2882)
>         at  
> java 
> .lang 
> .AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
>         at  
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)
>         at java.lang.StringBuffer.append(StringBuffer.java:320)
>         at java.io.StringWriter.write(StringWriter.java:60)
>         at  
> org 
> .antlr.stringtemplate.AutoIndentWriter.write(AutoIndentWriter.java: 
> 172)
>         at  
> org.antlr.stringtemplate.language.ASTExpr.writePOJO(ASTExpr.java:702)
>         at  
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:683)
>         at  
> org 
> .antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java: 
> 660)
>         at  
> org 
> .antlr 
> .stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java: 
> 86)
>         at  
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149)
>         at  
> org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705)
>         at  
> org.antlr.stringtemplate.language.ASTExpr.writeTemplate(ASTExpr.java: 
> 750)
>         at  
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:680)
>         at  
> org 
> .antlr 
> .stringtemplate.language.ASTExpr.writeIterableValue(ASTExpr.java:792)
>         at  
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:682)
>         at  
> org 
> .antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java: 
> 660)
>         at  
> org 
> .antlr 
> .stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java: 
> 86)
>         at  
> org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149)
>         at  
> org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705)
>         at  
> org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java: 
> 1670)
>         at  
> org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java: 
> 1661)
>         at FFMParser.file(FFMParser.java:127)
>         at CSVTest.main(CSVTest.java:33)
>
> Is there any suggestion so that program can generate 1GB file?
>
> 1)       Having multiple templates, each template generates small  
> string, then write the small strings to file.
> My input has format: ((Part1) (Part2)+ (Part3)). For each part2,  
> there is an xml element in the output. I can have 3 templates here.  
> In Antlr action, I will write each part to file when parsing is done  
> for the part.
>
> 2)       Any template engine that could write to file directly,  
> instead of returning a string...? (This might not be proper to  
> discussed here :D)
>
> I hope problem is stated clearly. Please let me know if you need  
> more information. Any suggestion is much appreciated. Thank you.
> Regards,
> Helen
>
>
>
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091017/d0ed042b/attachment.html