[antlr-interest] Parsing Large Files

Jim Idle jimi at temporal-wave.com
Thu Apr 1 06:54:08 PDT 2010


Kumaap0,

It looks like just some inexperience here. My first guess is that you have a lexer rule that matches and empty string and so you are getting an infinite number of tokens.

Look for:

LEXERRULE: ('xxxxx')* ;

Where a  * should be a + as that is the most common error.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Kumar, Amitesh
> Sent: Thursday, April 01, 2010 2:13 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] Parsing Large Files
> 
> Hi Guys what we are looking for is just parsing the file and recording
> the errors we don't need to keep a track of any tokens or a AST.
> Im getting
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>         at java.util.Arrays.copyOf(Arrays.java:2760)
>         at java.util.Arrays.copyOf(Arrays.java:2734)
>         at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
>         at java.util.ArrayList.add(ArrayList.java:351)
>         at
> org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:1
> 1
> 6)
>         at
> org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238)
>         at
> org.antlr.runtime.Parser.getCurrentInputSymbol(Parser.java:54)
>         at
> org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:104)
>         at DealsAll2Parser.header(DealsAll2Parser.java:123)
>         at DealsAll2Parser.file(DealsAll2Parser.java:67)
>         at AntlrMain.main(AntlrMain.java:53)
> I see where the error is coming from the CommonTokenStream is keeping
> track of all past tokens, how can I make it so it doesn't. Do I have to
> create my own Token Stream? Or is there a easy way.
> 
> Cheers
> Kumaap0
> 
> 
> ***********************************************************************
> ******
> More information on Standard Bank is available at www.standardbank.com
> 
> Everything in this email and any attachments relating to the official
> business
> of Standard Bank Group Limited and any or all subsidiaries, ("the
> Company"), is
> proprietary to the Company. It is confidential, legally privileged and
> protected
> by relevant laws. The Company does not own and endorse any other
> content.
> Views and opinions are those of the sender unless clearly stated as
> being
> that of the Company.
> 
> The person or persons addressed in this email are the sole authorised
> recipient. Please notify the sender immediately if it has
> unintentionally,
> or inadvertently reached you and do not read, disclose or use the
> content
> in any way and delete this e-mail from your system.
> 
> The Company cannot ensure that the integrity of this email has been
> maintained nor that it is free of errors, virus, interception or
> interference.
> The sender therefore does not accept liability for any errors or
> omissions
> in the contents of this message which arise as a result of e-mail
> transmission.
> If verification is required please request a hard-copy version. This
> message
> is provided for informational purposes and should not be construed as a
> solicitation or offer to buy or sell any securities or related
> financial instruments.
> ***********************************************************************
> ******
> 
> 
> This message has been scanned for viruses by BlackSpider MailControl -
> www.blackspider.com
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address





More information about the antlr-interest mailing list