[antlr-interest] (no subject)

Pascal Parrot pascal_parrot at hotmail.com
Thu Nov 15 11:45:34 PST 2012




Most of the information in tokenNames is already included in the keywords hashmap:  put("A1", exampleParser.A1);
And to one value corresponds one key. Maybe the information already in the map could be leveraged?


> Subject: Re: [antlr-interest] (no subject)
> From: parrt at cs.usfca.edu
> Date: Thu, 15 Nov 2012 10:44:46 -0800
> CC: sam at tunnelvisionlabs.com; antlr-interest at antlr.org
> To: pascal_parrot at hotmail.com
> 
> The issue is likely the tokenNames table:
> 
>         public static final String[] tokenNames = {
>                 "<INVALID>", "KEYWORD", "WS", "A1", "A2", "A3", "A4", "A5", "A6", "A7",
>                 "A8", "A9", "A10", "A11", "A12", "A13", "A14", "A15", "A16", "A17", "A18",
>> 
> Java can't have static arrays bigger than N / 6 due to limitation in .class file format. (The subject of the oracle/google android suit I was involved with, btw).
> 
> Not sure how to get around this. Maybe one big string and then another string encoding indexes of strings within.
> 
> Ter
> 
> On Nov 15, 2012, at 10:37 AM, Pascal Parrot wrote:
> 
> > 
> > 
> > 
> > 
> > One sample grammar (example.g4) is attached there:
> > http://www.antlr.org/pipermail/antlr-interest/attachments/20121114/c1188d89/attachment.zip
> > 
> > The initial version had a "string too long error". I no longer have the error if I use a hashmap for keywords, as described in the reference book (see attachment in link). However, I am getting a new error now, even when I use the -Xforce-atn option.
> > 
> > This is just an example grammar, so if the error is fixed, a new "too large, too long" error will probably pop up somewhere else. So, it brings up 2 questions:
> > 1) Is it in the antlr roadmap to check for java size limitations in the generated code and break it if necessary?
> > 2) If not, using a different target language is the only option for large grammars, isn't it?
> > 
> > Pascal 
> > 
> >> From: sam at tunnelvisionlabs.com
> >> To: pascal_parrot at hotmail.com; antlr-interest at antlr.org
> >> Subject: RE: [antlr-interest] (no subject)
> >> Date: Thu, 15 Nov 2012 15:15:29 +0000
> >> 
> >> ANTLR 4 is not currently optimized for this use case. You might be able to reduce the code size a bit by passing the -Xforce-atn flag when you generate your grammar.
> >> 
> >> To help with optimizing the generated code, can you provide me with one of the grammars that's causing a problem?
> >> 
> >> Thank you,
> >> --
> >> Sam Harwell
> >> Owner, Lead Developer
> >> http://tunnelvisionlabs.com
> >> 
> >> -----Original Message-----
> >> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Pascal Parrot
> >> Sent: Thursday, November 15, 2012 1:55 AM
> >> To: antlr-interest at antlr.org
> >> Subject: Re: [antlr-interest] (no subject)
> >> 
> >> 
> >> Jim,
> >> Yes, I looked a the generated code, but the error is on this line:
> >> protected static final PredictionContextCache _sharedContextCache = new PredictionContextCache(); PredictionContextCache does not appear anywhere else in the file and  _sharedContextCache is a parameter in a function.
> >> 
> >> Even if there was a huge something there, I wouldn't know what to do with it.
> >> 
> >> I guess my question is:
> >> Is antlr (java) suited for grammars with large sets of keywords and many parser rules?
> >> If it is, how should the grammar be organized so that the generated code fits within the limits of java in terms of size? Using hashmaps helps in the lexer, but what about in the parser?
> >> 
> >> Pascal
> >> 		 	   		  
> >> 
> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >> 
> >> 
> > 		 	   		  
> > 
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 
 		 	   		  


More information about the antlr-interest mailing list