[antlr-interest] (newbie) Code too large problem

Thomas Brandon tbrandonau at gmail.com
Thu Aug 2 23:41:07 PDT 2007


On 8/3/07, Terence Parr <parrt at cs.usfca.edu> wrote:
> Hmm...yeah, must have million keywords or other tokens in your grammar.
>
> I'm working on the solution (for v3.1) but that won't be available for a
> while.  I'd suggest doing a bunch of fragment rules and then having a
> non-fragment rule invoke a bunch of other rules that call all the fragment
> tokens...try to break up into functions.  Look at output java code.
> Ter
>
As Ter says, check the generated mTokens and see where the complexity is.
If it's a complicated prediction between a few rules, try moving them
into fragments accessed through a single rule as Ter says.
Or if you have a lot of keywords you could make them fragments and
have a single rule matching idents and keywords. Like:
fragment KEYWORD1: 'keyword1';
fragment KEYWORD2: 'keyword2';
ID: KEYWORD1 {$type=KEYWORD1;}
   | KEYWORD2 {$type=KEYWORD2;}
   | 'a'..'z'
   ;

Or you could replace keywords with a hashtable lookup in your
identifier rule. e.g.
ID: 'a'..'z' {$type = lookupKeyword($text);};
And in your members:
Map<String,Integer) keywords;
void initKeywords() {
  keywords = new HashMap<String,Integer>();
  keywords.put("keyword1", KEYWORD1);
  keywords.put("keyword2", KEYWORD2);
  ...
}
int lookupKeyword(String text) {
  Integer type = keywords.get(text);
  if(type != null)
    return type.intValue();
  else
    return ID;
}
With KEYWORD1 and KEYWORD2 in your tokens section.

Tom.
>
> On Aug 2, 2007, at 8:40 PM, ali azimi wrote:
>
> Hi,
>
> I have this error message and have been spending much time to remedy it
> without success. I have another grammar which is much bigger than this one
> and works perfectly. Could you please advice me? The code is big and I can
> not post it on the forum. However if anyone wants kindly to have a look at
> it I can mail. It must be some kind of looping happening in it. When I
> exclude some parts of the grammar by turning them to comments the debugger
> lets me to carry on.
>
> Thank you in advance. I appreciate it alot.
>
> Best regards,
>
> Al
>
> The error message:
>
> [04:26:33] 1 error
> [04:26:54] \tmp\antlrworks\FpCif4Lexer.java:4551: code too
> large
> [04:26:54]     public void mTokens() throws RecognitionException {
> [04:26:54]                 ^
> [04:26:55] 1 error
>
> ________________________________
> Be a better Globetrotter. Get better travel answers from someone who knows.
> Yahoo! Answers - Check it out.
>


More information about the antlr-interest mailing list