[antlr-interest] How to reduce the size of the generated lexer?

Mu Qiao qiaomuf at gmail.com
Sun Mar 6 17:18:41 PST 2011


Hi

I use c runtime libantlr3c-3.1.3. The generated lexer is bigger than
10 MB full of arrays of integers. I tried to see what was going on and
I found there was a rule:
NQCHAR_NO_ALPHANUM
    :   ~('\n'|'\r'|'
'|'\t'|'\\'|CARET|QMARK|COLON|AT|SEMIC|POUND|SLASH|BANG|TIMES|COMMA|PIPE|AMP|MINUS|PLUS|PCT|EQUALS|LSQUARE|RSQUARE|RPAREN|LPAREN|RBRACE|LBRACE|DOLLAR|TICK|DOT|LT|GT|SQUOTE|QUOTE|'a'..'z'|'A'..'Z'|'0'..'9')+;

If I remove the rule, the lexer is only 400 KB.

I'm still new to antlr and I'm not sure if there is any way to
refactor the rule and reduce the size of lexer. Could anyone please
help me out?

-- 
Best wishes,
Mu Qiao
GnuPG fingerprint: 92B1 B0C4 8D14 F8C4 EFA5  3ACC 30B3 0DE4 17B1 57E9


More information about the antlr-interest mailing list