[antlr-interest] Memory issue while lexer initialization

[elango m]

Thanks for your reply.

My lexer have around 60 tokens, i think its not huge.

I wanted to define keywords(around 50) in my lexer, something similar to the
following. When I remove this from the grammar I don't see the memory issue.
Is there any way to solve this problem?

KEYWORD : 'Keyword1' | 'Keyword2' | .... | 'Keyword50';

Thanks,
--Elango M


On Dec 28, 2007 11:40 PM, Terence Parr <parrt at cs.usfca.edu> wrote:

> hi. you must have one huge grammar.  Do you have a lexer with a
> million tokens?
> Ter
> On Dec 28, 2007, at 6:19 AM, elango m wrote:
>
> > Hi,
> >
> > I have a memory issue when I create an instance of the lexer (new
> > MyLexer(new ANTLRStringStream("expression")). The following code in
> > the lexer creates instances of short[] with a huge data, consuming
> > more than 25 Mb. I don't understand why it requires such a huge
> > memory. Any help is appreciated.
> >
> >     static {
> >         int numStates = DFA14_transitionS.length;
> >         DFA14_transition = new short[numStates][];
> >         for (int i=0; i<numStates; i++) {
> >             DFA14_transition[i] = DFA.unpackEncodedString
> > (DFA14_transitionS[i]);
> >         }
> >     }
> >
> > Thanks,
> > --Elango M
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20071231/25552d70/attachment.html