[antlr-interest] Why does Lexer of C++ run time target eat so much memory

[chain one]

Still waiting for help
I just wanna know, if c runtime target is  suitable for large input?

On 12/16/08, chain one <chainone at gmail.com> wrote:
> Hi,
> These days I am writing a parser for a kind of data file using C++. The
> format of the data file is simple, so the rules are simple.
> But when I feed a about 20M-size data file to the parser, the parser eats
> almost  600M+ memory.
> I am surprised by this result and I found most memory and time were consumed
> by the Lexer.
>
> Is there anything wrong with my grammar or it is the performance issue of
> ANTLR3 C++ runtime?
> I hope there is some way to get my parser  more lightweight.
>
> I attached the .g file to this mail and the data file(.txt format) could be
> got from this link:https://download.yousendit.com/Q01FSU5ONEhZY1IzZUE9PQ
>
> The test main function is:
> ========================================
>      int main(int argc, char * argv[])
>      {
>         pANTLR3_INPUT_STREAM           input;
>         pExpressDataLexer               lex;
>         pANTLR3_COMMON_TOKEN_STREAM    tokens;
>         pExpressDataParser              parser;
>
>         input  = antlr3AsciiFileStreamNew          ((pANTLR3_UINT8)argv[1]);
>         lex    = ExpressDataLexerNew                (input);
>         tokens = antlr3CommonTokenStreamSourceNew  (ANTLR3_SIZE_HINT,
> TOKENSOURCE(lex));
>         parser = ExpressDataParserNew               (tokens);
>
>         parser  ->syntax(parser);
>
>         parser ->free(parser);
>         tokens ->free(tokens);
>         lex    ->free(lex);
>         input  ->close(input);
>
>         return 0;
>      }
>

-- 
Sent from my mobile device