[antlr-interest] Antlr C code memory leak in Antlr 3.1.1

Tue Nov 11 18:10:08 PST 2008

Hi Jim

=>
 pANTLR3_INPUT_STREAM lInput=NULL;
 pANTLR3_COMMON_TOKEN_STREAM lPredigestTokenStream=NULL;
 pPLSQLpredigestParser lPredigestParser=NULL;
 pPLSQLpredigestLexer lPredigestLexer=NULL;
 lInput = antlr3NewAsciiStringInPlaceStream((pANTLR3_UINT8)SQL.c_str(), (ANTLR3_UINT64)SQL.size(), NULL);
 if(!lInput)
 {
  cout<<" input error"<setUcaseLA(lInput,ANTLR3_TRUE);

 lPredigestLexer = PLSQLpredigestLexerNew(lInput);
 if(!lPredigestLexer)
 {
  cout<<" lPredigestLexer error"<start_rule(lPredigestParser);

 lPredigestParser->free(lPredigestParser);
 lPredigestTokenStream->free(lPredigestTokenStream);
 lPredigestLexer->free(lPredigestLexer);
 lInput->close(lInput);
 lPredigestParser=NULL;
 lPredigestTokenStream=NULL;
 lPredigestLexer=NULL;
 lInput=NULL;

=>This is my all source code, and memory leak exist in PLSQLpredigestParser_start_rule_return psr = lPredigestParser->start_rule(lPredigestParser);

Actually, I have two type PLSQL's lexer and parser code from two type PLSQL g file. 

But only this file (PLSQLPredigest.g,I send to you before)exist memory leak (the source PLSQL.g is from website, and no memory leak issue,PLSQLpredigest is predigested from PLSQL.g) , and I seems to follow the "treeparser" sample to code (because I don't have any walker to new a node "antlr3CommonTreeNodeStreamNewTree"). 

Thank Jim.

Best Regards
Winfly Lin

> Subject: Re: [antlr-interest] Antlr C code memory leak in Antlr 3.1.1
> From: jimi at temporal-wave.com
> To: merry_winfly at hotmail.com
> CC: antlr-interest at antlr.org
> Date: Tue, 11 Nov 2008 08:52:30 -0800
> 
> On Tue, 2008-11-11 at 13:39 +0000, merry_winfly at hotmail.com wrote:
>> 
>> ==>
>> Actully, I find when (lPredigestParser->free(lPredigestParser);) the adapter and vector will be freed by this free action, but at last some tokenstream didn't be freed, I don't know if my source code issue or antlr's bug.
>> 
>> And any one know this, pls freely tell me the reason.
> 
> Did you call free on the token stream itself? I think that this is your
> issue. There is a treenode stream that you must free too. Check against
> that example tree parsers, but I will look at your issue today.
> 
> Jim
> 
>> 
>> Thanks a lot.
>> 
>> Best Regards
>> Winfly Lin
>> 
>> 
>> 
>> From: merry_winfly at hotmail.com
>> To: antlr-interest at antlr.org
>> Date: Mon, 10 Nov 2008 13:21:04 +0000
>> Subject: [antlr-interest] Antlr C code memory leak in Antlr 3.1.1
>> 
>> 
>> 
>> Hi All,
>> 
>> When I use the C code from antlr generator with predigest PLSQL parser , I find it exist obvious memory leak issue.
>> 
>> I use valgrind to check and the result is:
>> 
>> 
>> valgrind --leak-check=full ./treeMemory
>> 
>> ==9541== Memcheck, a memory error detector.
>> 
>> ==9541== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
>> 
>> ==9541== Using LibVEX rev 1854, a library for dynamic binary translation.
>> 
>> ==9541== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
>> 
>> ==9541== Using valgrind-3.3.1, a dynamic binary instrumentation framework.
>> 
>> ==9541== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
>> 
>> ==9541== For more details, rerun with: -v
>> 
>> ==9541==
>> ==9541==
>> ==9541== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 17 from 1)
>> ==9541== malloc/free: in use at exit: 8,008 bytes in 98 blocks.
>> ==9541== malloc/free: 8,847 allocs, 8,749 frees, 2,202,767 bytes allocated.
>> ==954 1== For counts of detected errors, rerun with: -v
>> ==9541== searching for pointers to 98 not-freed blocks.
>> ==9541== checked 149,260 bytes.
>> ==9541==
>> ==9541== 8,008 (7,560 direct, 448 indirect) bytes in 90 blocks are definitely lost in loss record 3 of 3
>> ==9541== at 0x40057C8: malloc (vg_replace_malloc.c:207)
>> ==9541== by 0x8055BFA: antlr3RewriteRuleElementStreamNewAE (antlr3rewritestreams.c:94)
>> ==9541== by 0x8055DA0: antlr3RewriteRuleTOKENStreamNewAE (antlr3rewritestreams.c:192)
>> ==9541== by 0x80B8125: select_list (PLSQLpredigestParser.c:51593)
>> ==9541== by 0x80B4E0F: select_expression (PLSQLpredigestParser.c:50283)
>> ==9541== by 0x80B3F4D: select_statement (PLSQLpredigestParser.c:49890)
>> ==9541== by 0x810C0F5: insert_command (PLSQLpredigestParser.c:83912)
>> ==9541== by 0x80B1E12: sql_command (PLSQLpredigest Parser.c:49133)
>> ==9541== by 0x80B13BE: sql_statement (PLSQLpredigestParser.c:48855)
>> ==9541== by 0x8083185: statement (PLSQLpredigestParser.c:31331)
>> ==9541== by 0x808110E: seq_of_statements (PLSQLpredigestParser.c:30542)
>> ==9541== by 0x8077D5C: start_rule (PLSQLpredigestParser.c:26944)
>> ==9541==
>> ==9541== LEAK SUMMARY:
>> ==9541== definitely lost: 7,560 bytes in 90 blocks.
>> ==9541== indirectly lost: 448 bytes in 8 blocks.
>> ==9541== possibly lost: 0 bytes in 0 blocks.
>> ==9541== still reachable: 0 bytes in 0 blocks.
>> ==9541== suppressed: 0 bytes in 0 blocks.
>> 
>> Source Code:
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> string SQL="insert into T K(a1,a2,a3) select a,b,c from t2 where a>9 union all select e,f,g from t5 where e>'winfly' ;";
>> 
>> pANTLR3_INPUT_STREAM lInput=NULL;
>> pANTLR3_COMMON_TOKEN_STREAM lPredigestTokenStream=NULL;
>> pPLSQLpredigestParser lPredigestParser=NULL;
>> pPLSQLpredigestLexer lPredigestLexer=NULL;
>> lInput = antlr3NewAsciiStringInPlaceStream((pANTLR3_UINT8)SQL.c_str(), (ANTLR3_UINT64)SQL.size(), NULL);
>> if(!lInput)
>> {
>> cout<<" input error"> 
>> lPredigestLexer = PLSQLpredigestLexerNew(lInput);
>> if(!lPredigestLexer)
>> {
>> cout<<" lPredigestLexer error"> 
>> lPredigestParser->free(lPredigestParser);
>> lPredigestTokenStream->free(lPredigestTokenStream);
>> lPredigestLexer->free(lPredigestLexer);
>> lInput->close(lInput);
>> 
>> 
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
>> Actually the memory leak exist PLSQLpredigestParser_start_rule_return psr = lPredigestParser->start_rule(lPredigestParser);
>> 
>> I don't know why so simple code cause so obvious memory leak
>> 
>> 
>> Could you help me to solve this? (I'm hurry for this code, thanks a lot!)
>> 
>> I list the information and attachment includes all code about it (C file ):
>> 
>> Platform: RedHat Enterprise 5 Server, virtual machine.
>> GCC version: 4.1.1
>> Antlr Version: 3.1.1
>> 
>> 
>> 
>> 
>> I don't know if it is enough information to trace this problem, but if you want to know any more information, pls freely contact me with this E-mail (merry_winfly at hotmail.com).
>> 
>> 
>> Thanks a lot.
>> 
>> 
>> Best Regards
>> Winfly Lin
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Get news, entertainment and everything you care about at Live.com. Check it out!
>> _________________________________________________________________
>> Connect to the next generation of MSN Messenger 
>> http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-us&source=wlmailtagline
>> 
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>> 
> 
_________________________________________________________________
Connect to the next generation of MSN Messenger 
http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-us&source=wlmailtagline