[antlr-interest] String size limit and how to free memory real-time?

[Ric Klaren]

Hi,

On 11/10/06, Rick Morgan <r.morgan at verizonbusiness.com> wrote:
> Delving further into my problem I found that the stack overflow occurs when
> the parser destructor is called.  It attempts to free the AST which in my
> case is a degenerate list of some sort, containing 25800 nodes.

This is a known gotcha. It seems that more and more people are running into it.

See also this link:

http://www.antlr.org/pipermail/antlr-interest/2006-July/016967.html

> so haven't got back to this.  Any suggestions on how to free up the AST
> real-time would be helpful, as well as warnings about corrupting it all.

Some quick copy paste & modify from the above link and the exprAST
example in the distro:

-----snip-----
options {
        language="Cpp";
}

{
#include <iostream>
#include <queue>

ExprParser::~ExprParser()
{
        destroy(returnAST);
}

using antlr::RefAST;
using antlr::nullAST;

void ExprParser::destroy( RefAST ast )
{
        if( ! ast )
                return;

        // std::cerr << "before destroy" << std::endl;

        std::queue<RefAST> stack;

        stack.push(ast);
        ast = nullAST;
        RefAST current;
        // while stuff left..
        while(!stack.empty()) {
                current = stack.front();
                stack.pop();

                RefAST child = current->getFirstChild();
                current->setFirstChild(nullAST);

                if( child )
                        stack.push(child);

                child = current->getNextSibling();
                current->setNextSibling(nullAST);

                if( child )
                        stack.push(child);

                current = nullAST;
                // std::cout << "size = " << stack.size() << std::endl;
        }
}
}

class ExprParser extends Parser;
options {  buildAST=true; }
{
public:
   // destroy function...
   void destroy(antlr::RefAST ast);
   // custom destructor...
   virtual ~ExprParser();
}

// rest of exprAST.g from example
----snip----

Usual disclaimers apply, I did not really test this very well, so I
might have introduced an error in the original code from Eugen.

You would have to apply the above pattern to all Parsers/TreeParsers
in your project that build trees.

Cheers,

Ric