[antlr-interest] nodes, hidden tokens, garbage collection

[Ric Klaren]

Hi,

On Wed, Apr 03, 2002 at 12:42:06PM -0800, John Allen Green wrote:
> Oi, I was hoping to avoid doing any surgery on Antlr. Getting rid of the
> reference counting would be very nice, but I think it would be a significant
> task.

Well I was planning to do some major surgery anyway ;)

> How were you planning to implement the memory pool? Use a vector<AST>, or
> something lower level? There's always the Boehm GCC route, as well.

First step custom would be adding a custom new/delete. Then problably do or
a subclassing trick, or some template trick to select the desired memory
pool (and/or the Boehm GC, I'm btw not sure how well the Boehm GC handles
circular things...). Lately I've been experimenting with a memory pool
implementation found in the Efficient C++ book from Bulka and Mayhew.

> Were you planning to eliminate RefToken and RefAST by replacing them with
> pointers?

At least RefAST would dissappear it's causing most 'problems' (or kludges)
and indeed it would become a plain old pointer. RefToken is less
troublesome (no heterogenous tokens and stuff like that).

> There's another option for my specific needs. There's a style of reference
> counting which uses two counters - one for strong references and one for
> weak references. Done right, it eliminates problems with cyclic
> dependencies. I've read about it, but I haven't found an example
> implementation yet. I could use this easily for my AST, but to implement it
> for the hidden tokens, I would have to hack or replace one or two chunks of
> the Antlr library.

Can't you use your own flavour of hidden tokens? I just had a small look at
the current implementation and I think it can be changed a little to solve
the reference counting problem. (basically play dirty in a few places and
store pointers to the objects in stead of using the RefToken) 

Or do it less dirty and use a single linked list of visible tokens and in
the visible tokens you store a vector (or single linked list) with all
hidden tokens before/after the visible one. I don't see why a double linked
list is necessary there (probably there because of direct port from java).

Another possibility is adding a custom new/delete to the hidden token
implementation. Allocate them from a pool attach the pool to the
lexer/parser and delete the pool with the lexer/parser object.

If you switch to another scheme of reference counting (you btw got a reference
on that style of ref counting?) you may have to do some antlr surgery.

HTH,

Ric
--
-----+++++*****************************************************+++++++++-------
    ---- Ric Klaren ----- klaren at cs.utwente.nl ----- +31 53 4893722  ----
-----+++++*****************************************************+++++++++-------
     Human beings, who are almost unique in having the ability to learn
   from the experience of others, are also remarkable for their apparent
         disinclination to do so. --- Douglas Adams, Last Chance to See


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/