[antlr-interest] TokenStreamRewriteEngine and C++

Ric Klaren klaren at cs.utwente.nl
Tue Apr 20 09:32:40 PDT 2004


Hi,

On Tue, Apr 20, 2004 at 09:13:18PM +0700, Andrey R. Urazov wrote:
> Is it possible to release some snapshot sooner? If not, could you
> possibly pinpoint what's the problem with the RefToken since I'll need
> the functionality of a rewrite engine within two weeks and will have to
> implement it on my own if it's not yet available by the time.

I have the rewrite engine and the TokenWithIndex class it needs that should
basically compile. I can post these without much trouble (Loring is sitting
on the other C++ stuff for the parser testing I don't recall offhand, but
he might still be battling red tape somewhere to get it released)

> And also, out of curiosity, what's so sucky about these refcounters in
> general?

The problem at the moment is that the guys implementing the C++ support
library port decided to use two flavours of the the reference counter one
generic one and one especially for the AST's. The generic one is pretty
dumb (but does the trick for run-of-the-mill refcounting) The special one
for AST refcounts makes it possible to do some limited casting in the AST
class hierarchy so you can cast/convert CommonAST's to
YourFavouriteCustomAST. The support lib uses the dumb version for Tokens
and with a modern g++ it barfs pretty badly (there might have been a time
that it worked). In the current state even the simplest custom token class
stuff seems to fail (or I missed something when I was looking at it, I did
not have much time then)

In general the reference counter that are used now give loads of trouble
with polymorphism. E.g. when using custom AST's and especially when using
heterogeneous AST's. Also there's a can of worms as soon as you tag a const
somewhere near these refcounters.

Anyway my local copy of the antlr source is basically pretty far there.
What needs to happen is that the ASTRefCount style smart pointers need to
be ported to a TokenRefCount. I now have some tinyc example with
tokenstreamrewrite support (ported from the CSharp example which again is
probably a port of some example somewhere from Terence's site). Only
problem is that it segfaults and I did not have time yet for another
debug/cleanup session.

It is also a little bit different than the java version in the sense that I
replaced the StringBuffer with io-streams which is more the C++ way for
this style of stuff.

I can give this stuff to you without much trouble, but in the current state
it's *very* rough so only start on it if you're not afraid to loose sanity.
I will probably not have much time to help with it in the coming week/2
weeks.

> By the way, does it make sense to your opinion to use smart pointers
> (shared pointers, in particular) from the boost library? Since boost is
> widely used by practitioners and some parts of it are considered as
> upcoming standards, I think that if functionality of the smart pointers
> from boost suits the needs of project, it's appealing to use them.

I'm not 100% sure but I think there's no smartpointer in C++ possible
that's smart enough to behave like the pointer I'd like for heterogenous
AST's. I've already looked at the boost smart pointers for a few times (and
some other implementations (Loki among them)) but I don't think they'll
solve the actual problem (that it's an object pretending to be a pointer)

> Thanks a lot for the work you do!

You're welcome!

Anycase drop me a line if you're interested in the segfaulting version.

Cheers,

Ric
--
-----+++++*****************************************************+++++++++-------
    ---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893722  ----
-----+++++*****************************************************+++++++++-------
 Why don't we just invite them to dinner and massacre them all when they're
  drunk? You heard the man. There's seven hundred thousand of them. Ah? ..
           So it'd have to be something simple with pasta, then.
                 From: Interesting Times by Terry Pratchet



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list