[antlr-interest] More, Status of C++ backend?

[Ruslan Zasukhin]

On 2/1/08 1:53 PM, "Mark Wright" <markwright at internode.on.net> wrote:

Hi Mark,

>>> Another idea is:  maybe it might be easier to find a way to
>>> optionally plug in Andrei Alexandrescu's flex_string instead of
>>> std::string. flex_string is used in the Boost Wave project,
>>> presumably for the same reason.
>> 
>> But does flex_string require allocation by new()?
>> I assume that YES.
> 
> Hello Ruslan,
> 
> flex_string uses policy based design, so you can choose
> whether you want it to use new() or your own allocator:
> 
> http://www.ddj.com/cpp/184403784

Okay, although note that std::string also allow you specify own allocator.
Right? Problem is in time we all need spend to find good mem pool, then plug
them all together into our projects.

DREAM IS: that ANTLR C++ itself include some good mem-pool from the box for
its AST nodes and allow us to use this mem pool for our own nodes also.

>> And again, profiler show main problems from this new/delete =>
>> malloc()/free() allocations.
> 
> There are platforms such as Windows and Solaris where the
> default malloc()/free() implementations have performance
> issues with:
> 
> - multi-threading mutexes.  On Solaris you can use libmtmalloc instead.
> - the underlying allocator.

Right. As well as even such simple thing as allocation of small-size objects
or objects of the same fixed size. When pool know such limits it can be more
effective. 
 
> Since you can choose to use your own allocator with flex_string,
> I think there is a good chance that flex_string using
> a per-thread memory pool (hence no mutex overhead) and a
> custom allocator could outperform implementations
> that do not have these features.

 
>> Stack of profiler show deep for our SQL parser e.g. On 25-30 methods.
>> Each method itself eat e..g. 0.9% of time.
>>     0.8% is in fact LA() calls.
>> 
>> So there is no obvious bottleneck place. Its spread over all calls.
>> 
>> In SqlLite's Lemon parser deep of stack is only about 7-10 methods.
>> And I am sure they do not do this overhead allocations.
>> 
>> 
>> --------
>> Aha, I see what you mean, Mark.
>> 
>> Probably flex_string uses POINTERs and do not copy inside?
> 
> You can choose the policies.
> 
>> Well, this may speed up 50% of potential ..
>> 
>> I will try to check this way. Should be relatively easy.
>> 
>> 
>> ---------
>> And one more thing.
>> 
>>     MEMORY POOL
>> 
>> I believe that C/C++ runtimes of ANTLR should be armed by this things
>> always.
> 
> This is easy to plug into flex_string.

> I think you will like Andrei's article.

:-) 

I will read of course. Also just yesterday I have see mention that latest
boost has mem pool also.

Up to know I have spent time with mem pool from old good compiller
Metrowerks CodeWarrior. Apps made with Codearrior have win few times Visual
C++ and GCC apps thanks to this.


-- 
Best regards,

Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc

Valentina - Joining Worlds of Information
http://www.paradigmasoft.com

[I feel the need: the need for speed]