[antlr-interest] [C] my v3 Parser no reuse() slower 20% than v2. With reuse() 2GB leaks, oops.

Ruslan Zasukhin ruslan_zasukhin at valentina-db.com
Thu Nov 17 12:43:11 PST 2011


Hi Jim,

Thank you  for your feedback, and I have update now...

1) I was able remove all  .text  usage in both Parser and TreeParser. GOOD.

2) BAD ... This have save 500MB,
   but I still have 1.5GB of allocations in my bench ...

And now I see (using Apple Instruments) that all this is eaten by PARSER.
Not by Lexer, and not by TreeParser.

I just see endless 
                    newPool
                    newPool
                    newPool
                    newPool
                    newPool
                    newPool

I will send you snapshoot off list so you can see that.

And now there is ZERO my code, which affect this.
Only ANTLR own logic...

This makes me think, that reuse() do not work as expected.

As I understand, when we do
        parser( reset )

It must mark all existed allocations as free in your pool,
So next run should reuse all that. Yes?

And note, that all my calls to parser, are very similar by size.
This is just 
    INSERT INTO( f1, f,2 ... f9 ) VALUES ( v1, v2, ... )

I.e. Pool really should not grow much after first / second iteration of
loop. But it grows like crazy.


I think you have own test app where you did test this ...
May be just increase loop count to million or such
To see that RAM on your computer go away ...


I very hope you will be able find issue and show how fix it
in sources of ANTLR 3.4 ...  Please?  May be some kind of objects is not
marked as free ?


======
Also interesting fact.

v3 without reuse                                 22.4 sec
v3 with reuse and 1.5GB allocation     20.4 sec

v2 with reuse                                      19.7


So if we will be able resolve this 1.5GB "leaks", there is yet hope to be at
least not slower of v2 ...


=======================
About your hope that V3 C should be much faster of v2 C++
So far I do not see this.

I see in profiles,
    parser                           36%    RAM only
    tree parser                    24%    RAM only
    execute of vdb engine  13%     insert recs into disk (!!) db

And when I am starting go deep by parser calls ... I just see that deep is
big  
        sql -> sql_single -> ....

And each step down just reduce 0.5-0.8%  ...

This is BODY of each rule of parser ...

And nothing really to optimize :(
Just a lots of small calls  ... NilNodes,  LT(), ...


=======================
My vision is that this is Nature of ANTLR ... We get many calls of parser
funcs ... Deep stack ... Although they are light they eat milliseconds ...


And fact that in C you need create structures with huge number of pointers
sometimes, then e.g NULL them,  in C++ virtual table of methods is created
once per class, not once per instances ... This fact can be one of hidden
bottleneck IMO. You can workaround this, if also will extract pointers into
single separate structure, so instances will have just a single pointer.


-- 
Best regards,

Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc

Valentina - Joining Worlds of Information
http://www.paradigmasoft.com

[I feel the need: the need for speed]




More information about the antlr-interest mailing list