[antlr-interest] FEATURE REQUEST: continued and memory/speed issues

[Ric Klaren]

On Wed, 29 Dec 2004 23:54:19 -0800 (PST), Akhilesh Mritunjai
<virtualaspirin at yahoo.com> wrote:
> Infact a lot of methods will need to be made virtual.
> For our project we needed a custom AST class with
> parent pointer. The standard method of making a custom
> factory class failed due to lack of virtual functions
> and now we are doing special dirty work in
> setNextSibling(), setFirstChild() functions of the
> custom AST class.

Being a bit more specific might be helpfull. The ASTFacotry does a lot
of non factory stuff like tree building basically I can make
everything virtual and provide all the rope to potentially hang
yourself ;) I guess I'll do that when I'm back at my  home...

> Additionally, I'd request you to make tokenNames and
> getTokenName variable/method in generated parser file
> as static (like in java code). Right now there is no
> way to access these methods without instantiating the
> parser (and lexer and zillion things), which seems a
> little painful for printing out just a token name in a
> debug statements.

Ack good you remind me. Had this request before but forgot about it.
Bit of a hectic time at the moment for me so I don't have too much
'antlr-time'. I'll see if I can change that for 2.7.5 if changes are
too big I'll probably make the changes for 2.7.6 in that case keep an
eye on snapshot releases to see when it's added (and remind me if
necessary)

> I'd second the request for moving most functions in
> header files for better inlining. We have made a good
> big grammar file and generated both Java and C++
> parser from it. On an 8 MB input file, the Java parser
> consumed about __33%__LESS__ memory (Java 217 MB / C++
>  372 MB, figures include JVM size, taken straight from
> /proc/<PID>/status/VmSize) and is TWICE as fast (Java
> 33.4 sec, C++ 68 sec). Compilers used were :
> * Java : Sun JDK 1.5 for RH linux 9.0
> * C++ : GNU gcc 3.4.3 for RH linux 9.0.
>   Flags used : -O3 -fexceptions -ffast-math
> -march=pentium4 -fexpensive-optimizations

Using more inlining might probably increase the memory footprint.
Results in the past have shown that it really depends on the grammar
wether Java/C++ performs better (at least in the past it was like
that) The C++ version has performace issues (but so has the java one)
It would take really structural changes to improve things. Inlining
changes usually are usually low priority for me. If you can tell me
which specific ones are the main problems I'll change them. (I'm low
on time so the more specific you are the easier and faster I can make
changes. I don't go on 'random' change sprees short before a release)

Tinkering with optimization options might give some gain (-O3
-fexpensive-optimiaztions etc. might not be optimal for your case)
Real gains for C++ can be gotten by using flex as lexer but that also
depends on your grammar.

Cheers,

Ric