[antlr-interest] Hand made lexer 600(!!!!!) times faster of ANTLR's one

Ruslan Zasukhin sunshine at public.kherson.ua
Thu Feb 24 01:18:48 PST 2005


Hi All,

Since again on this list raised performance issue
I want to point our recent discovery.

We have made simplest ANTLR lexer in 5 lines to parse
COMA/TAB separated files.
We use C++.

Making tests. 2 minutes for file on import.
We see that this is incredibly long.


In one day our developer make hand lexer,
And time is wow -- 0.2 seconds.

600 times guys!     600 times!!!


This is incredible.
This is very disappointed.

If to look into ANTLR then of course we see that xxxxxx

* LA() calls 

* creation of TOKENs that copy strings using std::string class
        
    in our version we simple return pointer to start and pointer
    to end of token in the PARSED text.
    - We do not copy any byte
    - we do not call any new operator
    - we do not create std::string objects
    - we do not destroy them
    - we do not call delete operators.
    
    instead of all above we have ZERO time.

* and probably exceptions...


I do not understand why in ANTLR C++ cannot be written small
string class in 100 lines of code which will increase speed of ANTLR in 10
times AT LEAST.

Many times I was blamed on this list;
    Hey Ruslan, at first go to profiler,
    and prove that ANTLR take more time than your SQL engine.

So guys, I can 100 times to say:

    ANTLR parser for SQL string take time LONGER than
    we can resolve after that query for table with million records!

Should I say anything more ?!           :-(

....................................................
I already do not know how to explain................
....................................................

Terrence, Rick, you have great cool parser.

    but it is NOT CORRECT to think that speed is not an issue.

Terrence, Rick, can you take into hands profiler and self to see what great
overhead bring us all above points?

Is it possible spend time to collect all advices on this list on code
optimization? I believe I have see at least 10 letters during last 2 years,
and each letter have told WE WAS ABLE INCREASE speed in times.

    It is just a shame to have so slow and so cool parser.
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:-) yes I know this letter looks to be emotional and angry.
    but I cannot understand why speed is the LAST ISSUE for ANTLR guys.

I hope that in nearest time we will get cool ANTLR with cool speed.
And slogan will be

    the best parser!
    the fastest parser!

-- 
Best regards,
Ruslan Zasukhin      [ I feel the need...the need for speed ]
-------------------------------------------------------------
e-mail: ruslan at paradigmasoft.com
web: http://www.paradigmasoft.com

To subscribe to the Valentina mail list go to:
http://lists.macserve.net/mailman/listinfo/valentina
-------------------------------------------------------------





More information about the antlr-interest mailing list