[antlr-interest] Lexer speed comparison

[Tom Moog]

a. Did you use flex -F or -f ?

b. Did you try to eliminate I/O time in some way (e.g. by parsing
from memory rather than a file) ?

c. Did the java tools time require 8-bit => 16 bit conversion or
were they 8 bit character scanners or was code conversion done
only once before the timing loop ?

d.   Did you allow "warm-up" time for the jit ?

Tom


On Wed, 25 Feb 2004, Uli Bubenheimer wrote:

> I ran some tests comparing Lexers generated by different tools in
> terms of speed. The sophistication of my Lexer specification that I
> used as input for the Lexer generators is about that of a lexer for C.
> The average numbers I found by repeatedly running the generated lexers
> on a large source file were these:
>
> ANTLR           8300ms
> JLex            6500ms
> JFlex 1.4pre5   2000ms
> Flex            1900ms
>
> I used current versions of all tools. For the ANTLR comparison I had
> to adjust the Lexer specification, but tried to make it as efficient
> as possible. For the Flex comparison, I basically called Flex's
> yylex() function from Java via JNI. Also in the case of Flex I
> generated fewer Java objects as output than for the others, and if I
> take that into account, Flex's time would probably be more similar to
> JFlex's.
>
> I was surprised to find that ANTLR generated the slowest lexer.
> Naturally, when interpreting these numbers keep in mind that the
> outcome may have been different on a different lexer specification or
> lexer input text. On the other hand, it makes sense to me that a DFA
> would be faster on average than a more powerful recursive-descent parser.
>
> Are there other performance numbers available? I have not been able to
> dig up anything substantial. I also wonder how ANTLR-generated LL(k)
> parsers perform compared to LALR(1) parsers.
>
> Uli
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/