[antlr-interest] Re: Lexer speed comparison

Terence Parr parrt at cs.usfca.edu
Thu Feb 26 10:10:59 PST 2004


On Feb 26, 2004, at 8:37 AM, Uli Bubenheimer wrote:
> I did not set testLiterals=false. My grammar has keywords.

One can turn off for grammar and then turn on again in the IDENTIFIER 
rule so that it changes token types for keywords properly. :)

>  I guess I
> could alternatively recognize them by setting the lookahead to
> something like k=10. But would such a large lookahead really speed
> things up?

Yeah, try it out if you have time with the testLiterals thing changed 
per the above.

Terence

> Unfortunately I am not able to publish the proprietary specification I
> used (it's a real-world grammar that my company uses for its
> products). But as I indicated, it is not terribly sophisticated. I
> would expect similar results, for example, with a lexer spec. for (a
> subset of) C or Java. Perhaps someone would be interested in running
> such tests?
>
> Uli
>
> --- In antlr-interest at yahoogroups.com, Terence Parr <parrt at c...>
> wrote:
>> Hi Tom,
>>
>> Good questions.
>>
>> Also, did you have testLiterals=false in your Lexer options?  WIth
> that
>> as true, it does a hashtable lookup for each token.
>>
>> Ter
>> On Feb 25, 2004, at 5:41 PM, Tom Moog wrote:
>>
>>>
>>> a. Did you use flex -F or -f ?
>>>
>>> b. Did you try to eliminate I/O time in some way (e.g. by parsing
>>> from memory rather than a file) ?
>>>
>>> c. Did the java tools time require 8-bit => 16 bit conversion or
>>> were they 8 bit character scanners or was code conversion done
>>> only once before the timing loop ?
>>>
>>> d.   Did you allow "warm-up" time for the jit ?
>>>
>>> Tom
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
--
Professor Comp. Sci., University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com
Cofounder, http://www.knowspam.net enjoy email again!
Cofounder, http://www.peerscope.com pure link sharing





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list