[antlr-interest] Re: Lexer - length/position as token delimiter?

[Mark Lentczner]

> However, because the parser is doing a lot of the work, which ideally 
> would be done by the lexical analyzer, we are concerned about 
> performance overhead.
Never prematurely optimize, I always say.  If your examples are at all 
indicative, I'd be surprised if there was any significant timing 
differences between approaches.

Unless, of course, you need to process some hugh number of records very 
quickly (relative to the target hardware), in which case you may need 
to make more drastic changes (Java -> C++ if you haven't already, or 
getting rid of Antlr and using a hand built lexer/parser pair.)

> I will complete the grammar for our sample message type and run a 
> batch of messages
> through it to get an idea of the performance.
That's the way to go.  If the timing doesn't meet some object measure 
of "fast enough" (which was determined **before** you ran the tests, 
yes?), then be sure to use a performance tool to see where the 
bottleneck is.  I wouldn't just assume that it is in the parser and 
that a more complicated scheme that moved the rules into the lexer 
would speed things up...

	- Mark

Mark Lentczner
markl at wheatfarm.org
http://www.wheatfarm.org/



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/