[antlr-interest] (follow up) setting, altering text in lexer rules

[Don Caton]

Ter:

> ok, so lexing time when from 1340ms to 2095ms when I added 
> all this crap in there.  That is more than 1.5x the cost in 
> time.  I can't justify that 50% increase in lexer time.  Also 
> the parser is taking more time...weird...a GC issue?
> 
> Wow. the
> 
> text.setLength(0);
> 
> in the nextToken() method costs 200ms out of that 2095ms.  
> When I remove all this machinery it goes back to what it was 
> in my notes time-wise so it's not a fluke.  Rats!

Is it really important to worry about things like this?  Something that
takes 2095ms in the particular Java implementation you're using isn't
necessarily going to take 2095ms in C++ or C# or any other target language.
Or for that matter, any other particular Java implementation.

IMO, it might be better to determine what features are (and aren't)
necessary, and then leave it up to the individual language implementers to
figure out how to implement them in the target language in the most
efficient manner possible.

You're really wearing two hats here: the architect, and the Java
implementer.  I hope that when you're wearing the first hat you're designing
the best parser you can, not the best parser implemented in Java that you
can.  

And I imagine that anyone who needs serious, heavy duty parsing where every
ms counts isn't going to be using Java anyhow, they're going to be using
C++.

Don