[antlr-interest] v2->v3 Skip chars in Lexer? Terrence?

Ruslan Zasukhin ruslan_zasukhin at valentina-db.com
Sun Apr 17 01:13:19 PDT 2011


On 4/16/11 9:27 PM, "Jim Idle" <jimi at temporal-wave.com> wrote:

> It is for performance and has been talked about for 4 years, so we don't
> need to start it again.

> You know the token type, so
> can handle it appropriately. It is a trivial piece of code and you will
> want a generic method/function for getting the string anyway. It takes
> less time to implement it than to worry about ! not being there any more
> :-)

And since Sunday, I'd like to add my vision.   :-)


*  how easy was solution in v2.
        IDENT:  DQUOTE!  something DQUOTE!

    What could be easier? :-)


* also best here is that !  Did as for Java so for C
Now in v3, for each target developers must provide different code.
Good?  Nope.


* I did very expect that in v3 I will be able specify some re-write rule
like in parser. But no it not works.  And btw, it seems there is no any
mention of this in docs and book.

    Attention: re-write syntax do not works in Lexer v3.


* yes, I understand I think, that main technical problem is that Token tend
to point  p1/p2  to input buffer string to avoid COPY.  Yes speed  is my own
favor feature of product :-)

But your Token already can  have COPY if needed, right?
Then for RARE case-sensitive , it will be great to automated way.


-- 
Best regards,

Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc

Valentina - Joining Worlds of Information
http://www.paradigmasoft.com

[I feel the need: the need for speed]




More information about the antlr-interest mailing list