[antlr-interest] v2->v3 Skip chars in Lexer? Terrence?

[Ruslan Zasukhin]

On 4/17/11 11:06 AM, "Ruslan Zasukhin" <ruslan_zasukhin at valentina-db.com>
wrote:

>> but basically it is easy to strip
>> leading and trailing characters as the tokens carry pointers, so get the
>> start pointer, increment it, get the end point, decrement it, now
>> 
>> Do not use the built in $token.text->chars as this is slow and just for
>> convenience. 
 
>> The token holds a pointer to the start of the text in the
>> original input stream, which is greatly faster and you don¹t do anything
>> at all to the token until and if you use it.

>> You know the token type, so can handle it appropriately.

Hmmm,

I have take a look, and I do not see way in C-target access token in lexer
rule.

Do you mean that I should care about these pointers LATER, in parser?

Butt hen this again looks as not best solution...
    Java developers will remove them in lexer,
    C developers in parser?

Some kind of Zoo ...


Please help   :-)

And note, that I am C++ developer with 20 years of experience,
    do all my best reading ANTLR WIKI and book,and examples,
    and which did work with ANTLR v2 for 10 years ...
    cannot resolve this *trivial* task in *the best way*
    for v3 for about 14 hours now.

I wonder how other C developers was able resolve this problem?

And may be docs, faqs, examples can be improved in this direction?
Thank you, in advance :-)

-- 
Best regards,

Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc

Valentina - Joining Worlds of Information
http://www.paradigmasoft.com

[I feel the need: the need for speed]