[antlr-interest] Re: feature request: Token.getOffset()

Anthony W Youngman Anthony.Youngman at ECA-International.com
Tue Dec 9 02:46:21 PST 2003


Hmmm ...

A "minimal expense" way of doing it at present would be to keep a
line-position count as you parse the line, then just add that to the
file-position count when you reset it in a newline.

Tabs are the fly in the ointment ... they increase the character count
by one, but the line count by some arbitrary amount ... I've always
hated tabs in text for that very reason.

Cheers,
Wol 

-----Original Message-----
From: lgcraymer [mailto:lgc at mail1.jpl.nasa.gov] 
Sent: 08 December 2003 21:04
To: antlr-interest at yahoogroups.com
Subject: [antlr-interest] Re: feature request: Token.getOffset()

--- In antlr-interest at yahoogroups.com, Ric Klaren <klaren at c...> wrote:
..
> (/grants loring permission to throw something at him) Hmmm so we can
get up
> to a 20% gain in our lexers by kicking out the column stuff ;) ? Or
move
> the checking of tabs out of the CharScanner::consume method and handle
them
> similarly to newline by explicit calls in the lexer?

The 20% would require heavy optimization of the lexer--right now, ANTLR
lexers are grossly inefficient.  Handling tab() like newline(), 
though, would be a good idea.  line/column information is generally
useful (it's hard to imagine a production batch translator that didn't 
need the line/column error reporting for text processing.  Ter's working
on doing a few things that should significantly improve ANTLR 
lexer performance.

--Loring
  
> Cheers,
> 
> Ric
> --
>
-----+++++*****************************************************+++++++++
-------
>     ---- Ric Klaren ----- j.klaren at u... ----- +31 53 4893722  ----
>
-----+++++*****************************************************+++++++++
-------
>    Words fly like arrows
>       as if we knew what was right and wrong. --- Chuang Tsu


 

Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ 






***********************************************************************************

This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system.

Telephone numbers for ECA International offices are: Sydney +61 (0)2 9911 7799, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333.

***********************************************************************************


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list