[antlr-interest] tab handling in ANTLR
Matthew Ford
Matthew.Ford at forward.com.au
Sat Dec 22 18:45:39 PST 2001
At the very least this code should be in the comments/docs so people know
what to do.
matthew
----- Original Message -----
From: "Terence Parr" <parrt at jguru.com>
To: "ANTLR mailinglist" <antlr-interest at yahoogroups.com>
Sent: Sunday, December 23, 2001 12:32 PM
Subject: [antlr-interest] tab handling in ANTLR
> Folks,
>
> Currently, you have to implement method tab() in your lexer to get it to
> track tabs as anything other than 1 char. I did this for flexibility.
> You might want tabs to count in some weird way. The common case isn't
> handled well though where you want it to count tabs as 4 or 8 or
> whatever columns. Here is what I said in the 2.7.1 release notes:
>
> > added column tracking support; tabs are counted as 1 unless you
> > override
> > tab(). Called from consume(); bumps by one by default. Overhead is
> > minimal; only called on tabs. extra increment for all consume()s now
> > extra int in CommonToken now.
> >
> > /** advance the current column number by an appropriate amount.
> > * If you do not override this to specify how much to jump for
> > * a tab, then tabs are counted as one char. This method is
> > * called from consume().
> > */
> > public void tab() {
> > // update inputState.column as function of
> > // inputState.column and tab stops.
> > // For example, if tab stops are columns 1 and 5 etc...
> > // and column is 3, then add 2 to column.
> > inputState.column++;
> > }
> >
> > added CharScanner.setColumn
>
> So, the question is: "should I add typical case tab handling"? I.e.,
> should I add:
>
> protected int tabsize = 8;
>
> public void tab() {
> int c = getColumn();
> int nc = ( ((c-1)/tabsize) + 1) * tabsize + 1; //
> calculate tab stop
> setColumn( nc );
> }
>
> public int setTabsize( int size ) {
> int oldsize = tabsize;
> tabsize = size;
> return oldsize;
> }
>
> to the default Lexer stuff? Speak now or forever hold your peace as
> they say...I'm rapidly trying to get thru my "fix it" list for 2.7.2. :)
>
> Ter
> --
> Chief Scientist & Co-founder, http://www.jguru.com
> Creator, ANTLR Parser Generator: http://www.antlr.org
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list