[antlr-interest] tab handling in ANTLR
Terence Parr
parrt at jguru.com
Sat Dec 22 17:32:41 PST 2001
Folks,
Currently, you have to implement method tab() in your lexer to get it to
track tabs as anything other than 1 char. I did this for flexibility.
You might want tabs to count in some weird way. The common case isn't
handled well though where you want it to count tabs as 4 or 8 or
whatever columns. Here is what I said in the 2.7.1 release notes:
> added column tracking support; tabs are counted as 1 unless you
> override
> tab(). Called from consume(); bumps by one by default. Overhead is
> minimal; only called on tabs. extra increment for all consume()s now
> extra int in CommonToken now.
>
> /** advance the current column number by an appropriate amount.
> * If you do not override this to specify how much to jump for
> * a tab, then tabs are counted as one char. This method is
> * called from consume().
> */
> public void tab() {
> // update inputState.column as function of
> // inputState.column and tab stops.
> // For example, if tab stops are columns 1 and 5 etc...
> // and column is 3, then add 2 to column.
> inputState.column++;
> }
>
> added CharScanner.setColumn
So, the question is: "should I add typical case tab handling"? I.e.,
should I add:
protected int tabsize = 8;
public void tab() {
int c = getColumn();
int nc = ( ((c-1)/tabsize) + 1) * tabsize + 1; //
calculate tab stop
setColumn( nc );
}
public int setTabsize( int size ) {
int oldsize = tabsize;
tabsize = size;
return oldsize;
}
to the default Lexer stuff? Speak now or forever hold your peace as
they say...I'm rapidly trying to get thru my "fix it" list for 2.7.2. :)
Ter
--
Chief Scientist & Co-founder, http://www.jguru.com
Creator, ANTLR Parser Generator: http://www.antlr.org
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list