[antlr-interest] Bug in column information in lexer?

[Ric Klaren]

Hi,

On Wed, Jun 19, 2002 at 03:28:14PM +0200, Silvain Piree wrote:
> I'm using a lexer to implement a preprocessor where line and 
> column information needs to be tracked.
> 
> I've encountered a problem where column information is incorrect.
> 
> I analyzed the problem and traced it back to method "rewind" 
> in class CharScanner:
> 
>     public void rewind(int pos) {
>         inputState.input.rewind(pos);
>         setColumn(inputState.tokenStartColumn);
>     }
> 
> Each time the lexer calls rewind, the column is set to an incorrect
> value. Without the setColumn() call everything works fine.
> 
> Anybody know what this method does and why it changes
> column information?

The mark/rewind pair is used for the guessing mode (at the start of a 
( .. ) => guessing mode is entered the spot is 'marked' and at a later
stage the inputstream is rewind to the start position)

C++ mode does not touch the column information in CharScanner.rewind. Also
I did not notice any anomalies with regard to column position. Is this a
Java mode bug?

Try removing the setColumn line. In general the lexer should not touch
line/column info during guessing so resetting should be redundant.

Cheers,

Ric
-- 
-----+++++*****************************************************+++++++++-------
    ---- Ric Klaren ----- klaren at cs.utwente.nl ----- +31 53 4893722  ----
-----+++++*****************************************************+++++++++-------
 'And this 'rebooting' business? Give it a good kicking, do you?' 'Oh, no,
  of course, we ... that is ... well, yes, in fact,' said Ponder. 'Adrian
    goes round the back and ... er ... prods it with his foot. But in a
     technical way,' he added. --- From: Hogfather by Terry Pratchett.


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/