[antlr-interest] Re: Stream Positions
Trey Spiva
Trey.Spiva at embarcadero.com
Mon Jan 7 14:46:10 PST 2002
>> I was not able to just override the consume() method because the
>> lexer can backtrack adn teh same character position may be visted
>> many times.
>
>Yeah, I was wondering about that...did you have to override rewind in
>the buffers? Is that it?
No, when InputBuffer::fill is called I retrieved the character as well as
the file position. I then
stored both the character and the file position on to the character queue.
Of course I had to
make the queue take a structure not a character
class CharacterData
{
public:
CharacterData(int c, long fp)
{
m_character = c;
m_filepos = fp;
}
int getCharacter() {return m_character;}
int getFilePosition() {return m_filepos;}
private:
int m_character;
long m_filepos;
};
CircularQueue<CharacterData> queue;
Now InputBuffer::LA(int I) looks like this.
inline int InputBuffer::LA(int i)
{
fill(i);
CharacterData data = queue.elementAt(markerOffset + i - 1);
return data.getCharacter();
//return queue.elementAt(markerOffset + i - 1);
}
I also added a getPosition method to InputBuffer.
/** Get the file position of a lookahead character */
inline long InputBuffer::getPosition(int i)
{
fill(i);
CharacterData data = queue.elementAt(markerOffset + i - 1);
return data.getFilePosition();
}
I then added the position data to the input state inside the
CharScanner::comsume method.
>>> I had to alter CharBuffer, CharScanner, CommonToken, InputBuffer,
>>> LexerSharedInputState, Token
>>>
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list