[antlr-interest] circular token buffer and overriding nextToken()

[netminka]

hi all,
I have tried overriding nextToken() because I need my lexer to return
dummy tokens at certain points and what happens is that the 
dummy token writes over a valid token in the token buffer, and my input
program is made incorrect. 

the k for my lexer is four. This problem happens when only one 
dummy token is inserted. It is erratic. 
the token buffer is circular and it's not clear to me where / how
I can safely insert a dummy token.

Can anyone who really understands this interface give me a clue?

Thanks VERY much.
megan 


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/