[antlr-interest] unget a character

[Paul J. Lucas]

On Thu, 23 Dec 2004, Terence Parr wrote:

> My only experience is javadoc like stuff where the sentinel tokens are used
> to merely switch lexers not rewind switch.

	Right, and, normally, this is what I do; but I want to handle
	the abnormal case for recovery.  The normal case is:

		{ <foo/> }

	i.e., XML embedded inside { } (this is XQuery).  I have two
	lexers: one for the "main" part of XQuery, and a second for the
	"XML" part of XQUery.  Above, encountering a '<' switches to the
	XML lexer and "/>" switches back to the main lexer which then
	sees '}'.

	But the abnormal case is where the user makes a mistake like:

		{ <foo> }

	i.e., the user forgot the '/'.  Hence, the XML lexer never
	switches back and the main lexer, so the XML lexer sees the '}'
	by mistake.

	I've actually managed to get this to work simply by duplicating
	the code that the main lexer does upon '}' (it's only a single
	line of code) so, while not ideal, it works.

	- Paul