[antlr-interest] Problems while reusing Lexer/Parser/TreeParser instances

micheal_jor open.zone at virgin.net
Wed Sep 24 22:14:17 PDT 2003


I'm tinkering with the code below[1*] to support reuse of
Lexer/Parser/Treeparser instances with ANTLR/C#:

1. I haven't tried this in a basic Lexer->Parser situation ;-)

2. Given a situation (a.k.a. pipeline) where Lexer1 and Lexer2 are
multiplexed using a TokenStreamSelector that then feeds a Parser, only
the first use of an instance of the pipeline is error-free. When
parsing the same file repeatedly, the pipeline reports unexpected
token errors on pass two and dies catastrophically on pass 3.

Any ideas?

I reset the state in (2) above as follows:
1. lexer1.resetState(<new-buffer-to-use>)
2. lexer2.resetText()
3. parser.resetState()

The TokenStreamSelector doesn't seem to need any tweaking.

Does (and how) the use of exotic Token classes like HiddenStreamToken
(sp?) affect reuse of L/P/T instances?




[1*] Code excerpts below:

> 		public virtual void resetState(InputBuffer ib)
> 		{
> 			text.Length = 0;
> 			traceDepth = 0;
> 			inputState.resetInput(ib);
> 		}

> 		public virtual void resetState()
> 		{
> 			traceDepth = 0;
> 			inputState.reset();
> 		}


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 

More information about the antlr-interest mailing list