[antlr-interest] Re: Problems while reusing Lexer/Parser/TreeParser instances

[micheal_jor]

In my scheme, the resetState() method for Lexers takes a new
Stream/InputBuffer. The actual input file/stream/buffer doesn't have
to be reset-able, just the internal state of the Lexer. Of course the
"new" stream/InputBuffer may have been opened on the same file as the
current one.

Do you have code that successfully supports Lexer/Parser/TreeParser reuse?

Cheers,

Micheal

> Seems like there will be a file closing/rewinding issue now that you 
> mention it ;)  Somehow the actual file must be reset not just the 
> internal state.
> 
> Ter
> 
> 
> On Saturday, September 27, 2003, at 09:11 AM, micheal_jor wrote:
> 
> > Would someone please post working code for reusing
> > Lexer/Parser/TreeParser instances please?. It can be in any of C++,
> > Java and C#.
> >
> > Cheers
> >
> > Micheal
> >
> >> Hi,
> >>
> >> I'm tinkering with the code below[1*] to support reuse of
> >> Lexer/Parser/Treeparser instances with ANTLR/C#:
> >>
> >> Issues:
> >> 1. I haven't tried this in a basic Lexer->Parser situation ;-)
> >>
> >> 2. Given a situation (a.k.a. pipeline) where Lexer1 and Lexer2 are
> >> multiplexed using a TokenStreamSelector that then feeds a Parser,
only
> >> the first use of an instance of the pipeline is error-free. When
> >> parsing the same file repeatedly, the pipeline reports unexpected
> >> token errors on pass two and dies catastrophically on pass 3.
> >>
> >> Any ideas?
> >>
> >> I reset the state in (2) above as follows:
> >> 1. lexer1.resetState(<new-buffer-to-use>)
> >> 2. lexer2.resetText()
> >> 3. parser.resetState()
> >>
> >> The TokenStreamSelector doesn't seem to need any tweaking.
> >>
> >> ***
> >> Does (and how) the use of exotic Token classes like HiddenStreamToken
> >> (sp?) affect reuse of L/P/T instances?
> >>
> >> Cheers,
> >>
> >> Micheal
> >>
> >> ------------------
> >>
> >> [1*] Code excerpts below:
> >>
> >>
> >> -----------------------------------------------------------
> >> CharScanner.cs
> >> -----------------------------------------------------------
> >> 520a523,529
> >>> 		public virtual void resetState(InputBuffer ib)
> >>> 		{
> >>> 			text.Length = 0;
> >>> 			traceDepth = 0;
> >>> 			inputState.resetInput(ib);
> >>> 		}
> >>>
> >>
> >> -----------------------------------------------------------
> >> Parser.cs
> >> -----------------------------------------------------------
> >> 283a284,289
> >>> 		public virtual void resetState()
> >>> 		{
> >>> 			traceDepth = 0;
> >>> 			inputState.reset();
> >>> 		}
> >>>
> >
> >
> >
> >
> > Your use of Yahoo! Groups is subject to 
> > http://docs.yahoo.com/info/terms/
> >
> >
> >
> --
> Professor Comp. Sci., University of San Francisco
> Creator, ANTLR Parser Generator, http://www.antlr.org
> Co-founder, http://www.jguru.com
> Co-founder, http://www.knowspam.net enjoy email again!
> Co-founder, http://www.peerscope.com pure link sharing


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/