[antlr-interest] Re: Reusing Lexer/Parser instances

[bogdan_mt]

--- In antlr-interest at yahoogroups.com, "defordusa" <defordusa at y...> 
wrote:
> My application needs to perform MANY small translations (on the 
order 
> of millions). In order to improve performance I would like to be 
able 
> to reuse instance of the lexer and parser.


You have to patch ANTLR a bit, to be able to do something like:

lexer.getInputState().getCharBuffer().setReader(new StringReader
(in.readLine()));

Unfortunately InputBuffer input is protected in 
LexerSharedInputState, so you have to add a public getter method, 
and maybe also cast the result to charBuffer.
In CharBuffer you have to add a setter method for Reader input.
    
And don't forget to call reset() on 
LexerSharedInputState.                    

Even better, implement your own StringReader, based on Java sources, 
so that you can set the StringReader input without creating a new 
StringReader each time.            

lexer.getInputState().getCharBuffer().getStringReader().setString
(in.readLine());

Regards,
Bogdan





> I originally posted a question about using the PipedWriter and 
> PipedReader classes found in Java 1.3 and above, as I figured this 
> way I could write something in the PipedWriter, call the 
appropriate 
> parser rule and get the results. However, I have been unable to 
get 
> this to work as I appear to need to somehow get the lexer to stop 
> trying to read on the PipedReader and thus the lexer blocks on a 
read.
> 
> I have been able to reuse the lexer and parser instances calling 
> setInputState and setTokenBuffer on the lexer and parser 
> respectively. However, I still have the overhead of millions of 
> instances being created, dramatically reducing performance.
> 
> Is there a better way of doing this?
> 
> 
> Here is a snippet of my code:
> 
> ...
> ...
> ...
> while( in.ready() )
> {
>   lexer.setInputState(new LexerSharedInputState(new StringReader
> (in.readLine())));
>   parser.setTokenBuffer( new TokenBuffer(lexer) );
>   parser.startRule();
> }
> ...
> ...
> ...


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/