[antlr-interest] Is there a safe and easy way to reuse LEXER and PARSER objects on C target?

[Ruslan Zasukhin]

On 12/11/09 6:17 PM, "Jim Idle" <jimi at temporal-wave.com> wrote:

And I hope this is already fixed during last 1.5 year?   :-)

I bet that re-create lexer/parser objects is much slower.
In v2 we have very fast drop this idea.
And v2 ANTLR was able easy enough reuse  lexer/parser and treeparser.


> Ok - that is probably a bug. I guess nobody tried to do that before ;-). For
> now, you will have to recreate the parser each time until I can fix it.
> 
> Jim
> 
> From: Ronghui Yu [mailto:stoneyrh at gmail.com]
> Sent: Friday, December 11, 2009 5:01 AM
> To: Jim Idle
> Cc: antlr-interest
> Subject: Re: [antlr-interest] Is there a safe and easy way to reuse LEXER and
> PARSER objects on C target?
> 
> Here is my pseudo code:
> 
> pLexer->pLexer->rec->reset(pLexer->pLexer->rec);
> pInputStream->data = (pANTLR3_UINT8)GetText().c_str();
> pInputStream->sizeBuf = (ANTLR3_UINT32)GetText().length();
> pInputStream->reset(pInputStream);
> pLexer->pLexer->setCharStream(pLexer->pLexer,pInputStream);
> pTokenStream->free(pTokenStream);
> pTokenStream = antlr3CommonTokenStreamSourceNew(TOKEN_SIZE_HINT,
> TOKENSOURCE(pLexer));
> pTokenStream->discardOffChannelToks(pTokenStream,ANTLR3_FALSE);
> pParser->pParser->setTokenStream(pParser->pParser,pTokenStream->tstream);
> 
> 
> It works most of the time, but occasionally violation access occurs. It
> doesn't work if applied to a grammar file importing another grammar. For
> example, I have a a keywords.g file is imported to the main grammar. When
> setting the token stream on the main grammar, the token string of embedded
> parser does not get updated automatically.
> 
> On Fri, Dec 11, 2009 at 1:11 AM, Jim Idle <jimi at temporal-wave.com> wrote:
> To be honest, I would think you can hardly measure the time taken to create
> them, but you have to call the reset() methods and set the character stream
> and the token stream. There were issues with that at one point but I think I
> have fixed them all now. You can see how to reuse the lexer by looking at the
> examples in the examples download. Specifically the Java parser example will
> help here.
> 
> Jim
> 
> From: antlr-interest-bounces at antlr.org
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Ronghui Yu
> Sent: Thursday, December 10, 2009 8:21 AM
> To: antlr-interest
> Subject: [antlr-interest] Is there a safe and easy way to reuse LEXER and
> PARSER objects on C target?
> 
> Hi, All,
> 
> On my project, I have a parser for parsing different statements again and
> again. In order to save a little time on initialization, I would like to reuse
> the LEXER and PARSER objects created the first time, something like this:
> 
> if (bInitialized)
> {
>     reinitialize();
> }
> else
> {
>    initialize();
>    bInitialized = true;
> }
> 
> The problem now is how to write reinitialize() safely. I have no idea on which
> fields of LEXER or PARSER objects must be reset to which status. Then my
> current code works most of the time, but it encounters NULL pointer
> occasionally(I am sure the grammar file is good because if I don't reuse the
> LEXER and PARSER objects, everything goes fine).
> 
> Anybody could give me some ideas?
> 
> Thanks in advance.
> 
> --
> ===========================
> Regards
> Ronghui Yu
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: 
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 
> 
> 
> --
> ===========================
> Regards
> Ronghui Yu
> 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: 
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
Best regards,

Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc

Valentina - Joining Worlds of Information
http://www.paradigmasoft.com

[I feel the need: the need for speed]