[antlr-interest] Is there a safe and easy way to reuse LEXER and PARSER objects on C target?

Ronghui Yu stoneyrh at gmail.com
Sat Dec 12 04:17:51 PST 2009


Thank you

On Sat, Dec 12, 2009 at 12:17 AM, Jim Idle <jimi at temporal-wave.com> wrote:

>  Ok – that is probably a bug. I guess nobody tried to do that before ;-).
> For now, you will have to recreate the parser each time until I can fix it.
>
>
>
> Jim
>
>
>
> *From:* Ronghui Yu [mailto:stoneyrh at gmail.com]
> *Sent:* Friday, December 11, 2009 5:01 AM
> *To:* Jim Idle
> *Cc:* antlr-interest
> *Subject:* Re: [antlr-interest] Is there a safe and easy way to reuse
> LEXER and PARSER objects on C target?
>
>
>
> Here is my pseudo code:
>
>
>
> pLexer->pLexer->rec->reset(pLexer->pLexer->rec);
>
> pInputStream->data = (pANTLR3_UINT8)GetText().c_str();
>
> pInputStream->sizeBuf = (ANTLR3_UINT32)GetText().length();
>
> pInputStream->reset(pInputStream);
>
> pLexer->pLexer->setCharStream(pLexer->pLexer,pInputStream);
>
> pTokenStream->free(pTokenStream);
>
> pTokenStream = antlr3CommonTokenStreamSourceNew(TOKEN_SIZE_HINT,
> TOKENSOURCE(pLexer));
>
> pTokenStream->discardOffChannelToks(pTokenStream,ANTLR3_FALSE);
>
> pParser->pParser->setTokenStream(pParser->pParser,pTokenStream->tstream);
>
>
>
>
>
> It works most of the time, but occasionally violation access occurs. It
> doesn't work if applied to a grammar file importing another grammar. For
> example, I have a a keywords.g file is imported to the main grammar. When
> setting the token stream on the main grammar, the token string
> of embedded parser does not get updated automatically.
>
>
>
> On Fri, Dec 11, 2009 at 1:11 AM, Jim Idle <jimi at temporal-wave.com> wrote:
>
> To be honest, I would think you can hardly measure the time taken to create
> them, but you have to call the reset() methods and set the character stream
> and the token stream. There were issues with that at one point but I think I
> have fixed them all now. You can see how to reuse the lexer by looking at
> the examples in the examples download. Specifically the Java parser example
> will help here.
>
>
>
> Jim
>
>
>
> *From:* antlr-interest-bounces at antlr.org [mailto:
> antlr-interest-bounces at antlr.org] *On Behalf Of *Ronghui Yu
> *Sent:* Thursday, December 10, 2009 8:21 AM
> *To:* antlr-interest
> *Subject:* [antlr-interest] Is there a safe and easy way to reuse LEXER
> and PARSER objects on C target?
>
>
>
> Hi, All,
>
>
>
> On my project, I have a parser for parsing different statements again and
> again. In order to save a little time on initialization, I would like to
> reuse the LEXER and PARSER objects created the first time, something like
> this:
>
>
>
> if (bInitialized)
>
> {
>
>     reinitialize();
>
> }
>
> else
>
> {
>
>    initialize();
>
>    bInitialized = true;
>
> }
>
>
>
> The problem now is how to write reinitialize() safely. I have no idea on
> which fields of LEXER or PARSER objects must be reset to which status. Then
> my current code works most of the time, but it encounters NULL pointer
> occasionally(I am sure the grammar file is good because if I don't reuse the
> LEXER and PARSER objects, everything goes fine).
>
>
>
> Anybody could give me some ideas?
>
>
>
> Thanks in advance.
>
> --
> ===========================
> Regards
> Ronghui Yu
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>
>
>
> --
> ===========================
> Regards
> Ronghui Yu
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>


-- 
===========================
Regards
Ronghui Yu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091212/379e2f68/attachment.html 


More information about the antlr-interest mailing list