[antlr-interest] Is there a safe and easy way to reuse LEXER and PARSER objects on C target?

Ronghui Yu stoneyrh at gmail.com
Fri Dec 11 05:00:37 PST 2009


Here is my pseudo code:

pLexer->pLexer->rec->reset(pLexer->pLexer->rec);
pInputStream->data = (pANTLR3_UINT8)GetText().c_str();
pInputStream->sizeBuf = (ANTLR3_UINT32)GetText().length();
pInputStream->reset(pInputStream);
pLexer->pLexer->setCharStream(pLexer->pLexer,pInputStream);
pTokenStream->free(pTokenStream);
pTokenStream = antlr3CommonTokenStreamSourceNew(TOKEN_SIZE_HINT,
TOKENSOURCE(pLexer));
pTokenStream->discardOffChannelToks(pTokenStream,ANTLR3_FALSE);
pParser->pParser->setTokenStream(pParser->pParser,pTokenStream->tstream);


It works most of the time, but occasionally violation access occurs. It
doesn't work if applied to a grammar file importing another grammar. For
example, I have a a keywords.g file is imported to the main grammar. When
setting the token stream on the main grammar, the token string
of embedded parser does not get updated automatically.


On Fri, Dec 11, 2009 at 1:11 AM, Jim Idle <jimi at temporal-wave.com> wrote:

>  To be honest, I would think you can hardly measure the time taken to
> create them, but you have to call the reset() methods and set the
> character stream and the token stream. There were issues with that at one
> point but I think I have fixed them all now. You can see how to reuse the
> lexer by looking at the examples in the examples download. Specifically
> the Java parser example will help here.
>
>
>
> Jim
>
>
>
> *From:* antlr-interest-bounces at antlr.org [mailto:
> antlr-interest-bounces at antlr.org] *On Behalf Of *Ronghui Yu
> *Sent:* Thursday, December 10, 2009 8:21 AM
> *To:* antlr-interest
> *Subject:* [antlr-interest] Is there a safe and easy way to reuse LEXER
> and PARSER objects on C target?
>
>
>
> Hi, All,
>
>
>
> On my project, I have a parser for parsing different statements again and
> again. In order to save a little time on initialization, I would like to
> reuse the LEXER and PARSER objects created the first time, something like
> this:
>
>
>
> if (bInitialized)
>
> {
>
>     reinitialize();
>
> }
>
> else
>
> {
>
>    initialize();
>
>    bInitialized = true;
>
> }
>
>
>
> The problem now is how to write reinitialize() safely. I have no idea on
> which fields of LEXER or PARSER objects must be reset to which status. Then
> my current code works most of the time, but it encounters NULL pointer
> occasionally(I am sure the grammar file is good because if I don't reuse the
> LEXER and PARSER objects, everything goes fine).
>
>
>
> Anybody could give me some ideas?
>
>
>
> Thanks in advance.
>
> --
> ===========================
> Regards
> Ronghui Yu
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>


-- 
===========================
Regards
Ronghui Yu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091211/8b1b8d48/attachment.html 


More information about the antlr-interest mailing list