[antlr-interest] Possible bug in C-target runtime related to off-channel tokens

[Jim Idle]

Thanks for looking into it - I will add a fix for this and the two or three
other things that have come up recently. Probably at the weekend.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of troy runkel
> Sent: Tuesday, October 09, 2007 12:35 PM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] Possible bug in C-target runtime related to
> off-channel tokens
> 
> The grammar I'm working on is configured to write comments and
> whitespace to the hidden token channel.  I found that if the text I'm
> trying to parse begins with one of these hidden tokens the token is
> sent to the parser instead of being skipped.
> 
> It looks like the problem is in the fillBuffer function in
> antlr3tokenstream.cpp.  At the end of fillBuffer we call
> skipOffTokenChannels and then initialize the
> tokenStream->tstream->istream->cachedSize variable.  However,
> skipOffTokenChannels refers to
> tokenStream->tstream->istream->cachedSize.  In my test case the cached
> size was 0 so skipOffTokenChannels failed to skip the hidden channel
> tokens.
> 
> I fixed the problem by simply moving the call to skipOffTokenChannels
> after setting the tokenStream->tstream->istream->cachedSize variable.
> 
> I believe you'll only see this problem if you also set
> discardOffChannelToks to FALSE in the token stream (which is what I'm
> doing).
> 
> Troy Runkel
> 
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.488 / Virus Database: 269.14.5/1058 - Release Date:
> 10/8/2007 4:54 PM
> 

No virus found in this outgoing message.
Checked by AVG Free Edition. 
Version: 7.5.488 / Virus Database: 269.14.5/1058 - Release Date: 10/8/2007
4:54 PM
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20071009/9e93ce3b/attachment.html