[antlr-interest] FailedPredicateException leads to infinite loop - bug in the Lexer?
Cliff Hudson
cliff.s.hudson at gmail.com
Tue Mar 30 10:31:46 PDT 2010
I've been all over the archives, but perhaps my search terms were
inadequate. I'll look again with that date in mind. Thanks.
On Tue, Mar 30, 2010 at 8:11 AM, Ron Hunter-Duvar <
ron.hunter-duvar at oracle.com> wrote:
> Hi Cliff,
>
> I reported this same problem on February 10 on this list. It's an Antlr
> bug, and my emails on it had the work around (which requires you to
> implement a custom superclass if you haven't already). If you search the
> list archives you should be able to find it.
>
> Ron
>
>
> Cliff Hudson wrote:
>
>> I have been trying to work through an issue with an infinite loop caused
>> when no tokens can be matched because a predicate has failed its test.
>> The
>> problem appears to be in the Lexer.NextToken() (looking at CSharp2
>> sources)
>> method, which I have copied here for reference:
>>
>> /// <summary>
>> /// Return a token from this source; i.e., Match a token on
>> the char stream.
>> /// </summary>
>> public virtual IToken NextToken()
>> {
>> while (true)
>> {
>> state.token = null;
>> state.channel = Token.DEFAULT_CHANNEL;
>> state.tokenStartCharIndex = input.Index;
>> state.tokenStartCharPositionInLine =
>> input.CharPositionInLine;
>> state.tokenStartLine = input.Line;
>> state.text = null;
>> if (input.LA(1) ==
>> (int)CharStreamConstants.EOF)
>> {
>> return Token.EOF_TOKEN;
>> }
>> try
>> {
>> mTokens();
>> if (state.token == null)
>> {
>> Emit();
>> }
>> else if (state.token ==
>> Token.SKIP_TOKEN)
>> {
>> continue;
>> }
>> return state.token;
>> }
>> catch (NoViableAltException nva) {
>> ReportError(nva);
>> Recover(nva); // throw out current
>> char and try again
>> }
>> catch (RecognitionException re) {
>> ReportError(re);
>> // Match() routine has already
>> called Recover()
>> }
>> }
>> }
>>
>> Note the RecognitionException clause. This is the clause which will
>> catch the FailedPredicateException(). Unfortunately, because the
>> FailedPredicateException gets thrown just before Match() occurs in the
>> rule, Recover will *not* have been called by the rule or its callees,
>> and therefore the DFA will continue to try processing the same token.
>> It would appear that there should instead be a specific
>> FailedPredicateException clause which does the same thing as the
>> NoViableAltException clause.
>>
>> I have seen two other people ask questions about this error, and in
>> neither case was a suitable response given. Has this bug been fixed
>> in non-released builds? Can someone give me an up-or-down on whether
>> this is a correct and appropriate fix?
>>
>> Thanks.
>>
>> - Cliff
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe:
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>>
>>
>
> --
> Ron Hunter-Duvar | Software Developer V | 403-272-6580
> Oracle Service Engineering
> Gulf Canada Square 401 - 9th Avenue S.W., Calgary, AB, Canada T2P 3C5
>
> All opinions expressed here are mine, and do not necessarily represent
> those of my employer.
>
>
More information about the antlr-interest
mailing list