[antlr-interest] Ambiguity error in lexer generation

[Jim Idle]

It is likely that they ARE consistently reported, but they are not
consistently making it to the screen, in the Netbeans IDE output - why don't
you try running it from the command line? Also, are you sure you are not
making any changes at all to the lexer?

 

Many times, you may feel that you have disambiguated with a synpred, but you
haven't - this is especially the case if your lexer rule has only one
alternative - antlr will say that as there is only the one alternative,
there is no point in using the synpred and ignore it. You need to combine
the things you are trying to disambiguate into the one rule with a common
leadin and then you probably won't need the synpred anyway.

 

It sounds to me like your rule for unquoted string is the same as the one
for identifier, but unless you publish it for us, we can't really help
beyond that.

 

Jim

 

From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Alex Kinneer
Sent: 19 September 2007 18:55
To: Loring Craymer; antlr-interest at antlr.org
Subject: Re: [antlr-interest] Ambiguity error in lexer generation

 


I understand the warnings (at least I'm pretty sure I do), but I'm still not
clear on why they are being reported, and more importantly why they are not
being reported consistently. What I am trying to emphasize is that if I run
antlr on the same grammar file multiple times, I sometimes get these
warnings, and sometimes don't. And that seems like a bug to me. Either the
lexer rules are ambiguous, or they aren't, right? So why would it sometimes
say they are and sometimes not, when I'm just running antlr on the exact
same grammar?
 
More importantly, I don't think the lexer rules even are ambiguous, except
to the extent that antlr advertises it can resolve automatically without
warning. For example, as best I can tell, the grammar doesn't specify
anything more ambiguous than what the Java 1.5 grammar for antlr 3.0 does
(e.g. how does antlr distinguish the keyword 'class' from an Identifier in
that grammar), yet the Java 1.5 grammar doesn't seem to produce any warnings
(at least in that regard).
 
-Alex



  _____  


> Date: Wed, 19 Sep 2007 10:12:44 -0700
> From: lgcraymer at yahoo.com
> Subject: Re: [antlr-interest] Ambiguity error in lexer generation
> To: kinneera at hotmail.com; antlr-interest at antlr.org
> 
> You have a probable error and two warnings here:
> 1.) The analysis failure is probably an error issued
> as a warning. You might have to disambiguate with a
> synpred.
> 2.) The two "disabled alternative" warnings tell you
> that 'v' will be recognized as T22 and 'g' will be
> recognized as T16. This is the type of resolution
> that you were looking for.
> 
> --Loring
> 
> --- Alex Kinneer <kinneera at hotmail.com> wrote:
> 
> > 
> > I'm hoping somebody can offer some insight as to why
> > antlr would *nondeterministically* report lexer
> > ambiguity warnings. That is to say, when I run the
> > following commands:
> > 
> > rm TestLang__.g
> > rm *.class;
> > java org.antlr.Tool TestLang.g
> > 
> > It sometimes, but not always, generates warnings of
> > this sort:
> > 
> > warning(205): TestLang.g:1:8: ANTLR could not
> > analyze this decision in rule Tokens; often this is
> > because of recursive rule references visible from
> > the left edge of alternatives. ANTLR will re-analyze
> > the decision with a fixed lookahead of k=1. Consider
> > using "options {k=1;}" for that decision and
> > possibly adding a syntactic predicate.
> > warning(209): TestLang.g:20:1: Multiple token rules
> > can match input such as "'v'": T22, T24, T25,
> > UNQUOTED_STRING, JAVA_ID
> > As a result, tokens(s)
> > JAVA_ID,UNQUOTED_STRING,T24,T25 were disabled for
> > that input
> > warning(209): TestLang.g:13:1: Multiple token rules
> > can match input such as "'g'": T16, T18,
> > UNQUOTED_STRING, JAVA_ID
> > ...
> > 
> > 
> > It is primarily the inconsistent reporting of these
> > warnings that is very perplexing to me. There's also
> > nothing obvious to me in the grammar that should be
> > causing the warnings. There is a generic Java
> > identifier lexer rule that could cause an ambiguity
> > with a set of keywords, but it is my understanding
> > that antlr should be able to resolve this ambiguity
> > (by giving preference to the keywords). The unquoted
> > string rule has a semantic predicate that should
> > prevent ambiguity.
> > 
> > Thanks,
> > Alex
> > 
> >
> _________________________________________________________________
> > Gear up for HaloR 3 with free downloads and an
> > exclusive offer. It's our way of saying thanks for
> > using Windows LiveT.
> >
> http://gethalo3gear.com?ocid=SeptemberWLHalo3_WLHMTxt_2
> 
> 
> 
>
____________________________________________________________________________
________
> Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user
panel and lay it on us.
http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 
> 



  _____  

Make your little one a shining star! Shine
<http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us>  on!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070920/6a4307dc/attachment-0001.html