[antlr-interest] Ambiguity error in lexer generation

[Alex Kinneer]

I understand the warnings (at least I'm pretty sure I do), but I'm still not clear on why they are being reported, and more importantly why they are not being reported consistently. What I am trying to emphasize is that if I run antlr on the same grammar file multiple times, I sometimes get these warnings, and sometimes don't. And that seems like a bug to me. Either the lexer rules are ambiguous, or they aren't, right? So why would it sometimes say they are and sometimes not, when I'm just running antlr on the exact same grammar?
 
More importantly, I don't think the lexer rules even are ambiguous, except to the extent that antlr advertises it can resolve automatically without warning. For example, as best I can tell, the grammar doesn't specify anything more ambiguous than what the Java 1.5 grammar for antlr 3.0 does (e.g. how does antlr distinguish the keyword 'class' from an Identifier in that grammar), yet the Java 1.5 grammar doesn't seem to produce any warnings (at least in that regard).
 
-Alex



> Date: Wed, 19 Sep 2007 10:12:44 -0700> From: lgcraymer at yahoo.com> Subject: Re: [antlr-interest] Ambiguity error in lexer generation> To: kinneera at hotmail.com; antlr-interest at antlr.org> > You have a probable error and two warnings here:> 1.) The analysis failure is probably an error issued> as a warning. You might have to disambiguate with a> synpred.> 2.) The two "disabled alternative" warnings tell you> that 'v' will be recognized as T22 and 'g' will be> recognized as T16. This is the type of resolution> that you were looking for.> > --Loring> > --- Alex Kinneer <kinneera at hotmail.com> wrote:> > > > > I'm hoping somebody can offer some insight as to why> > antlr would *nondeterministically* report lexer> > ambiguity warnings. That is to say, when I run the> > following commands:> > > > rm TestLang__.g> > rm *.class;> > java org.antlr.Tool TestLang.g> > > > It sometimes, but not always, generates warnings of> > this sort:> > > > warning(205): TestLang.g:1:8: ANTLR could not> > analyze this decision in rule Tokens; often this is> > because of recursive rule references visible from> > the left edge of alternatives. ANTLR will re-analyze> > the decision with a fixed lookahead of k=1. Consider> > using "options {k=1;}" for that decision and> > possibly adding a syntactic predicate.> > warning(209): TestLang.g:20:1: Multiple token rules> > can match input such as "'v'": T22, T24, T25,> > UNQUOTED_STRING, JAVA_ID> > As a result, tokens(s)> > JAVA_ID,UNQUOTED_STRING,T24,T25 were disabled for> > that input> > warning(209): TestLang.g:13:1: Multiple token rules> > can match input such as "'g'": T16, T18,> > UNQUOTED_STRING, JAVA_ID> > ...> > > > > > It is primarily the inconsistent reporting of these> > warnings that is very perplexing to me. There's also> > nothing obvious to me in the grammar that should be> > causing the warnings. There is a generic Java> > identifier lexer rule that could cause an ambiguity> > with a set of keywords, but it is my understanding> > that antlr should be able to resolve this ambiguity> > (by giving preference to the keywords). The unquoted> > string rule has a semantic predicate that should> > prevent ambiguity.> > > > Thanks,> > Alex> > > >> _________________________________________________________________> > Gear up for Halo® 3 with free downloads and an> > exclusive offer. It’s our way of saying thanks for> > using Windows Live™.> >> http://gethalo3gear.com?ocid=SeptemberWLHalo3_WLHMTxt_2> > > > ____________________________________________________________________________________> Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 > 
_________________________________________________________________
Capture your memories in an online journal!
http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070919/ee4b46cb/attachment.html