[antlr-interest] lexical nondeterminism warning

tinker tinker at sogetthis.com
Wed Jan 4 00:25:28 PST 2006


Hi,
   I am trying to define the lexer for a scripting language that has
the following constructs:
   LT: '<';     // the less than character
   LE : "<=" | "=<" ;   // less than or equal to
   NEQ : "<>" ;   // Not equal to
   END : "</script>";  // end of script tag
   COMMENT: "<!-" ; // HTML comment begin

  When I tried to process the above grammar file with antlr (v2.7.6)
with lookahead(k) set to 3, I got 4 lexical nondeterminism warnings,
between COMMENT and each of LT, LE, NEQ and END.
    I rewrote LT as following to eliminate the lexical nondeterminism
between it and  COMMENT -
   LT: {!(LA(2)=='!' && LA(3)=='-')}?'<' ; //match LT only when it's
not followed by "!-"

  However, i am still left with the following three warnings:
=========================================
warning:lexical nondeterminism between rules LE and COMMENT upon
     k==1:'<'
     k==2:'<','='
     k==3:<end-of-token>
warning:lexical nondeterminism between rules NEQ and COMMENT upon
     k==1:'<'
     k==2:'>'
     k==3:<end-of-token>
warning:lexical nondeterminism between rules END and COMMENT upon
     k==1:'<'
     k==2:'/'
     k==3:'s'
=========================================

I was wondering what was the recommended way of dealing with such
warnings? Is there any way to rewrite the definitions so that the
warnings go away? Should I add the conditional statements (if
LA(2)....) to each rule?

Any help you can provide would be greatly appreciated.
Thanks,
T


More information about the antlr-interest mailing list