[antlr-interest] Re: lexical nondeterminism warning
Xue Yong Zhi
seclib at seclib.com
Wed Jan 4 08:48:01 PST 2006
Your snippet is fine. I even tried it:
1. Create test.g:
class L extends Lexer;
options
{
exportVocab=Test;
k = 3;
charVocabulary='\u0003'..'\uFFFF';
}
LT: '<'; // the less than character
LE : "<=" | "=<" ; // less than or equal to
NEQ : "<>" ; // Not equal to
END : "</script>"; // end of script tag
COMMENT: "<!-" ; // HTML comment begin
2. Run it:
$ java -classpath antlr.jar antlr.Tool test.g
ANTLR Parser Generator Version 2.7.6 (2005-12-22) 1989-2005
No warning at all. Would you check the grammar and see if there is other
rules/options?
tinker wrote:
> Hi,
> I am trying to define the lexer for a scripting language that has
> the following constructs:
> LT: '<'; // the less than character
> LE : "<=" | "=<" ; // less than or equal to
> NEQ : "<>" ; // Not equal to
> END : "</script>"; // end of script tag
> COMMENT: "<!-" ; // HTML comment begin
>
> When I tried to process the above grammar file with antlr (v2.7.6)
> with lookahead(k) set to 3, I got 4 lexical nondeterminism warnings,
> between COMMENT and each of LT, LE, NEQ and END.
> I rewrote LT as following to eliminate the lexical nondeterminism
> between it and COMMENT -
> LT: {!(LA(2)=='!' && LA(3)=='-')}?'<' ; //match LT only when it's
> not followed by "!-"
>
> However, i am still left with the following three warnings:
> =========================================
> warning:lexical nondeterminism between rules LE and COMMENT upon
> k==1:'<'
> k==2:'<','='
> k==3:<end-of-token>
> warning:lexical nondeterminism between rules NEQ and COMMENT upon
> k==1:'<'
> k==2:'>'
> k==3:<end-of-token>
> warning:lexical nondeterminism between rules END and COMMENT upon
> k==1:'<'
> k==2:'/'
> k==3:'s'
> =========================================
>
> I was wondering what was the recommended way of dealing with such
> warnings? Is there any way to rewrite the definitions so that the
> warnings go away? Should I add the conditional statements (if
> LA(2)....) to each rule?
>
> Any help you can provide would be greatly appreciated.
> Thanks,
> T
>
More information about the antlr-interest
mailing list