[antlr-interest] Re: lexical nondeterminism warning

Xue Yong Zhi seclib at seclib.com
Thu Jan 5 09:33:52 PST 2006


Well, now you meet antlr's "linear approximate lookahead":)
Please read this:
http://www.antlr.org/doc/glossary.html#Linear_approximate_lookahead
and related entries in antlr's FAQ.

You may find my blog usefully as well:
http://seclib.blogspot.com/2005/11/linear-approximate-lookahead.html

Note: antlr may generate correct code even if it gives warnings, but you 
have to read the generated code to be sure.

In your case, the best way is to seperate SINGLE_COMMENT and 
MULTI_COMMENT in lexer, while create a parser rule to group them together:

comment
:SINGLE_COMMENT
|MULTI_COMMENT

If you do so, the lexer does not need to compression the lookahead set 
for nextToken(), and you still have a maintainable grammar.

> COMMENT
> :
>  SINGLE_COMMENT
>  |
>   MULTI_COMMENT
> ;
>  ---------------------------------------------
> But when I try to compile the grammar now, I get the warnings again.
> They are the same warnings as before, and are reproduced below:
> =========================================
> warning:lexical nondeterminism between rules LE and COMMENT upon
>     k==1:'<'
>     k==2:'<','='
>     k==3:<end-of-token>
> warning:lexical nondeterminism between rules NEQ and COMMENT upon
>     k==1:'<'
>     k==2:'>'
>     k==3:<end-of-token>
> warning:lexical nondeterminism between rules END and COMMENT upon
>     k==1:'<'
>     k==2:'/'
>     k==3:'s'
> =========================================
> 
> So can anyone tell me why this is happening, and what I can do to get
> around this?
> 
> Thanks,
> T
> 


-- 
Xue Yong Zhi
http://seclib.blogspot.com



More information about the antlr-interest mailing list