[antlr-interest] strange lexical nondeterminism

[mzukowski at yci.com]

You're hitting a limitation of the lookahead compression that ANTLR uses.
To do what you want you will have to use a syntactic predicate.  

You may have luck doing this:

REL_TIME: '0'..'2' '0'..'9' ':' '0'..'5' '0'..'9';
REL_TIME2: '0'..'9' '0'..'9' '0'..'9' '.' '0'..'2' '0'..'9' ':'
'0'..'5' '0'..'9' {$setType(REL_TIME);};
INT: ('0'..'9')+;

Monty

-----Original Message-----
From: Simon Kellett [mailto:skellett at anitesystems.de] 
Sent: Monday, March 08, 2004 5:56 AM
To: antlr-interest at yahoogroups.com
Subject: [antlr-interest] strange lexical nondeterminism

Two of the rules in my Lexer are:

// optional 3 digit day, followed by hh:mm
REL_TIME: ('0'..'9' '0'..'9' '0'..'9' '.')? '0'..'2' '0'..'9' ':'
'0'..'5' '0'..'9';
INT: ('0'..'9')+;

But I get the following warning (k=10):

lmp_grammer.g: warning:lexical nondeterminism between rules REL_TIME
and INT upon
lmp_grammer.g:     k==1:'0'..'9'
lmp_grammer.g:     k==2:'0'..'9'
lmp_grammer.g:     k==3:'0'..'9'
lmp_grammer.g:     k==4:'0'..'5'
lmp_grammer.g:     k==5:'0'..'9'
lmp_grammer.g:     k==6:<end-of-token>,'0'..'9'
lmp_grammer.g:     k==7:<end-of-token>
lmp_grammer.g:     k==8:<end-of-token>,'0'..'5'
lmp_grammer.g:     k==9:<end-of-token>,'0'..'9'
lmp_grammer.g:     k==10:<end-of-token>

If I remove the optional "?" then the warning disappears.

Surely the Lexer (with k>3) should always know whether it is dealing
with a REL_TIME or an INT ? (if the 4th char is a '.' or the 3rd is a
':' then the token must be a REL_TIME).

I could understand the nondeterminism if k=3 !!

But the generated code is OK: the REL_TIME is checked for first, and
only later the INT is check for.

TIA, Simon



 
Yahoo! Groups Links



 


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/