[antlr-interest] Re: Non Determinism Problem in Antler

[Thomas Brandon]

Well, you can just left-factor the rules by combining them, so add 
something like:

SPECIAL_OR_FUNCTION
    :   (SPECIAL)=> SPECIAL! FLAG { $setType(SPECIAL_FLAG); }
    |   VARIABLE LPAREN { $setType(BEGIN_FUNCTION); }
    ;

instead (and add SPECIAL_FLAG and BEGIN_FUCNTION to tokens).

But you may want to move some of the stuff to the parser unless you 
have a particular reason to do it in the lexer. Then you can use 
literals handling. So your lexer would have:

tokens {
    SPECIAL="special";
}

VARIABLE
    :    ...
    ;

With literals testing on either for the whole lexer or just VARIABLE. 
And your parser would have:

special_flag
    :   SPECIAL! FLAG;
    ;

begin_function
    :   VARIABLE LPAREN!
    ;

Otherwise you'll presumably have non-determinism between your 
BEGIN_FUNCTION rule and all other rules that start with 
('a'..'z'|'A'..'Z'|'_'), like any other literal or an ident rule.

Or you could just move the special flag handling to the parser and 
have a rule matching idents, literals and function begins like:

IDENT
    :    ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
         ( LPAREN! { $setType(BEGIN_FUNCTION); } )?
    ;

with literals testing on for that rule or whole lexer.
    
Tom.
--- In antlr-interest at yahoogroups.com, "Sriram Easwaran" 
<sriram.easwaran at s...> wrote:
> 
> Hi,
> I am having a nondeterminism waring on my Lexer.
> Here is what I have in my Lexer
> 
> protected SPECIAL  : "special::" ;
> protected VARIABLE  : ('a'..'z'|'A'..'Z'|'_') 
('a'..'z'|'A'..'Z'|'_'|'0'..'9')* ;
> 
> SPECIAL_FLAG : SPECIAL! FLAG ;         // Match SPECIAL followed by 
some FLAG
> 
> BEGIN_FUNCTION : ( VARIABLE LPAREN! ); // Match the beginning of 
any function
> 
> On compiling the grammar
> Warning: lexical nondeterminism between rules SPECIAL_FLAG and 
BEGIN_FUNCTION upon k==1:'s' k==2:'p
> 
> I am not sure how to get rid of this problem and would appreicate 
if someone can help me match both these lexer rules.
> 
> Thanks
> Sriram


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/