[antlr-interest] Lexical nondeterminism

[Putrycz, Erik]

Hi Chandu,

I'm not an expert but I think I understand your problem:
APPEAR: "APPEAR";
PG_IDENT  options { testLiterals = true;}
     :('A'..'Z')('A'..'Z'|'0'..'9'|'_')*;

If something starts with "A", with a lookahead factor of 1, ANTLR won't
know which rule to use.
Why don't you remove the "APPEAR" from the parser and move it to the
Lexer?
Your PG_IDENT rule should find it.


Erik Putrycz, Ph.D - Research Associate / erik.putrycz at nrc-cnrc.gc.ca /
(613) 990 0681
Institute for Information Technology - Software Engineering Group
National Research Council, Canada - Building M-50, 1200 Montreal Road
Ottawa, Ontario, CANADA K1A 0R6
-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Chandu Dondeti
Sent: Sunday, April 23, 2006 5:49 PM
To: antlr-interest at antlr.org
Subject: [antlr-interest] Lexical nondeterminism


Hello every one, 

I am new to this mailing list. I am posting a problem regarding an error
and It would be a great help if you could tell me what wrong with it.\\

This is the grammar which i wrote . When i compiled it its giving me an
error saying
"LEXICAL NON DETERMINISM BETWEEN RULES APPEAR AND PF_IDENT UP ON K==1"

PSQL.g
class PsqParser extends Parser;

options
{
    buildAST = true;
    k =4;
}
//tokens
//{
//SET
//PROGRAM
//EQUAL
//SEMICOLON
//}

start: question ;
question: set_context pose_query ;

set_context: SET PROGRAM EQUAL prog_name SEMICOLON ;

pose_query: POSE COLON alternate_query QUESTION_MARK;

alternate_query: what_queries
                 | where_queries
                 | which_queries
                 | does_queries
                 | can_queries
                 | are_queries;

what_queries: WHAT are_is what_optins OF shared_targets;

are_is: ARE | IS;

what_optins:     PARAMETERS | RETURN_TYPE | LOCAL_VARS | DATA_TYPE;

shared_targets: pf_name | symbol;

where_queries: WHERE are_does where_rest;

are_does: ARE | DOES;

where_rest: DECLARATIONS OF symbol
          | DOUBLE_QUOTE search_option DOUBLE_QUOTE APPEAR;

search_option: assignment
             | parameter_list
             | boolean_expr;

assignment: symbol ASSIGN_OP expr;

parameter_list: symbol (COMMA symbol)*; 
//parameter_list COMMA symbol;

boolean_expr: expr REL_OP expr;

which_queries: WHICH proc_func which_optins which_rest;

proc_func: PROC | FUNC;

which_optins: CAN | USE | MODIFY | READ;

which_rest: CALL pf_name
          | pf_name CALL
          | symbol;

does_queries: DOES pf_name call_use shared_targets;

call_use: CALL | USE;

can_queries: CAN pf_name call_use shared_targets;

are_queries: ARE symbol COMMA symbol are_optins;

are_optins: COMPATIBLE | EQUIVALENT;


expr: term ((PLUS | MINUS ) term)*;

 term : factor ( ( MULT | DIV | MOD ) factor )*
    ;
 
factor: LPAREN expr RPAREN
      | symbol
      | number;

prog_name: PG_IDENT;

pf_name: PF_IDENT;

symbol: SYM_IDENT;

number: NUMBER;

class PsqLexer extends Lexer;

   SET:"SET";
   PROGRAM: "PROGRAM";
   EQUAL: '=';
   SEMICOLON:';';
   POSE: "POSE";
   COLON: ':';
   QUESTION_MARK: '?';
   WHAT: "WHAT";
   OF: "OF";
   ARE: "ARE";
   IS: "IS";
   PARAMETERS: "PARAMETERS";
   RETURN_TYPE: "RETURN_TYPE";
   LOCAL_VARS: "LOCAL_VARS";
   DATA_TYPE: "DATA_TYPE";
   WHERE: "WHERE";
   DOES: "DOES";
   DECLARATIONS: "DECLARATIONS";
   DOUBLE_QUOTE: '"';
   APPEAR: "APPEAR";
   ASSIGN_OP: ":=";
   COMMA: ',';
   REL_OP: "AND" '|' "OR";
   WHICH: "WHICH";
   PROC: "PROC";
   FUNC: "FUNC";   
   CAN: "CAN";
   USE: "USE";
   MODIFY: "MODIFY";
   READ: "READ";
   CALL: "CALL";
   COMPATIBLE: "COMPATIBLE";
   EQUIVALENT: "EQUIVALENT";
   PLUS: '+';
   MINUS: '-';
   MULT: '*';
   DIV: '/';
   MOD: '%';
   LPAREN: '(';
   RPAREN: ')';
   PF_IDENT  options{ testLiterals = true;}
   :('A'..'Z')('a'..'z'|'A'..'Z'|'0'..'9'|'_')*;   
   PG_IDENT  options { testLiterals = true;}
     :('A'..'Z')('A'..'Z'|'0'..'9'|'_')*;
   SYM_IDENT  options { testLiterals = true;}
            :('a'..'z')('a'..'z'|'0'..'9'|'_')*;       
   NUMBER 
    :
        ('0'..'9')+
    ;
   WS  :   (   ' '
        |   '\t'
        |   '\n'  { newline(); }
        |   '\r' ('\n')?   { newline(); }
        )
        {$setType(Token.SKIP);} //ignore this token
    ;