[antlr-interest] newbie question about nondeterminism between keywords and identifiers

[David Guy]

Thanks very much Martin.

 

I tried your suggestion, and it worked fine. I defined tokens (e.g.,
TYPE_DATE) and then override the type when the literal "date" matches in
the parser so my types are correct. 

 

I have a strong suspicion that there is a better solution, but I'm
getting the results I need.

 

Thanks again,

 

David

 

________________________________

From: Martin Nordin [mailto:martin.nordin at gmail.com] 
Sent: Saturday, February 03, 2007 12:36 AM
To: David Guy
Cc: antlr-interest at antlr.org
Subject: Re: [antlr-interest] newbie question about nondeterminism
between keywords and identifiers

 

Hi again. 

Here is a version that allows date : date; and does some printouts :)

header {
import java.io.* ; }

class DateLexer extends Lexer;

options { k=1; }
WS
  :
  (' ' 
  | '\t' 
  | '\r' '\n' { newline(); } 
  | '\n'      { newline(); }
  ) 
  { $setType( Token.SKIP); } ;
    
IDENT
options {testLiterals=true;}
  : ('_'|'a'..'z')('_'|'a'..'z'|'0'..'9')*
  ;

COLON  : ':';
SEMI   : ';'; 
  
class DateParser extends Parser;

{
  public static void main(String[] args) 
  {

    // Use a try/catch block for parser exceptions
    try {
      InputStream input  = new StringBufferInputStream("date : date;
date1 : date; an_int : int;"); 
      DateLexer   lexer  = new DateLexer(input);
      DateParser  parser = new DateParser(lexer);
      parser.declarations();
    }
    catch (Exception e) {
      System.err.println("parser exception: "+e); 
      e.printStackTrace();   // so we can get stack trace        
    }
  }
}   

decl:
  theVar:IDENT COLON theType:IDENT SEMI { System.out.println("Type
declaration. Variable: " + theVar.getText () + " Type: " +
theType.getText() ); } 
  ;
  
declarations :
  (decl)*
  ;
  
 



On 2/2/07, David Guy < dguy at bea.com <mailto:dguy at bea.com> > wrote:

I have tried putting my IDENT lexer rule last, but TYPE_DATE still
matches first. No matter what order I have tried, TYPE_DATE matches
first.

 

If "mydate : date" , "mydate" gets tokenized as IDENT, TYPE_DATE,

 

I am using ANTLR 2.7.2 in case that matters.

 

I am parsing a pre-existing language that I cannot change. Actually,
"date" can be an identifier and a type.

"date : date" is valid.

 

 

_______________________________________________________________________

Notice:  This email message, together with any attachments, may contain

information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated


entities,  that may be confidential,  proprietary,  copyrighted  and/or

legally privileged, and is intended solely for the use of the individual

or entity named in this message. If you are not the intended recipient,


and have received this message in error, please immediately return this

by email and then delete it.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070205/1de7fcdb/attachment.html