[antlr-interest] Getting started with Antlr: Resolving a conflicting lexer rule

Howard Lewis Ship hlship at gmail.com
Wed Dec 3 09:49:15 PST 2008


I'm working on an improved property expression language for Tapestry
5.1. My first step was to recreate what T5.0 accepted in the past
(using an ad-hoc parser based on regular expressions).

lexer grammar PELexer;


@header
{
package propexp;
}



fragment LETTER
	:	('a'..'z'|'A'..'Z');
fragment DIGIT
	:	'0'..'9';	
fragment SIGN
	:	('+'|'-')?;
LPAREN 	:	'(';
RPAREN 	:	')';

LBRACKET
	:	 '[';
RBRACKET:	']';

LBRACE	:	'{';
RBRACE	:	'}';

IDENTIFIER
	:	LETTER (LETTER | DIGIT | '_')+;
DEREF 	:	 '.';
SAFEDEREF
	:	'?.';

RANGEOP :	'.' (options { greedy = true; } : '.');

INT 	:	SIGN DIGIT+;
FLOAT	:	SIGN DIGIT?  '.' DIGIT+;


NULL 	:	'null';
TRUE	:	'true';
FALSE	:	'false';
THIS	:	'this';

WS 	:	(' '|'\t'|'\n'|'\r')+ { skip(); };


My problem is a conflict between FLOAT and RANGEOP in my parser grammar:

parser grammar PEParser;


options
{
  tokenVocab=PELexer;	
  output=AST;		
  ASTLabelType=CommonTree;
}

tokens
{
  INVOKE;
}


@header
{
package propexp;
}



	
start 	:	expression^ EOF!;
			
expression
	:	term DEREF expression -> ^(DEREF term expression)
	|	term SAFEDEREF expression -> ^(SAFEDEREF term expression)
	|	term;
	
term	:	reservedliteral
	|	range
	|	INT
	|	FLOAT
	|	IDENTIFIER
	|	id=IDENTIFIER LPAREN RPAREN -> ^(INVOKE $id)
	;
	
reservedliteral: NULL | TRUE | FALSE | THIS;
	
range 	:	from=INT RANGEOP to=INT  -> ^(RANGEOP $from $to);	


(Obviosly, I've been playing around with this, trying different
options, such as greedy).

Everything works fine except for ranges:

     input: 1..10
line 1:2 required (...)+ loop did not match anything at character '.'
       AST: .10


In other words, I can't get it to see that a RANGE (i.e. "1..10")
should take precedence over a FLOAT (i.e., "1.234") ... it gets as far
as the "1." in a FLOAT, then gags on the extra ".".

I'm at a loss as to fixing this; I've been working through the book
and the FAQ and haven't stumbled over a solution yet.

Thanks in advance for any help!


-- 
Howard M. Lewis Ship

Creator Apache Tapestry and Apache HiveMind


More information about the antlr-interest mailing list