[antlr-interest] Problem with lexical nondeterminism - ANTLR 2.7.7

Daniels, Troy (US SSA) troy.daniels at baesystems.com
Thu Jan 3 07:00:39 PST 2008


The problem is that your rules are ambiguous with fixed look-ahead,
which is what antlr 2 uses.  Given 4 digits and k=4 look-ahead, either
rule could match, hence the warning when compiling the grammar.  You
need a semantic predicate to disambiguate it:
 
APAC_NUMERIC_TICKER 
    : (DIGIT+ COMMA) => ((DIGIT)+ (COMMA) (CHAR) (CHAR)) 
    ; 

The sample input you give is precisely where it is ambiguous.  1350
looks like 4 digits, which will match NUMBER or the beginning of
APAC_NUMBER_TICKER.
 
Troy


  _____  

	From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of
clive.i.hill at jpmorgan.com
	Sent: Thursday, January 03, 2008 8:25 AM
	To: antlr-interest at antlr.org
	Subject: [antlr-interest] Problem with lexical nondeterminism -
ANTLR 2.7.7
	
	

	Hi, 
	
	I get the following warning when building my lexer from
stragegy.g: 
	
	/Grammar/strategy.g: warning:lexical nondeterminism between
rules NUMBER and APAC_NUMERIC_TICKER upon 
	./Grammar/strategy.g:     k==1:'0'..'9' 
	./Grammar/strategy.g:     k==2:'0'..'9' 
	./Grammar/strategy.g:     k==3:'0'..'9' 
	./Grammar/strategy.g:     k==4:'0'..'9' 
	
	Passing the following string fails: 
	
	"spx dec 1350 c 43 44" 
	
	As it takes 1350 to be an APAC_NUMERIC_TICKER when I it to be
considered to be a NUMBER.  Interestingly, the following works: 
	
	"spx dec 135 c 43 44" 
	
	My definition for the lexer rule is: 
	
	APAC_NUMERIC_TICKER 
	    : ((DIGIT)+ (COMMA) (CHAR) (CHAR)) 
	    ; 
	
	I want this to only be used if a string entered "has at least
one DIGIT and is followed by a COMMA then two CHARs".  This is clearly
not the case for the string 1350 - and yet it matches,  and then throws
the exception: 
	
	
antlr.TokenStreamRecognitionException: expecting ',', found ' '



	Can anyone suggest a way to only match this rule if it is in the
correct format ((DIGIT)+ (COMMA) (CHAR) (CHAR)) ? 
	
	Thanks in advance, 
	
	Clive 
	
	
	
	Attached is my grammar file. 
	
	
	

	
  _____  


	

	Generally, this communication is for informational purposes only
and it is not intended as an offer or solicitation for the purchase or
sale of any financial instrument or as an official confirmation of any
transaction. In the event you are receiving the offering materials
attached below related to your interest in hedge funds or private
equity, this communication may be intended as an offer or solicitation
for the purchase or sale of such fund(s). All market prices, data and
other information are not warranted as to completeness or accuracy and
are subject to change without notice. Any comments or statements made
herein do not necessarily reflect those of JPMorgan Chase & Co., its
subsidiaries and affiliates. This transmission may contain information
that is privileged, confidential, legally privileged, and/or exempt from
disclosure under applicable law. If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance thereon)
is STRICTLY PROHIBITED. Although this transmission and any attachments
are believed to be free of any virus or other defect that might affect
any computer system into which it is received and opened, it is the
responsibility of the recipient to ensure that it is virus free and no
responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and
affiliates, as applicable, for any loss or damage arising in any way
from its use. If you received this transmission in error, please
immediately contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you. Please refer to
http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK
legal entities. 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080103/9f257a3b/attachment.html 


More information about the antlr-interest mailing list