[antlr-interest] FW: nondeterminism in lexer rule

[Abhijit Nandkumar Ghonge]

Hi people,
	The following script doesn't work with the grammer I have
defined:

start
	# Script body goes here
	var1 = 43
	if(var2 == 43) then
		goto IFINIT
	else
		goto ENDSCRIPT
	endif

	IFINIT:
		print("IFINIT WORKED")
	ENDSCRIPT:
end

Now, in the above script start and end  describes the start and end of
script. This language is case insensitive i.e if/IF are both valid. The
Script throws error at IFINIT saying [line 13:8: expecting LITERAL,
found 'IF']. I guess this is because the nextToken() method in lexer has
a if-else loop which sees IF and returns expected token to be IF rather
than IFINIT:

else if ((LA(1)=='i') && (LA(2)=='f') && (true) && (true) && (true)) {
	mIF(true);
	theRetToken=_returnToken;
} else if (((LA(1) >= 'a' && LA(1) <= 'z')) && (true) && (true) &&
(true) && (true)) {
	mLITERAL(true);
	theRetToken=_returnToken;
}


 Can I put some logic wherein it will compare the whole token IFINIT
with IF  rather character by character.

I have declared IF as literal with following option for the grammer and
my label IFINIT/ENDSCRIPT forms part of token LITERAL which is defined
as below:

options {
  charVocabulary = '\0'..'\377';
  testLiterals = false;   // don't automatically test for literals
  k = 5;                  // eight characters of lookahead to
distinguish 'end' and  'endif'
  caseSensitive = false;
  caseSensitiveLiterals = false;
}

IF
	:	"if"
	;


LITERAL
	: (('a'..'z') ('a'..'z'|'0'..'9'| '_' | '@' | '$')*) 
	;


Please help.

Thanks & regards,
Abhijit.




 



**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070301/82e24d85/attachment-0001.html