[antlr-interest] Re: Understanding of testLiterals=true

Andrew Deren andrew at adersoftware.com
Thu May 15 18:53:45 PDT 2003


I know there is a bug in the C# version (might be in other too). It was
happening to me too. What I ended up doing is overwriting testLiteralsTable
method in the lexer:

	public override int testLiteralsTable(int ttype)
	{
		try
		{
			int literalsIndex =
(int)literals[text.ToString().ToLower()];
			ttype = literalsIndex;
			return ttype;
		}
		catch
		{
			return ttype;
		}		
	}


-----Original Message-----
From: cgodfrey86 [mailto:cgodfrey at epnet.com] 
Sent: Thursday, May 15, 2003 9:19 AM
To: antlr-interest at yahoogroups.com

One issue I have in defining my grammar in this manner is that I am 
unable to detect this example token (GT_OP as "gt") regardless of 
case.

When I add the following options

class UserLexer extends Lexer;
options {
  k=4;
  caseSensitive=false;
  caseSensitiveLiterals=false;
}

I don't recognize GT as a token. Is this as expected?

Thanks In Advance.

--- In antlr-interest at yahoogroups.com, "cgodfrey86" <cgodfrey at e...> 
wrote:
> Hello,
> 
> I just wanted to make validate my understanding of test 
literals=true 
> option.
> 
> This grammar file is fine when Token GT_OP is defined in the tokens 
> section. If instead, I defined a rule for the GT_OP token as 
> 
> GT_OP : "gt";
> 
> I get lexical nondeterminism error.
> 
> So, am I correct in the understanding that testLiterals will test 
> only what is defined in the tokens section?
> 
> userquery.g: warning:lexical nondeterminism between rules GT_OP and 
> TAG upon
> userquery.g:     k==1:'g'
> userquery.g:     k==2:'t'
> userquery.g:     k==3:<end-of-token>
> userquery.g:     k==4:<end-of-token>
> 
> options
> {
> 	language = "CSharp";
> }
> 
> class UserLexer extends Lexer;
> options {
>   k=4;
>   testLiterals=true;
> 
> }
> 
> tokens {
> GT_OP = "gt";
> }
> 
> 
> TAG
> options { testLiterals=true; }
>  	:
> 	('a'..'z')('a'..'z')
> 	;


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 






 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list