[antlr-interest] Re: Understanding of testLiterals=true

micheal_jor open.zone at virgin.net
Fri May 16 21:38:16 PDT 2003


Hi Andrew,

You can submit a full bug report for this issue that you mention. All 
I'd need is enough information and perhaps a sample grammar that 
demonstrates it.

Cheers,

Micheal

> I know there is a bug in the C# version (might be in other too). It 
was
> happening to me too. What I ended up doing is overwriting 
testLiteralsTable
> method in the lexer:
> 
> 	public override int testLiteralsTable(int ttype)
> 	{
> 		try
> 		{
> 			int literalsIndex =
> (int)literals[text.ToString().ToLower()];
> 			ttype = literalsIndex;
> 			return ttype;
> 		}
> 		catch
> 		{
> 			return ttype;
> 		}		
> 	}
> 
> 
> -----Original Message-----
> From: cgodfrey86 [mailto:cgodfrey at e...] 
> Sent: Thursday, May 15, 2003 9:19 AM
> To: antlr-interest at yahoogroups.com
> 
> One issue I have in defining my grammar in this manner is that I am 
> unable to detect this example token (GT_OP as "gt") regardless of 
> case.
> 
> When I add the following options
> 
> class UserLexer extends Lexer;
> options {
>   k=4;
>   caseSensitive=false;
>   caseSensitiveLiterals=false;
> }
> 
> I don't recognize GT as a token. Is this as expected?
> 
> Thanks In Advance.
> 
> --- In antlr-interest at yahoogroups.com, "cgodfrey86" <cgodfrey at e...> 
> wrote:
> > Hello,
> > 
> > I just wanted to make validate my understanding of test 
> literals=true 
> > option.
> > 
> > This grammar file is fine when Token GT_OP is defined in the 
tokens 
> > section. If instead, I defined a rule for the GT_OP token as 
> > 
> > GT_OP : "gt";
> > 
> > I get lexical nondeterminism error.
> > 
> > So, am I correct in the understanding that testLiterals will test 
> > only what is defined in the tokens section?
> > 
> > userquery.g: warning:lexical nondeterminism between rules GT_OP 
and 
> > TAG upon
> > userquery.g:     k==1:'g'
> > userquery.g:     k==2:'t'
> > userquery.g:     k==3:<end-of-token>
> > userquery.g:     k==4:<end-of-token>
> > 
> > options
> > {
> > 	language = "CSharp";
> > }
> > 
> > class UserLexer extends Lexer;
> > options {
> >   k=4;
> >   testLiterals=true;
> > 
> > }
> > 
> > tokens {
> > GT_OP = "gt";
> > }
> > 
> > 
> > TAG
> > options { testLiterals=true; }
> >  	:
> > 	('a'..'z')('a'..'z')
> > 	;
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to 
http://docs.yahoo.com/info/terms/


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list