[antlr-interest] How to swich the recognition scope in Lexer

[Micheal J]

Hi,

> Hei Micheal,
> 
> The way you proposed to change the rule would not work as it is still
> undeterministic when processed by Lexer ("should I recognize an
> IDENTIFIER or MYTOKEN!?). I'm not sure what takes precedence here.

Have built the altered grammar and tried it?. It should just work.

> The
> proposed parser rule also alter the nature of language. This 
> was anyway
> just an example - the more general problem is that in some 
> languages you
> may need that the key words are under some condition (scope) 
> recognized
> as literals (e.g "...City=Kansas City, ... Idol=Joe Idol etc.).
> 
> In the 2.7.7 you could fix this by adding to your lexer definition:
> 
> class Testlexer extends Lexer;
> 
> {
>   private static List<String> ident_stack = new LinkedList<String>();
>   
>   // Test the token text against the literals table
>   // Override this method to perform a different literals test
>   public int testLiteralsTable(int ttype) {

<[ SNIPPED ]>

>   }
> }

Ok. I thought you meant using the default implementation of testLiterals().
Try something like:

Tokens {
	MYTOKEN;
}

Then, you can use actions in the IDENTIFIER rule to achieve a similar effect
as your old testLiteralsTable():

IDENTIFIER: ('a'..'z'| ... ) ;
	{ if ( getText().equals("mytoken") && someOtherConditionIsMet() ) {
$type=MYTOKEN } }
	
Micheal