[antlr-interest] So I wish one token has two types

[Micheal J]

Jigang,

> I have two kinds of tokens, CHAR and ID, to identify 
> 
> CHAR: LowerCaseChar;
> ID: (LowerCaseChar)+;
> 
> a single char, e.g. 's', could be either type of CHAR of ID, 

Can't be resolved in a lexer (well, it may be possible but it usuallly gets
very messy). Let the parser resolve it.

> the following lexical rule can only assign one type, 
> 
> ID: LowerCaseChar {$setType(CHAR);} ( LowerCaseChar)* 
> {$setType(ID);} //get only one type protected LowerCaseChar: 'a'..'z';
> 
> In parser I need 's' be scanned as type of CHAR or ID in 
> different context. 

The relevant parser rule that doesn't care just accepts either:
	.... ( CHAR | ID )  ...

If at some other point only a single char is acceptable, use CHAR in the
parser rule. Same for ID.
	... CHAR ...

> Micheal J ever told me to see rule INT_LITERAL of csharp_v1 
> example, I think that method can only differ situation like
> 
>      INTEGER: (DIGIT)+;
> from
>      REAL: (DIGIT)+ '.' (DIGIT)+;
> 
> use a rule like 
> 
> VALUE: (DIGIT)+ {$setType(INTEGER);} ('.' (DIGIT)+ 
> {$setType(REAL);})?;

This [can] break[s] down if you also have DOT as a token. 

> if same text such as "32" is treated as INTEGER in one 
> occasion, while in other case it need to be recognised as 
> REAL the above method could not work.

And such context-depedent decisions usually belongs in a parser. Just have a
NUMERIC_LITERAL token and let the parser determine what it means.

> Probaly the anwser is in the example, but the csharp_v1 
> grammar is in a big size. I have not find the solution.

Not really. In any case, you are just interested in the INT_LITERAL rule
right?

> So I wish one token has two types, thus 's' feeding to parser 
> is of CHAR or ID depending by my needs.

That's silly ;-)
A token is either one thing or the other.


Micheal

-----------------------
The best way to contact me is via the list/forum. My time is very limited.