[antlr-interest] Unnecessary Java output file diffs

[Jim Idle]

Sent from my iPhone

On May 27, 2009, at 12:55 PM, Gavin Lambert <antlr at mirality.co.nz>  
wrote:

> At 04:55 28/05/2009, Jim Idle wrote:
>> The idea is that a token is defined as specific type, but can also  
>> exist as another type at the same time. So, if the parser needs to  
>> know if there is an ID, the code generated asks if the token is a  
>> real ID OR has a superimposed ID and so reduces all the complexity  
>> of using keywords as identifiers (and probably other case, but this  
>> is the specific problem domain), to an extra || operation and an  
>> additional LA type operation, something like this:
>>
>> int LA1_0 = input.LA(1);
>> int LA1_0_s = input.LAS(1);
>>
>> if ( (LA1_0==ID)||(LA1_0_s==ID ) {
>>                alt1=1;
>>            }
>
> Shouldn't any number of other token types be able to be superimposed  
> on a token?  It seems like a waste to arbitrarily restrict this to  
> one other type.
>
Perhaps but I have not through all the implications of that. I think  
performance might suck and I can see people getting into all sorts of  
trouble.

> (In terms of implementation, you could make LAS return a BitSet of  
> the possible types, or you could leave the decision-making to the  
> token itself; eg. use TokenType instead of int and have it define  
> comparison and containment operations.  Or something else along  
> those lines.)
>
I thought about bollean LAS(1, Toktype) but again it means more  
complicated code and then there is the DFA to consider etc

Anyway I will have a play with it soon and see if it is even practical.

Jim