[antlr-interest] How to do a specific operation once on every token in the _parser_?

[Jan Harder]

Terence Parr schrieb:
>
> On Oct 4, 2006, at 7:18 AM, Jan Harder wrote:
>
>> Hello,
>>
>> I wrote a parser for visual basic using ANTLR 2.7.6 and I want to 
>> compute the halstead metric for parts of the code. Basically, this 
>> means that I need to classify all tokens in two groups (identifiers, 
>> literals vs. everything else) and count the occurrences of tokens 
>> with the same text. The classification and the counting is very easy 
>> to do, but my problem is that I am not sure _where_ to do this.
>
> How about just subclassing the common token stream and adding your 
> counting in there?
> Ter
Hello Terence,

I already discarded the idea of using the token stream because 
identifiers are problematic in visual basic 6. Not all keywords are also 
reserved words - it depends on the context what they are. So I need the 
parser to distinguish identifiers from keywords. But you made me 
reconsider the token stream idea and I think I found a way to solve this 
problem using both a TokenStream and the parser - not nice but it should 
work.

I'm going implement a TokenStream that simply counts every token as an 
operator (and keeps maps for operators and operands with their occurence 
count as its data). When the parser recognizes a token as an identifier, 
it will notify the TokenStream about that, so that the token stream can 
decrement the operator count for that symbol and increment it as an operand.

Regards, and thank you,
Jan Harder