[antlr-interest] How to do a specific operation once on every token in the _parser_?
Jan Harder
harder at tzi.de
Wed Oct 4 07:18:22 PDT 2006
Hello,
I wrote a parser for visual basic using ANTLR 2.7.6 and I want to
compute the halstead metric for parts of the code. Basically, this means
that I need to classify all tokens in two groups (identifiers, literals
vs. everything else) and count the occurrences of tokens with the same
text. The classification and the counting is very easy to do, but my
problem is that I am not sure _where_ to do this. The main problem is
that it must be ensured that each token is counted only once. This
metric is of interest for each function in the code, so I need to this
in the parser, where I know where functions start and where they end.
Unfortunately I can't do the measurement on the AST tree because it does
not contain all tokens. Things like brackets are not added to the tree.
It is not an option to change the tree.
Of course I don't want to insert an counting action for every token that
can appear in a function. What I need is a place where each token comes
along exactly once and can be counted easily. I thought of using the
parsers consume() function. My idea was to overwirte it and add the
classification counting code for the metric there by inspecting LT(1)
after the consume-implementation from LLKParser was called. I guess this
way ensures that every token is inspected exactly once, but I'm not sure
about that. Is it safe to do it that way or is the consume method
affected be the rewind operation on the token stream so that the same
token can be handeled by consume() more than once?
Thanks for your help.
Jan Harder
PS: I generate C++ code
More information about the antlr-interest
mailing list