[antlr-interest] More on TokenWithIndex
Paul J. Lucas
pauljlucas at mac.com
Tue Oct 26 21:20:21 PDT 2004
I need to preserve all original tokens. I've been looking at
Terrence's TokenWithIndex approach. Consider this:
declareThing
: DECLARE!
( baseURIDecl
| functionDecl
| // ...
)
;
baseURIDecl
: b:BASE_URI^ uri=stringLiteral
{
#b.setType( BASE_URI_DECL );
}
;
i.e., this language has a bunch of "declare" statements each of
which begin with the keyword "declare". Not surprisingly,
"declare" has been left-factored.
I want each token in the generated AST to have min/max indicies
into a list of all tokens, but I want *all* the min/max to
include factored tokens as well: in this case, I want the min
index for BASE_URL_DECL to be that of the DECLARE.
A practicle application of this (i.e., why I want this) is to do
something that many IDE editors do: if there is an error in a
statement, I want to underline the entire statement with a red
squiggly line. For the case at hand, that includes "declare".
As written, Terrence's solution will not include any left-
factored token indicies. So the question is: what's a good way
to get what I want?
One way is to somehow pass the left-factored token "down" to the
other rules that can then obtain its index and set their min
accordingly.
A similar problem also occurs with discarded tokens, e.g.,
'(' and ')' in parenthesizedExpr. If you were to have:
(3 + 4)
0123456
I would want the min/max for '+' to be 0/6 and not 1/5. For
this case, one could set the min/max explicitly easily since
it's all in the same fule, i.e.:
parenthesizedExpr
: l:LPAREN! e:expr r:RPAREN!
{
#e.setMinMax( l.getMin(), r.getMax() );
}
;
(Or something like that.) But is there something better/easier?
- Paul
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list