[antlr-interest] Fwd: Re: Trying to keep whitespace in an AST
Johannes Luber
jaluber at gmx.de
Fri Feb 8 07:24:14 PST 2008
> So I guess I need an AST that keeps newlines and
> comments associated
> with particular nodes. The pretty printing is done
> by our code so the
> grammar doesn't need to do that.
Well, there are problems with the naive way of simply attaching newlines
and comments. The AST grammar doesn't recognize "The comments and
whitespace everywhere" requirement. So you can insert at every possible
point "(WS | COMMENT)*", but that would be ugly.
One could try to tell the parser to check for WS and COMMENT in addition
to the expected node, but that would interfere with syntactic
predicates, which don't check for theses variants. One could try to
change the template but I'm not sure how feasible this is. There could
be other problems lurking.
The way I'd go is to use channels. Put newlines and COMMENT into a
channel separate from the unneeded whitespace. This would reduce the
problem into accessing the off-channel tokens at the right places and to
keep them after grammar transformations. But so far I haven't checked
out, how one can do the latter properly.
Johannes
More information about the antlr-interest
mailing list