[antlr-interest] How to combine rules into one node for an AST?

[Gavin Lambert]

At 16:33 25/11/2007, Steve Bennett wrote:
 >Say I have a rule like this:
 >
 >Rule: MARKER sometext moretext ENDMARKER
 >-> ^(MARKER sometext moretext);
 >
 >I end up with a MARKER node with two subnodes. What's the best 
way
 >to end up with a single subnode that is the concatenation of 
those
 >two rules? Do I have to do it manually?

That depends on what you mean.  You can make it have a single 
subnode containing both of those rules by changing it to:
   -> ^(MARKER ^(TEXT sometext moretext))

... but then of course the TEXT node has two subnodes, so this is 
just moving the problem.

A single node of a tree must be a single token, so if the parser 
is handed multiple tokens it will have to put them as multiple 
nodes in the tree, unless you do something fancy.  Ideally, you 
should merge as much text as you can in the lexer, although this 
isn't always possible.