[antlr-interest] [3.1.1] ANTLR3_MIN_TOKEN_TYPE define possibly incorrect

[Sam Barnett-Cormack]

> rewrite_this
>    :  ^( node=NEEDS_TO_BE_REWRITTEN b=body )
>      -> { createNewNode( $node, $b, LT( 1 ) ) }
>    ;
> 
> 
> In createNewNode I use $node, $b and the next sibling of $node to
> determine what to do. Now I found that when $node has no next sibling,
> as in the situation (2) below, LT(1) still returns a node (I would have
> expected NULL).
> 
> |
> +- NEEDS_TO_BE_REWRITTEN   <-- (1)  LT(1) on this one is fine,
> |    |                              it returns (2)
> |    +- SOME_BODY_NODE
> |
> +- NEEDS_TO_BE_REWRITTEN   <-- (2) LT(1) on this one also returns a node
>      |
>      +- SOME_BODY_NODE
> 
> The type of this node is 3 or ANTLR3_TOKEN_UP, which, from the comments
> in antlr3commontoken.h seems to be an imaginary token that signals the
> end of the stream.
> 
> At first I only checked that LT(1) returned something non-NULL, but
> since a node was returned in (2) I ended up creating the wrong node to
> return in the rewrite rule. Then I found out that the type returned was
> ANTLR3_TOKEN_UP and that I could use ANTLR3_MIN_TOKEN_TYPE to determine
> whether I had a 'valid' node or not.
> 
> Should LT(1) return a node in (2), or does that signal that something's
> amiss? If the behavior of LT(1) is correct, how can I determine that a
> node has no next sibling?

<UP> signals the end of the current "level" of tree. I've not played 
with it much, but I believe the idea is that, in a tree parser, you 
don't need to pay attention to <UP> and <DOWN>. If you've got a true 
single-root tree, there will always be an <UP> just before the EOF. 
However, my understanding is that tree-grammar writers don't have to 
bother with these. LT will still show them, I guess. Wouldn't it be 
better to have the tree parser control the application of the rule by 
only applying it "under" the node above? Or if this *is* at the top 
level, your solution might well work, as you've demonstrated that you 
only get the <UP> when there isn't a next sibling.

-- 
Sam Barnett-Cormack