[antlr-interest] Strange bug in ANTLR version higher than 3.1.2

[Jim Idle]

Franklin,


So I spent some time debugging your parser and basically the problem is to do with the way you are building the tree (well actually the problem might be the syntax of the language you are parsing, but you probably don't control that ;-). The new tree stream is making some assumptions about the parents of siblings and you have not kept those relationships intact. So, you can traverse down and along the tree but the new stream also looks at parents to see if it needs to add UP nodes in to the stream. You parents are the wrong parents, so it makes the wrong decision. The old stream will work because it doesn't need that information as it has buffered them all at once.

I think that the issue may well be that you are doing this directly on the nodes, and not via the tree adaptor, but the same thing would happen if the tree adaptor does not call setParent() - pretty sure it does though. 

So first, here is your grammar rewritten without anything but standard rewrite rules (lexer skipped and compressed for space). You can see that this is somewhat, err... 'simpler' ;-)

start
    : tier+ EOF!
    ;

tier
    : c+=content+ ';' d+=depContent+ '.'
        -> ^(TIER ^(WORD $c $d)+ )
    ;

content
    :   word |   INT
    ;

word
    : ID
    ;

depContent
    : i=INT -> ^(PHO $i)
    ;

This builds exactly the same tree as you do, but builds it correctly. Your tree parser then walks this perfectly. I would stick with something like this myself and if you need to check cardinality, set a validity flag before the rewrite and rewrite conditionally.

But the other reason for doing this is that if you now look at the generated Java code for the tier rule, you will see how to use the adaptor to add children to nodes, and this ought to preserve the parent child relationships properly. Basically, if you feel that you MUST write the tree yourself, then write a small piece of grammar that does what you want to do, and use the code that ANTLR generates - that way you will get it correct.

So there is no bug in the new TreeNodeStream and you should go back to it.

Cheers,

Jim