[antlr-interest] Java.g -> AST == exception

[Stefan Mätje]

Am 30.08.2012 17:55, schrieb Pedro Francisco:
> On ANTLR's site's Java.g (OpenJDK version) we find the following rule
> (sligthly adapted here):
>
> normalParameterDecl
>      :   variableModifiers type IDENTIFIER
>          dblSquareBrckt*
>      ;
>
> dblSquareBrckt : '[' ']' ;
>
> I tried applying a simple rewrite rule (on a variant of Java.g) by doing:
> normalParameterDecl
>      :   variableModifiers type IDENTIFIER dblSquareBrckt* ->
> variableModifiers type IDENTIFIER dblSquareBrckt*
>      ;
>
> Though it compiles, when I run it on a Java file I get:
> Exception in thread "main"
> org.antlr.runtime.tree.RewriteEmptyStreamException: rule
> variableModifiers
> 	at org.antlr.runtime.tree.RewriteRuleElementStream._next(RewriteRuleElementStream.java:158)
> 	at org.antlr.runtime.tree.RewriteRuleElementStream.nextTree(RewriteRuleElementStream.java:145)
> 	at com.example.javap.JavaPParser.normalParameterDecl(JavaPParser.java:9832)
>          (...)
>
> The workaround is below: (notice the '?' in variableModifiers after the ->)
> normalParameterDecl
>      :   variableModifiers type IDENTIFIER dblSquareBrckt* ->
> variableModifiers? type IDENTIFIER dblSquareBrckt*
>      ;
>
> Though I was able to workaround it, I don't understand what is
> happening. Is it expected? (I didn't check with a vanilla Java.g but I
> don't touch variableModifiers rule so I thought it should work...).
> So, what is going on here?

It's in some way surprising that you have to use a different syntax in 
the parser part of the rule (before ->) and in the rewrite part of the 
rule (after ->). In the parser part you simply use "variableModifiers" 
which rule presumably could be completely empty (no public / private... 
present). But variableModifiers hides this property from the calling 
rule on the parser level.

But on the rewrite rule level you must TELL Antlr that the resulting 
tree of variableModifiers may be empty. Therefore you need the "?" 
added. Without the "?" ANTLR expects to get something as result from 
"variableModifiers".

You won't need the "?" if the "variableModifier" rule would always 
return something. This would be the case if the source always has
some "public" or "private" or "protected" in front of the type 
designator. This kind of source code would hide the possible crash that 
you experienced.

This is my experience so far with ANTLR. Hope that clarifies the 
behavior a little bit. But as you can derive from this post I also was 
hit by this unexpected behaviour.

Best regards,
	Stefan