[antlr-interest] RewriteEmptyStreamException when changing from quoted string to token?

[Jon Schewe]

Johannes Luber wrote:
> Ola Bini schrieb:
>   
>> Jon Schewe wrote:
>>     
>>> I have the following parser rule and lexer rule (others exist as
>>> well). I'm using antlr 3.0.1.  If I change '*' in multExpr to PRODUCT
>>> then at
>>> runtime I get an exception.  If I leave it at '*', all is fine.  What am
>>> I doing wrong here?
>>>    [junit] token PRODUCT
>>>     [junit] org.antlr.runtime.tree.RewriteEmptyStreamException: token
>>> PRODUCT
>>>     [junit]     at
>>> org.antlr.runtime.tree.RewriteRuleElementStream._next(RewriteRuleElementStream.java:158)
>>>
>>>     [junit]     at
>>> org.antlr.runtime.tree.RewriteRuleTokenStream.next(RewriteRuleTokenStream.java:57)
>>>
>>>  
>>>
>>> /**
>>>  * @param negate if true, negate all expressions by multiplying by -1
>>>  */   multExpr[boolean negate]
>>>     :   atom ('*' atom)* -> {negate}? ^(PRODUCT ^(NUMBER DOUBLE["-1"])
>>> atom+)
>>>                              ->           ^(PRODUCT atom+)
>>>     ;
>>>
>>> PRODUCT         :   '*'     ;
>>>   
>>>       
>> I have seen the same problem, and have no idea why this happens.
>>
>>     
>
> The reason is simple: '*' and PRODUCT have different tokens. If you
> change a literal to a token, then change all occurrences of it. You have
> overlooked '*' in "atom ('*' atom)*".
>   
I think you missed my point.  The only occurrance of '*' or PRODUCT on
the left side of a rewrite rule in my grammar is the one that you see
there.  If I change all occurrances of '*' to PRODUCT and have the
PRODUCT lexer rule, then I get the exception.  However if I do not have
the PRODUCT lexer rule and use the string literal '*' on the left hand
side of my rewrite rules and insert an imaginary token (one that isn't
referenced in the lexer) in my rewrite rules it works fine.

I have another case where I'm getting the same exception as well.  If I
add another case to my atom parser rule that doesn't cause a
non-determinism, I get the same exception when parsing a file that
previously parsed just fine and doesn't have anything in it that should
cause the new rule to be executed.

What causes this exception?  Most other error messages I get from antlr
are very clear stating that there are no viable alternatives at a
particular line number.  Here I get this exception.

-- 
Jon Schewe | http://mtu.net/~jpschewe
If you see an attachment named signature.asc, this is my digital
signature.
See http://www.gnupg.org for more information.

For I am convinced that neither death nor life, neither angels 
nor demons, neither the present nor the future, nor any 
powers, neither height nor depth, nor anything else in all 
creation, will be able to separate us from the love of God that 
is in Christ Jesus our Lord. - Romans 8:38-39