[antlr-interest] Template Rewrite Rule consumes one more Token than expected

[Gavin Lambert]

At 01:51 22/04/2008, Felix Dorner wrote:
 >Right, but it does not delete the ";" from the corresponding 
Token
 >stream.

That's irrelevant.

 >I expect the "a*b" to be replaced by "where's the 
semicolon?",  but
 >the semicolon should remain in the token stream.
 >
 >The template rewrite rule for an "expression" should only 
replace
 >the tokens that were matched for that expression.

That's not how it works.  The parser outputs an AST, presumed to 
be a complete representation of the parsed content.  The tree 
parser takes the AST as input, and based on the nodes present in 
the tree either executes some actions or generates some output 
based on templates.  If there isn't a node for a token, it doesn't 
go and magically re-insert it into the output, it assumes you 
intentionally left it out of the AST because you don't want it in 
the output (eg. maybe you've replaced it by something more 
interesting, or maybe it's just structural and is irrelevant in 
the AST anyway).

If you want to get the semicolon in the output, then remove the ! 
from it so that it gets inserted into the AST.  The tree parser 
can only see what's in the AST.