[antlr-interest] BUG: Constant (literal) token is incorrectly assumed to be present in AST rewrite

[Foust]

Austin  wrote:
> At the same time, this is pretty clearly a bug: antlr is confusing two
> tokens that look the same, but one of which doesn't exist.

The way to do this is to create a synthetic token (called "imaginary", for
some reason), by adding a section to your grammar just after the "options":

tokens {
	T_TERCIARY = '?';
}

Then use that token in your tree output:

	-> ^(TERCIARY logical_or expression? conditional_expr?) ;

Brent

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Austin Hastings
> Sent: Wednesday, August 13, 2008 6:07 PM
> To: antlr-interest
> Subject: [antlr-interest] BUG: Constant (literal) token is incorrectly
> assumed to be present in AST rewrite
> 
> Howdy,
> 
> I have a C-like grammar with this entry:
> 
> conditional_expr
>     : logical_or ( '?' expression ':' conditional_expr )?
>        -> ^( '?' logical_or expression? conditional_expr?)
>     ;
> 
> I am using antlr v3.1. In this case, the generated code assumes that
> the
> rewrite-rule's '?' token is the same as the possibly-nonexistent '?' in
> the matching rule, and tries to dereference the "stream" that would
> contain that token, if it were present.
> 
> In the case where no conditional expression is present, the rewrite
> should collapse into ^( '?' logical_or )
> 
> I suspect that there's a better way to handle this particular rewrite
> scenario. I'd like to hear it.
> 
> At the same time, this is pretty clearly a bug: antlr is confusing two
> tokens that look the same, but one of which doesn't exist.
> 
> =Austin