[antlr-interest] changing a token type at the parser level

[Mark Lentczner]

On May 14, 2004, at 1:03 PM, Chris Black wrote:
> I know I can change a token type in the lexer, but I'd like to change
> a token's type during tree construction in the parser but I don't seem
> to be able to figure out how to do so.

I do this all the time:
	anyOp: ( OP_1 | OP_2 | OP_3 | OP_4 | OP_5 ) { ##->setType(OP); };

However, there is a much cleaner way to do what you want:

	token {
	    ABSENT;
	    // a token to use as a place holder when optional elements are 
absent
	}

	line:
	    FIELD^ DELIM!
	    optionalField DELIM!
	    FIELD
	    (DELIM! FIELD)*
	    NEWLINE!
	    ;

	optionalField:
	      FIELD
	    | empty { ## = #[ABSENT]; }
	    ;

	empty: ;

The optional field is explicit in the grammar, and it always returns an 
AST, either a FIELD or ABSENT.

	- Mark

Mark Lentczner
markl at wheatfarm.org
http://www.wheatfarm.org/



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/