[antlr-interest] changing a token type at the parser level

[Chris Black]

I know I can change a token type in the lexer, but I'd like to change
a token's type during tree construction in the parser but I don't seem
to be able to figure out how to do so.

The situation is that I have data coming in that is either:
FIELD DELIM FIELD DELIM FIELD (DELIM FIELD)*
OR
FIELD DELIM DELIM FIELD (DELIM FIELD)*

That is, sometimes there is a field between the first and second
delims, and sometimes there is nothing. To make things easier for the
tree parser, I would really like to do something like this:

token {
     OPTIONAL;
}

line: FIELD^ DELIM! (o:FIELD
	{ o.setType(OPTIONAL); System.err.println("found optional: " +
o.getText()); }
// {#o = #([OPTIONAL,o.getText()]); } // how do I set token type?
	)? DELIM! FIELD (DELIM! FIELD)* 
	NEWLINE ;

I have tried with both the o.setType syntax and the #([...]) syntax
which is now commented out. I would like the output tree to look like:
#(FIELD OPTIONAL FIELD FIELD...)
or
#(FIELD FIELD FIELD FIELD...)

depending on if the optional bit is there.
I guess I could just INSERT an imaginary token in the AST and go by
that, but I was wondering/hoping if I could just change the token type
somehow.

Thanks in advance,
Chris



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/