[antlr-interest] Adding imaginary tokens to the TokenStream inthe PARSER?

[Jim Idle]

No reason at all not to do this and in fact the tree rewriting stuff is
designed to insert tokens that you declare in the tokens{} section when
you need them. Usually you use them to make the tree structure more
apparent, or unambiguous or perhaps to insert default values for
functions and so on.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Susan Jolly
> Sent: Monday, July 30, 2007 7:19 AM
> To: 'Gavin Lambert'; antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Adding imaginary tokens to the
> TokenStream inthe PARSER?
> 
> I did mean parser. I'm referring to inserting additional tokens into
> the
> previously generated TokenStream passed from the lexer to the parser.
> These
> tokens are "imaginary" in the sense that they don't reference any of
> the
> input text tokenized by the lexer.
> 
> This is somewhat the opposite of using a StringTemplate.  A
> StringTemplate
> is essentially text with holes in it where you stick pieces of the
> input. In
> my case I'm using the parser to find occasional virtual holes in the
> input
> where I need to insert additional text. Doing this directly to the
> TokenStream turns out to the most useful approach for this
application.
> 
> Susan
>