[antlr-interest] insert token in lexer?

[Kay Roepke]

On 30. May 2006, at 20:30 Uhr, Putrycz, Erik wrote:

> Is it possible to insert a token in a lexer rule?
>
> On a specific rule, I'd like to change the type of the token with a
> $setType(...) and I'd also would like to insert another token.
>
> I know I could do that with a TokenStream but I'd rather avoid writing
> that in Java if possible...

I'm afraid that it is not possible without doing Java coding.
nextToken() always returns the token in _returnToken. So there can be  
only one token at a time.
Surely you could provide some nextToken replacement in subclass that  
does it differently, and provide some
hooks for adding more than one token. Then let nextToken loop over  
the accumulated tokens until it runs
out and only then let it call another lexer rule. That way you could  
add as many tokens as you want.

But question is, why do you want to do that? It sounds a bit far out ;)

HTH,

-k