[antlr-interest] insert token in lexer?
shmuel siegel
antlr at shmuelhome.mine.nu
Thu Jun 1 00:23:02 PDT 2006
Kay Roepke wrote:
>
> On 30. May 2006, at 20:30 Uhr, Putrycz, Erik wrote:
>
>> Is it possible to insert a token in a lexer rule?
>>
>> On a specific rule, I'd like to change the type of the token with a
>> $setType(...) and I'd also would like to insert another token.
>>
>> I know I could do that with a TokenStream but I'd rather avoid writing
>> that in Java if possible...
>
> I'm afraid that it is not possible without doing Java coding.
> nextToken() always returns the token in _returnToken. So there can be
> only one token at a time.
> Surely you could provide some nextToken replacement in subclass that
> does it differently, and provide some
> hooks for adding more than one token. Then let nextToken loop over the
> accumulated tokens until it runs
> out and only then let it call another lexer rule. That way you could add
> as many tokens as you want.
>
> But question is, why do you want to do that? It sounds a bit far out ;)
>
> HTH,
>
> -k
>
It is one way of implementing javascript's virtual semicolon
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.8.0/352 - Release Date: 5/30/2006
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.8.0/352 - Release Date: 5/30/2006
More information about the antlr-interest
mailing list