[antlr-interest] Comment rule matches links

Thomas Brandon tbrandonau at gmail.com
Tue Aug 26 04:54:38 PDT 2008


On Tue, Aug 26, 2008 at 9:31 PM, Jenny Balfer
<ai06087 at lehre.ba-stuttgart.de> wrote:
>
>> An alternate solution is to analyse the context of the / in the lexer
>> to determine if it is a regular expression or division (e.g. a
>> division cannot directly follow an equals, so if the last non-hidden
>> token was an equals it must be a regular expression). Depending on
>> your language it may be hard to formulate and implement an adequate
>> set of rules at the lexer level.
>
> That sounds like a solution for me! How can I access the last non-hidden
> token in the lexer? I think I'll try that out.
>
>
There is no standard way. You could override nextToken to keep a copy.
Or you could pass the lexer a copy of the token stream. The LA\LB
methods won't work as it's being built but you could use getToken()
and just find the last non-hidden one. Or you could subclass the
existing CommonTokenSream to provide access to the protected internals
where tokens.get(skipOffTokenChannelsReverse(tokens.size() - 1))
should do it.

Tom.


More information about the antlr-interest mailing list