[antlr-interest] Context Sensitive Keyword Support?

[Scott Stanchfield]

Two things I don't like about that approach:

* The scanner would need to match each possibility against the text -
adds time to the scan when you really only need one of the
possibilities. My approach has the parser tell the scanner what tokens
are possible and the scanner will only consider those possibilities

* It really only works for very discrete token values. Suppose the
parser were switching to an embedded language; that embedded language
could use multi-word tokens or delimit things like comments in a very
different manner.

I'll have to tinker with this when I get a chance... don't know that
that's likely until after we've performed (in April) the play I'm
directing...
-- Scott

----------------------------------------
Scott Stanchfield
http://javadude.com



On Wed, Feb 16, 2011 at 10:56 PM, Michael Bedward
<michael.bedward at gmail.com> wrote:
> On 17 February 2011 09:21, Jim Idle <jimi at temporal-wave.com> wrote:
>> I think that the quantum token idea is a much better one in that a token can
>> simultaneously be ID and WHERE or any other token that it is flagged as
>> being possible to be. This removes context from the lexer and allows the
>> parser to decide.
>>
>
> Yes please !!!  This seems like a wonderfully elegant and very useful idea.
>
> Michael
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>