[antlr-interest] Context Sensitive Keyword Support?

Jim Idle jimi at temporal-wave.com
Wed Feb 16 15:49:14 PST 2011


Yeah - I am skeptical of the performance of scannerless parsing, but if
performance is not the issue, it can help a lot here. I think that the
rats parsing technique also ends up with weak error reporting though I
think it records information to help with this right?

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Terence Parr
> Sent: Wednesday, February 16, 2011 3:43 PM
> To: antlr-interest Interest
> Subject: Re: [antlr-interest] Context Sensitive Keyword Support?
>
> I also want to try a scannerless version of ANTLR.  Rats guy Robert
> Grimm has groovy combined language / modularity feature.
> ter
> On Feb 16, 2011, at 2:21 PM, Jim Idle wrote:
>
> > I think that every time we talk about this, we realize that the
> > problems lie in lookahead where you cannot decide what to do in
> > certain cases. I think that the quantum token idea is a much better
> > one in that a token can simultaneously be ID and WHERE or any other
> > token that it is flagged as being possible to be. This removes
> context
> > from the lexer and allows the parser to decide.
> >
> > Predicates would still work and any other ambiguity should work out?
> > It means that input.LA(n) would also need
> >
> > input.LA(n, TYPE)
> >  and perhaps
> > input.LT(n).getType(TYPE)
> >
> > which would return TYPE if LT(n)/LA(n) can be one, otherwise the main
> > type that it actually is. Could have Boolean versions instead/as well
> > of course.
> >
> > WHERE :: ID, KEYWORD : 'WHERE' ;
> >
> > Or something like that. I think that this is well worth looking in
> to.
> >
> >
> >
> > Jim
> >
> >> -----Original Message-----
> >> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> >> bounces at antlr.org] On Behalf Of Terence Parr
> >> Sent: Wednesday, February 16, 2011 2:09 PM
> >> To: antlr-interest Interest
> >> Subject: Re: [antlr-interest] Context Sensitive Keyword Support?
> >>
> >> hiya. yep, i had a similar proposal way back when for v3.0 but I
> >> abandoned; can't remember why at this point. i love idea but impl
> had
> >> some issues.
> >> ter
> >> On Feb 16, 2011, at 2:02 PM, Scott Stanchfield wrote:
> >>
> >>> Has anyone looked at the context-sensitive-scanning proposal I sent
> >>> a while back?
> >>>
> >>>   http://javadude.com/articles/antlr-context-sensitive-scanner.html
> >>>
> >>> -- Scott
> >>>
> >>> ----------------------------------------
> >>> Scott Stanchfield
> >>> http://javadude.com
> >>>
> >>>
> >>>
> >>> On Wed, Feb 16, 2011 at 4:38 PM, Jim Idle <jimi at temporal-wave.com>
> >> wrote:
> >>>> No you do this in the parser by allowing the keywords where ID is
> >>>> expected then by predicating the WHERE keyword in the LINQ rules:
> >>>>
> >>>>
> >>>> .... ((WHERE)=>where_clause)? ....
> >>>>
> >>>>
> >>>> id : ID | WHERE | ....
> >>>>
> >>>>
> >>>> I have fully working C# lexer, parser, tree walker if that is what
> >>>> you are trying to build. It is a commercial product though.
> >>>>
> >>>> Jim
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> >>>>> bounces at antlr.org] On Behalf Of chris king
> >>>>> Sent: Wednesday, February 16, 2011 1:32 PM
> >>>>> To: antlr-interest at antlr.org
> >>>>> Subject: [antlr-interest] Context Sensitive Keyword Support?
> >>>>>
> >>>>> Does ANTLR support context sensitive keywords? For example, in C#
> >>>>> "where" is a keyword only in a LINQ expression. To support this
> >>>>> ANTLR would need to change the lexer behavior given feedback from
> >>>>> the parser letting it know what context it is in. From what I
> hear
> >>>>> ANTLR lexes the entire file before passing it onto the parser. If
> >>>>> that's true than I guess it doesn't support this language
> concept.
> >>>>> In that case, is there a way to work around?
> >>>>>
> >>>>> Thanks,
> >>>>> Chris
> >>>>>
> >>>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >>>>> Unsubscribe:
> >>>>> http://www.antlr.org/mailman/options/antlr-interest/your-
> >>>>> email-address
> >>>>
> >>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >>>> Unsubscribe:
> >>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-
> >> addres
> >>>> s
> >>>>
> >>>
> >>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >>> Unsubscribe:
> >>> http://www.antlr.org/mailman/options/antlr-interest/your-email-
> >> address
> >>
> >>
> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >> Unsubscribe:
> >> http://www.antlr.org/mailman/options/antlr-interest/your-
> >> email-address
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address


More information about the antlr-interest mailing list