[antlr-interest] lexer woes
Matt Benson
gudnabrsam at yahoo.com
Thu Mar 6 08:03:30 PST 2008
--- Loring Craymer <lgcraymer at yahoo.com> wrote:
> There is also some significant analysis to remove
> BACKTRACK_SYNPREDs when they are not needed. For
> this case, they would not be needed (LL* is
> sufficient), so they are removed and you are back to
> the FOLLOW set problem.
Okay... I definitely need to keep reading Dragon if I
want to actually understand the FOLLOW set problem;
however, going back to your first response, that's
just how it goes, fragments are of dubious utility for
the time being and as long as I bear that in mind I
just might be able to come up with a properly
functioning, if less-than-aesthetically-specified,
lexer? Does that sum it up? Thanks for all your
attention here, L.
-Matt
>
> --Loring
>
>
>
> ----- Original Message ----
> > From: Matt Benson <gudnabrsam at yahoo.com>
> > To: Loring Craymer <lgcraymer at yahoo.com>; Gavin
> Lambert <antlr at mirality.co.nz>; Antlr List
> <antlr-interest at antlr.org>
> > Sent: Wednesday, March 5, 2008 3:55:13 PM
> > Subject: Re: [antlr-interest] lexer woes
> >
> >
> > --- Loring Craymer wrote:
> >
> > > Setting backtracking in combined grammars only
> > > affects the parser (again, see the
> prefixWithSynPred
> > > code); from the code, though, setting
> > > backtracking=true in lexer grammars does
> introduce
> > > synpreds at all decision points.
> > >
> >
> > From the code... but try to tool this:
> > ----
> > lexer grammar EasyLexer;
> > options {
> > backtrack=true;
> > memoize=true;
> > }
> >
> > Ident
> > : Letter (Letter|Digit)*
> > ;
> >
> > fragment
> > Letter
> > : 'A'..'Z'|'a'..'z'|'_'
> > ;
> >
> > fragment
> > Digit
> > : '0'..'9'
> > ;
> >
> > ---
> >
> > Does it work, or have I had too much crack?
> >
> > -Matt
> >
> > > --Loring
> > >
> > > ----- Original Message ----
> > > > From: Gavin Lambert
> > > > To: Loring Craymer ; Matt
> > > Benson ; Antlr List
> > >
> > > > Sent: Wednesday, March 5, 2008 12:09:41 AM
> > > > Subject: Re: [antlr-interest] lexer woes
> > > >
> > > > At 11:35 5/03/2008, Loring Craymer wrote:
> > > > >> 1) Whether backtracking mode is supported
> for
> > > lexers,
> > > > >1.) Yes--see calls to prefixWithSynPred()
> in
> > > antlr.g
> > > >
> > > > I think that answer is a little misleading :)
> > > >
> > > > As far as I know, the "backtrack=true" option
> has
> > > no effect in the
> > > > lexer. However, backtracking in general is
> > > supported through
> > > > adding your own syntactic predicates.
> > > >
> > > > Note that syntactic predicates can only be
> used to
> > > disambiguate
> > > > between alts in a single rule -- you can't use
> > > them to distinguish
> > > > between tokens at the top level. (If you need
> to
> > > do that, you'll
> > > > have to merge the rules.)
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
____________________________________________________________________________________
> > > Never miss a thing. Make Yahoo your home page.
> > > http://www.yahoo.com/r/hs
> > >
> >
> >
> >
> >
> >
>
____________________________________________________________________________________
> > Never miss a thing. Make Yahoo your home page.
> > http://www.yahoo.com/r/hs
> >
>
>
>
>
>
>
____________________________________________________________________________________
> Looking for last minute shopping deals?
> Find them fast with Yahoo! Search.
>
http://tools.search.yahoo.com/newsearch/category.php?category=shopping
>
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
More information about the antlr-interest
mailing list