[antlr-interest] how to let parser control lexer state.

femto gary femtowin at gmail.com
Fri Apr 27 20:47:14 PDT 2007


yes, so that's the problem.
syntax predicate or semantic predicate all happpens at parser level
and can't be used. because lex tokenStream are already constanted..

On 4/28/07, David Holroyd <dave at badgers-in-foil.co.uk> wrote:
> On Fri, Apr 27, 2007 at 10:42:24AM -0700, Loring Craymer wrote:
> > That's what semantic predicates (in the parser) are
> > for--there are no lexical issues involved in this
> > case.  You want to do the symbol table lookup and test
> > in a predicate.
>
> Here-docs are a lexing problem, I think -- content on subsequent lines
> must not be interpreted with normal lexer rules until the document
> terminator is seen..?
>
> > --- femtowin1 <femtowin1 at gmail.com> wrote:
> >
> > > Hi all, in antlr3, can parser control lexer state
> > > and decide how lexer lexing? some grammar has
> > > ambiguity
> > > decided upon by parser knowledge.
> > >   for ruby grammar <<
> > > x << 1
> > > test
> > > 1
> > > if x is a variable, then << is shift operator,
> > > otherwise it is a heredoc. so lexing must know
> > > from the symbol table whether x has been defined
> > > beforehand. But current antlrv3 implementation,
> > > lexer lexing to a constant token stream, and feed
> > > it into parser, so can't achieve this effect.
>
> --
> http://david.holroyd.me.uk/
>


-- 
Best Regards
XRuby http://xruby.com
femto http://hi.baidu.com/femto


More information about the antlr-interest mailing list