[antlr-interest] Why ANTLR doesn't check existence of lexical symbols?
Greg Lindholm
glindholm at yahoo.com
Wed Jun 11 14:12:26 PDT 2003
--- mzukowski at yci.com wrote:
> Parsers use symbols for AST node types as well. Just because the
> parser
> knows about a symbol doesn't mean the lexer has to generate it.
>
I don't realize these were used for AST node types.
But, doesn't Antlr know the difference between an AST node type and
something that it's suppose to match from a token stream?
> I'm having a hard time understanding how someone can mentally add a
> new
> token to a parser rule but not put that token into a lexer.
>
I don't make coding mistakes on purpose. It's more a problem of typos,
forgetting and just plain mistakes. The point is, a friendly tools
should warn you of any obvious errors that it is able to detect.
> At the very least these are trivially caught by your unit tests which
> excersize your grammar rules.
>
IMHO I don't believe the words "trivial", "unit tests" and "grammar"
belong in the same sentence :) Unless you mean "trival unit tests" or
"trivial grammar" :)
Greg
> Monty
>
> -----Original Message-----
> From: Greg Lindholm [mailto:glindholm at yahoo.com]
> Sent: Wednesday, June 11, 2003 1:25 PM
> To: antlr-interest at yahoogroups.com
> Subject: Re: [antlr-interest] Why ANTLR doesn't check existence of
> lexical
> symbols?
>
>
> Hi Ter,
>
> It seems to me that there are 2 case;
> 1) the lexer is in the same file or
> 2) the parser does an import of the token symbols
>
> In either case if Antlr encounters undefined tokens in a parser and
> has to
> generate new symbols I think you got a problem that at the very least
> deserves a warning message.
>
> Cause if the lexer doesn't know about the symbol it's never going to
> create
> a token of that type.
>
> Or am I missing something?
>
>
> --- Terence Parr <parrt at jguru.com> wrote:
> >
> > On Wednesday, June 11, 2003, at 02:11 AM, Hrvoje Nezic wrote:
> >
> > > Hi,
> > >
> > > If some lexical symbol is referenced in parser grammar,
> > > but is not actually defined in lexer, ANTLR doesn't generate
> error
> > > or warning messages, so this can be detected only at runtime on
> > > parser testing. I find it very inconvenient, because you have to
> > > check existence of token symbols manually.
> > > Is there any reason why ANTLR behaves like this, and
> > > is there any workaround?
> >
> > The problem is that ANTLR can be hooked up to any TokenStream
> object.
> >
> > Further, the lexer may not be defined in the same grammar file.
> > ANTLR
> > cannot answer this question, I guess is the answer (though not the
> > one
> > you are hoping for) ;)
> >
> > Terence
> > --
> > Co-founder, http://www.jguru.com
> > Creator, ANTLR Parser Generator: http://www.antlr.org Co-founder,
> > http://www.peerscope.com link sharing, pure-n-simple Lecturer in
> Comp.
> > Sci., University of San Francisco
> >
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
> > http://docs.yahoo.com/info/terms/
> >
> >
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
> http://calendar.yahoo.com
>
>
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
>
>
>
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
>
>
__________________________________
Do you Yahoo!?
Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
http://calendar.yahoo.com
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list