[antlr-interest] Why ANTLR doesn't check existence of lexical symbols?

[mzukowski at yci.com]

Parsers use symbols for AST node types as well.  Just because the parser
knows about a symbol doesn't mean the lexer has to generate it.

I'm having a hard time understanding how someone can mentally add a new
token to a parser rule but not put that token into a lexer.

At the very least these are trivially caught by your unit tests which
excersize your grammar rules.

Monty

-----Original Message-----
From: Greg Lindholm [mailto:glindholm at yahoo.com] 
Sent: Wednesday, June 11, 2003 1:25 PM
To: antlr-interest at yahoogroups.com
Subject: Re: [antlr-interest] Why ANTLR doesn't check existence of lexical
symbols?


Hi Ter,

It seems to me that there are 2 case;
1) the lexer is in the same file or
2) the parser does an import of the token symbols

In either case if Antlr encounters undefined tokens in a parser and has to
generate new symbols I think you got a problem that at the very least
deserves a warning message.

Cause if the lexer doesn't know about the symbol it's never going to create
a token of that type.

Or am I missing something?


--- Terence Parr <parrt at jguru.com> wrote:
> 
> On Wednesday, June 11, 2003, at 02:11  AM, Hrvoje Nezic wrote:
> 
> > Hi,
> >
> > If some lexical symbol is referenced in parser grammar,
> > but is not actually defined in lexer, ANTLR doesn't generate error 
> > or warning messages, so this can be detected only at runtime on 
> > parser testing. I find it very inconvenient, because you have to 
> > check existence of token symbols manually.
> > Is there any reason why ANTLR behaves like this, and
> > is there any workaround?
> 
> The problem is that ANTLR can be hooked up to any TokenStream object.
>  
> Further, the lexer may not be defined in the same grammar file.
> ANTLR 
> cannot answer this question, I guess is the answer (though not the
> one 
> you are hoping for) ;)
> 
> Terence
> --
> Co-founder, http://www.jguru.com
> Creator, ANTLR Parser Generator: http://www.antlr.org Co-founder, 
> http://www.peerscope.com link sharing, pure-n-simple Lecturer in Comp. 
> Sci., University of San Francisco
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to 
> http://docs.yahoo.com/info/terms/
> 
> 


__________________________________
Do you Yahoo!?
Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
http://calendar.yahoo.com

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/