[antlr-interest] Tool for testing lexers/parsers/tree parsers

bob mcwhirter bob at werken.com
Thu Jan 10 12:59:43 PST 2002


On Thu, 10 Jan 2002 mzukowski at bco.com wrote:

> There is a potential problem in testing specific rules in a parser in that
> the EOF token may not be part of the token sets used for lookahead in
> decisions.  This may also be a problem for "protected" lexer rules.  If you
> find such a problem I would recommend simply writing a new rule with the EOF
> included.  For example:
> 
> exprTestRule: expr EOF;
> 
> In fact I would recommend doing this for any rules you are going to test to
> make sure that all of your test input is used.  Otherwise you might only
> parse as much as the rule wants to see at the same time you would be
> assuming that your entire input has been parsed.

Would it be possible, though, to test normal rules, without the EOF, and
then check that nextToken() produces EOF?

	ie:

		parser.exprRule();
		assertEquals( TokenTypes.EOF, lexer.nextToken() );

This is at least what I've done to test hand-written recursive descent
parsers and lexers (for SAXPath).

	-bob


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list