[antlr-interest] ANTLR NUB

Jan Nielsen jan.sture.nielsen at gmail.com
Mon Jan 21 09:32:11 PST 2008


Hello all,

I am a completely non-useful body when it comes to ANTLR, and grammars
in general. But I have a problem which smells like I could solve with
the help of ANTLR, but I'm finding myself leaning away from it after
reading The Definitive Guide...I think I would benefit greatly from
Ter's forthcoming recipe book...I apologize for pedantic nature and
the length of this email.

I am hoping to get a little advice from you ANTLR experts of the ilk -
"why are you doing that?!?" or "there is a better way...". So, let me
describe what I need:

I have a simple API need:

  List<Date> getValidDates(
    String expression
    );

where "expression" is a simple domain specific language of the form:

  from <start-date> [to <end-date>]
    [excluding <period> [, <period>]]
    [including <period> [, <period>]]

and <period> take a couple of forms: a specific date, a date range, a
day of week, or a range of days of week, a month, or a range of
months:

  "21/January/2008"
  "21/January/2008-28/January/2008"
  "Monday-Thursday"
  "June-July"
  "Monday[3]/January"

I would also like to support pre-defined dates, like "Dr. Martin
Luther King Day" which is defined at the "third Monday of January".

Here are a few examples of valid expressions:

  "from 1-January-2008"
  "from 1-January-2008 to 1-January-2009"
  "from 1-January-2008 to 1-January-2009 excluding 21-January-2008"
  "from 1-January-2008 to 1-January-2009 excluding 21-January-2008"
  "from 1-January-2008 to 1-January-2009 excluding Thursday-Sunday"
  "from 1-January-2008 to 1-January-2009 excluding Thursday-Sunday
including June-July"
  "from 1-January-2008 to 1-January-2009 excluding Monday-Thursday
including 21-January-2008"
  "from 1-January-2008 to 1-January-2009 excluding Monday-Thursday
including 'Dr. Martin Luther King Day'"

A "including" after an "excluding", i.e., to the right of, overrides
the exclusion.

My first stab at a grammar:

grammar T;

prog
    : 'from' date ('to' date)?
      ('including' period)? (',' period)*
      ('excluding' period)? (',' period)*
    ;

date
    : DAY_OF_MONTH '/' MONTH '/' YEAR
    ;

period
    : day_of_month_period
    | day_of_week_period
    ;

day_of_month_period
    : DAY_OF_MONTH (MONTH)? (YEAR)?
    ;

day_of_week_period
    : DAY_OF_WEEK ('[' OCCURRENCE ']')? (YEAR)?
    ;

OCCURRENCE
    : '1'..'4'
    ;

YEAR
    : '1'..'9' '0'..'9' '0'..'9' '0'..'9'
    ;

MONTH
    : 'January'
    | 'February'
    | 'March'
    | 'April'
    | 'May'
    | 'June'
    | 'July'
    | 'August'
    | 'September'
    | 'October'
    | 'November'
    | 'December'
    ;

DAY_OF_MONTH : '1'..'9' | '1'..'2' '0'..'9' | '30' | '31';

DAY_OF_WEEK
    : 'Monday'
    | 'Tuesday'
    | 'Wednesday'
    | 'Thursday'
    | 'Friday'
    | 'Saturday'
    | 'Sunday'
    ;

WS  :  (' '|'\r'|'\t'|'\u000C'|'\n') {$channel=HIDDEN;}
    ;

COMMENT
    :   '/*' ( options {greedy=false;} : . )* '*/' {$channel=HIDDEN;}
    ;

LINE_COMMENT
    : '//' ~('\n'|'\r')* '\r'? '\n' {$channel=HIDDEN;}
    ;

You, unlike me, may be able to see that this does not work. The
parsing of date is unsuccessful. And the range construct is not
represented either. I think these issues are solvable, it'll just take
me a while. But once I have a parser for my expression, how do I
actually use the parser to implement my API???

To answer this question I used gUnit to generate a JUnit test (see
below) for me from which I learned how I can hook-in the parser, but
I'm a bit worried about the apparent parser API needs. Specifically,
I'm worried about the vacuuming needed to get the results back to an
API. Is there a better way to integrate the parser into my program?
Once I have the parser integrated, my plan is to process each of these
dates to spit out the date list...is that the right way to do it?

Any help you can provide is greatly appreciated.

-Jan


gunit T;

prog
    :
    << "from 12/February/2008" >> OK
    << "from 12/February/2008 to 1/March/2009" >> OK

which produces JUnit code of the form:

import junit.framework.TestCase;
import java.io.*;
import java.lang.reflect.*;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;

public class TestT extends TestCase {
	String stdout;
	String stderr;

	public void testProg1() throws Exception {
		// test input: " "from 12/February/2008" "
		Object retval = execParser("prog", "from 12/February/2008", false);
		Object actual = examineParserExecResult(27, retval);
		Object expecting = "OK";

		assertEquals("testing rule " + "prog", expecting, actual);
	}

	public void testProg2() throws Exception {
		// test input: " "from 12/February/2008 to 1/March/2009" "
		Object retval = execParser("prog",
				"from 12/February/2008 to 1/March/2009", false);
		Object actual = examineParserExecResult(27, retval);
		Object expecting = "OK";

		assertEquals("testing rule prog", expecting, actual);
	}

	// Invoke target parser.rule
	public Object execParser(String testRuleName, String testInput,
			boolean isFile) throws Exception {
		CharStream input;
		/** Set up ANTLR input stream based on input source, file or String */
		if (isFile == true) {
			input = new ANTLRFileStream(testInput);
		} else {
			input = new ANTLRStringStream(testInput);
		}
		try {
			TLexer lexer = new TLexer(input);
			CommonTokenStream tokens = new CommonTokenStream(lexer);
			TParser parser = new TParser(tokens);
			/** Use Reflection to get rule method from parser */
			Method ruleName = Class.forName("TParser").getMethod(testRuleName);

			/** Start of I/O Redirecting */
			PipedInputStream pipedIn = new PipedInputStream();
			PipedOutputStream pipedOut = new PipedOutputStream();
			PipedInputStream pipedErrIn = new PipedInputStream();
			PipedOutputStream pipedErrOut = new PipedOutputStream();
			try {
				pipedOut.connect(pipedIn);
				pipedErrOut.connect(pipedErrIn);
			} catch (IOException e) {
				System.err.println("connection failed...");
				System.exit(1);
			}
			PrintStream console = System.out;
			PrintStream consoleErr = System.err;
			PrintStream ps = new PrintStream(pipedOut);
			PrintStream ps2 = new PrintStream(pipedErrOut);
			System.setOut(ps);
			System.setErr(ps2);
			/** End of redirecting */

			/** Invoke grammar rule, and store if there is a return value */
			Object ruleReturn = ruleName.invoke(parser);
			String astString = null;
			/** If rule has return value, determine if it's an AST */
			if (ruleReturn != null) {
				/** If return object is instanceof AST, get the toStringTree */
				if (ruleReturn.toString().indexOf(testRuleName + "_return") > 0) {
					try { // NullPointerException may happen here...
						Class _return = Class.forName("TParser" + "$"
								+ testRuleName + "_return");
						Method[] methods = _return.getDeclaredMethods();
						for (Method method : methods) {
							if (method.getName().equals("getTree")) {
								Method returnName = _return
										.getMethod("getTree");
								CommonTree tree = (CommonTree) returnName
										.invoke(ruleReturn);
								astString = tree.toStringTree();
							}
						}
					} catch (Exception e) {
						System.err.println(e);
					}
				}
			}

			org.antlr.gunit.gUnitExecuter.StreamVacuum stdoutVacuum = new
org.antlr.gunit.gUnitExecuter.StreamVacuum(
					pipedIn);
			org.antlr.gunit.gUnitExecuter.StreamVacuum stderrVacuum = new
org.antlr.gunit.gUnitExecuter.StreamVacuum(
					pipedErrIn);
			ps.close();
			ps2.close();
			System.setOut(console); // Reset standard output
			System.setErr(consoleErr); // Reset standard err out
			this.stdout = null;
			this.stderr = null;
			stdoutVacuum.start();
			stderrVacuum.start();
			stdoutVacuum.join();
			stderrVacuum.join();
			// retVal could be actual return object from rule, stderr or stdout
			if (stderrVacuum.toString().length() > 0) {
				this.stderr = stderrVacuum.toString();
				return this.stderr;
			}
			if (stdoutVacuum.toString().length() > 0) {
				this.stdout = stdoutVacuum.toString();
			}
			if (astString != null) { // Return toStringTree of AST
				return astString;
			}
			if (ruleReturn != null) {
				return ruleReturn;
			}
			if (stderrVacuum.toString().length() == 0
					&& stdoutVacuum.toString().length() == 0) {
				return null;
			}
		} catch (ClassNotFoundException e) {
			e.printStackTrace();
			System.exit(1);
		} catch (SecurityException e) {
			e.printStackTrace();
			System.exit(1);
		} catch (NoSuchMethodException e) {
			e.printStackTrace();
			System.exit(1);
		} catch (IllegalAccessException e) {
			e.printStackTrace();
			System.exit(1);
		} catch (InvocationTargetException e) {
			e.printStackTrace();
			System.exit(1);
		} catch (InterruptedException e) {
			e.printStackTrace();
			System.exit(1);
		} catch (Exception e) {
			e.printStackTrace();
			System.exit(1);
		}
		return stdout;
	}

	// Modify the return value if the expected token type is OK or FAIL
	public Object examineParserExecResult(int tokenType, Object retVal) {
		if (tokenType == 27) { // expected Token: OK
			if (this.stderr == null) {
				return "OK";
			} else {
				return "FAIL";
			}
		} else if (tokenType == 28) { // expected Token: FAIL
			if (this.stderr != null) {
				return "FAIL";
			} else {
				return "OK";
			}
		} else { // return the same object for the other token types
			return retVal;
		}
	}

}


More information about the antlr-interest mailing list