[antlr-interest] garmmar inheritance problem
Oleg N. Sukhodolsky
son at sparc.spb.su
Fri Apr 16 04:17:48 PDT 2004
The grammar you've sent is incomplete :( there is nosuch rule - assertOrInvariantCondition
When I jsut modify my own to no specify Lexer and use
"assert"^ expression ( COLON expression )? SEMI
it fails to parse
assert true;
perhaps you do something special in methods you define in yours parser,
but I don't fine any place whare you call them and so don't understand
if they help to handle "assert" problem :(
Oleg.
On Thu, Apr 15, 2004 at 07:53:31AM -0500, Jon Schewe wrote:
> I've actually had this problem before and solved it in a slightly
> different way. Attached is my java14.g. I did it all in the parser.
>
> On Thu, 2004-04-15 at 05:19, Oleg N. Sukhodolsky wrote:
> > Hi,
> >
> > I've found some problems with grammar inheritance and I wonder
> > if they are bugs or features. And if they are features is there are
> > any workaround for them.
> >
> > The problem is as follows:
> > I get java.g grammar from examples and create java14.g to add assert statement.
> > For this reason in java.g I rename statement to traditionalStatement and
> > add rule:
> > statement : traditionalStatement;
> >
> > in java14.g I change rule for statement:
> > statement
> > : traditionalStatement
> > | assertStatement
> > ;
> > And add:
> > assertStatement
> > : "assert"^ expression ( COLON expression )? SEMI
> > ;
> >
> > here is the first problem after compilation "assert" is not added to
> > keywords hash and so lexer consider it as IDENT.
> > I can workaround it if I add token ASSERT to lexer and will use it in the
> > rule:
> > assertStatement
> > : ASSERT^ expression ( COLON expression )? SEMI
> > ;
> >
> > class Java14Lexer extends JavaLexer;
> > ...
> > tokens {
> > ASSERT="assert";
> > }
> >
> > Althought, this works until I add some new tokens to recognizer. In this case
> > "assert" will be added to the hash with wrong id. And this is second problem.
> >
> > And the third problem is that if I want extends 14 recognizer to add 15 features
> > then Java15Lexer doesnt have "assert" in the keywords hash until I specify it
> > explicitly (but in this case it will have wrong id).
> >
> > I've tested this with antlr 2.7.(2|3|4rc1).
> >
> > So, the questions are:
> > 1. Are these problems bugs or not?
> > 2. if these are features is there any way to workaround them?
> >
> > Thanks in advance, Oleg.
> >
> > P.S. here are java14.g and java15.g
> > ----- java14.g ------------
> > class Java14Recognizer extends JavaRecognizer;
> >
> > // Options don't get inherited, copy of option block required.
> > options {
> > k = 2; // two token lookahead
> > exportVocab=Java14; // Call its vocabulary "GeneratedJava14"
> > codeGenMakeSwitchThreshold = 2; // Some optimizations
> > codeGenBitsetTestThreshold = 3;
> > defaultErrorHandler = false; // Don't generate parser error handlers
> > buildAST = true;
> > }
> >
> > // overrides the statement production in java.g, adds assertStatement
> > statement
> > : traditionalStatement
> > | assertStatement
> > ;
> >
> > // assert statement, available since JDK 1.4
> > assertStatement
> > : ASSERT^ expression ( COLON expression )? SEMI
> > ;
> >
> > class Java14Lexer extends JavaLexer;
> >
> > options {
> > exportVocab=Java14; // call the vocabulary "Java14",
> > testLiterals=false; // don't automatically test for literals
> > k=4; // four characters of lookahead
> > charVocabulary='\u0003'..'\uFFFE';
> > codeGenBitsetTestThreshold=20;
> > }
> >
> > tokens {
> > ASSERT="assert";
> > }
> >
> > protected
> > FLOAT_SUFFIX
> > : 'f'|'F'|'d'|'D'
> > ;
> > ----- java14.g ------------
> > ----- java15.g ------------
> > class Java15Recognizer extends Java14Recognizer;
> >
> > // Options don't get inherited, copy of option block required.
> > options {
> > k = 2; // two token lookahead
> > exportVocab=Java15; // Call its vocabulary "GeneratedJava15"
> > codeGenMakeSwitchThreshold = 2; // Some optimizations
> > codeGenBitsetTestThreshold = 3;
> > defaultErrorHandler = false; // Don't generate parser error handlers
> > buildAST = true;
> > }
> >
> > // just some statement
> > statement
> > : traditionalStatement
> > | assertStatement
> > ;
> >
> > class Java15Lexer extends Java14Lexer;
> >
> > options {
> > exportVocab=Java15; // call the vocabulary "Java15",
> > testLiterals=false; // don't automatically test for literals
> > k=4; // four characters of lookahead
> > charVocabulary='\u0003'..'\uFFFE';
> > codeGenBitsetTestThreshold=20;
> > }
> >
> > protected
> > FLOAT_SUFFIX
> > : 'f'|'F'|'d'|'D'
> > ;
> > ----- java15.g ------------
> >
> >
> >
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> --
> Jon Schewe | http://mtu.net/~jpschewe
> GPG signature at http://mtu.net/~jpschewe/gpg.sig.html
> For I am convinced that neither death nor life, neither angels
> nor demons, neither the present nor the future, nor any
> powers, neither height nor depth, nor anything else in all
> creation, will be able to separate us from the love of God that
> is in Christ Jesus our Lord. - Romans 8:38-39
> header {
> /*
> * Copyright (c) 2000
> * Jon Schewe. All rights reserved
> *
> * Redistribution and use in source and binary forms, with or without
> * modification, are permitted provided that the following conditions
> * are met:
> * 1. Redistributions of source code must retain the above copyright
> * notice, this list of conditions and the following disclaimer.
> * 2. Redistributions in binary form must reproduce the above copyright
> * notice, this list of conditions and the following disclaimer in the
> * documentation and/or other materials provided with the distribution.
> *
> * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
> * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
> * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> * SUCH DAMAGE.
> *
> * I'd appreciate comments/suggestions on the code jpschewe at mtu.net
> */
> package net.mtu.eggplant.dbc;
>
> import net.mtu.eggplant.util.StringPair;
> import net.mtu.eggplant.util.Pair;
>
> import java.util.Iterator;
> import java.util.Set;
> import java.util.HashSet;
> import java.util.List;
> import java.util.LinkedList;
> import java.util.LinkedHashSet;
>
> import org.apache.commons.logging.Log;
> import org.apache.commons.logging.LogFactory;
> }
>
> /**
> * Parser for 1.4, adds assert keyword. This file is generated by
> * antlr.
> *
> * @version $Revision: 1.6 $
> */
> class Java14Recognizer extends JavaRecognizer;
>
> {
>
> private static final Log LOG = LogFactory.getLog(Java14Recognizer.class);
>
> // Define some helper methods, JPS copied in from java.g because antlr is being stupid
>
> public void print(final String s) {
> System.out.println("Parser: " + s);
> }
>
> /**
> set the symbol table object to use.
> **/
> public void setSymtab(final Symtab symtab) {
> _symtab = symtab;
> }
>
> private Symtab _symtab;
>
> /**
> Get the symbol table object being used.
> **/
> public Symtab getSymtab() {
> return _symtab;
> }
>
> /**
> Given the id Token from the implements clause find the package the
> interface is defined in. Parse the interface for assertions and add
> an AssertInterface object to the current class so we can check against
> it later.
> **/
> private void parseImplementedInterface(final Token t) {
> String interfaceName = t.getText();
> String packageName = "";
> if(interfaceName.indexOf('.') > 0) {
> //already qualified interface, break up into package and name
> int lastDot = interfaceName.lastIndexOf('.');
> packageName = interfaceName.substring(0, lastDot);
> interfaceName = interfaceName.substring(lastDot+1);
> }
> else {
> //Need to figure out the right package here, search through the
> //current package, _imports
> }
>
> //actually parse the file so we can check
> //against it later, if no file can be found then don't add it to the
> //class, otherwise add it to the current class with
> //getSymtab().getCurrentClass().addInterface(assertInterface)
>
> }
>
> /**
> add an assert. This should get cached with the file object?
> @param asserts asserts, ordered as they appear in the code.
> **/
> private void addAsserts(final List asserts, final Token jdClose) {
> if(asserts != null && asserts.size() > 0) {
> int line = jdClose.getLine();
> int column = jdClose.getColumn()-1 + jdClose.getText().length();
> //System.out.println("adding assert"
> // + " line: " + line
> // + " column: " + column
> // );
> StringBuffer codeFrag = new StringBuffer();
> Iterator iter = asserts.iterator();
> while(iter.hasNext()) {
> AssertToken assertToken = (AssertToken)iter.next();
> String code = CodeGenerator.generateAssertion(assertToken);
> codeFrag.append(code);
> }
> CodeFragment codeFragment = new CodeFragment(new CodePoint(line, column), codeFrag.toString(), CodeFragmentType.ASSERT);
> getSymtab().addCodeFragment(codeFragment);
> }
> }
>
>
> private List _invariants = new LinkedList();
>
> /**
> add invariants to this class
>
> @pre (invariant != null)
> **/
> private void addInvariant(final Token invariant) {
> if(! (invariant instanceof AssertToken)) {
> throw new RuntimeException("Expecting AssertToken! " + invariant.getClass());
> }
> _invariants.add(invariant);
> }
>
> /**
> get the invariants for this class.
> Invariants are in the order they appear in the source.
> **/
> private List getInvariants() {
> return _invariants;
> }
>
> /**
> Clear the list of invariants for this class.
> **/
> private void clearInvariants() {
> _invariants = new LinkedList();
> }
>
>
> private List _preConditions = new LinkedList();
> /**
> Get the list of preconditions that have been seen since the last clear.
> Preconditions are in the order they appear in the source.
> **/
> public List getPreConditions() {
> return _preConditions;
> }
>
> /**
> Add a precondition to the list of preconditions.
> **/
> public void addPreCondition(final Token pre) {
> if(! (pre instanceof AssertToken)) {
> throw new RuntimeException("Expecting AssertToken! " + pre.getClass());
> }
> _preConditions.add(pre);
> }
>
> /**
> clear out the list of preconditions.
> **/
> public void clearPreConditions() {
> _preConditions = new LinkedList();
> }
>
> private List _postConditions = new LinkedList();
> /**
> Get the list of postconditions that have been seen since the last clear.
> Postconditions are in the order they appear in the source.
> **/
> public List getPostConditions() {
> return _postConditions;
> }
>
> /**
> clear out the list of postconditions.
> **/
> public void addPostCondition(final Token post) {
> if(! (post instanceof AssertToken)) {
> throw new RuntimeException("Expecting AssertToken! " + post.getClass());
> }
> _postConditions.add(post);
> }
>
> /**
> clear out the list of postconditions.
> **/
> public void clearPostConditions() {
> _postConditions = new LinkedList();
> }
>
> }
>
> statement
> {
> CodePointPair dummyStartEnd;
> }
> :
>
> // A list of statements in curly braces -- start a new scope!
> dummyStartEnd=compoundStatement
>
> // class definition
> | classDefinition
>
> // final class definition
> | "final" classDefinition
>
> // abstract class definition
> | "abstract" classDefinition
>
> // declarations are ambiguous with "ID DOT" relative to expression
> // statements. Must backtrack to be sure. Could use a semantic
> // predicate to test symbol table to see what the type was coming
> // up, but that's pretty hard without a symbol table ;)
> | (declaration)=> declaration SEMI
>
> // An expression statement. This could be a method call,
> // assignment statement, or any other expression evaluated for
> // side-effects.
> | expression SEMI
>
> // Attach a label to the front of a statement
> | IDENT COLON statement
>
> // If-else statement
> | "if" LPAREN expression RPAREN statement
> (
> // CONFLICT: the old "dangling-else" problem...
> // ANTLR generates proper code matching
> // as soon as possible. Hush warning.
> options {
> warnWhenFollowAmbig = false;
> }
> :
> "else" statement
> )?
>
> // For statement
> | "for"
> LPAREN
> forInit SEMI // initializer
> forCond SEMI // condition test
> forIter // updater
> RPAREN
> statement // statement to loop over
>
> // While statement
> | "while" LPAREN expression RPAREN statement
>
> // do-while statement
> | "do" statement "while" LPAREN expression RPAREN SEMI
>
> // get out of a loop (or switch)
> | "break" (IDENT)? SEMI
>
> // do next iteration of a loop
> | "continue" (IDENT)? SEMI
>
> // Return an expression
> | ret:"return" (expression)? semi:SEMI
> {
> //keep track of these points for post conditions
> CodePoint retcp = new CodePoint(ret.getLine(), ret.getColumn()-1);
> //add 1 so that code is inserted after the semi colon
> CodePoint semicp = new CodePoint(semi.getLine(), semi.getColumn());
> getSymtab().getCurrentMethod().addExit(new CodePointPair(retcp, semicp));
> }
>
> // switch/case statement
> | "switch" LPAREN expression RPAREN LCURLY
> ( casesGroup )*
> RCURLY
>
> // exception try-catch block
> | tryBlock
>
> // throw an exception
> | "throw" expression SEMI
>
> // synchronize a statement
> | "synchronized" LPAREN expression RPAREN dummyStartEnd=compoundStatement
>
> // empty statement
> | SEMI
>
> //assertion, checks invariants too for the class definitions allowed in statement
> | assertOrInvariantCondition
>
> //JDK 1.4 assert keyword
> | "assert" expression (COLON expression)? SEMI
>
> ;
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list