[antlr-interest] submitted AST factory changes to repository

[Terence Parr]

Folks,

I think I have fixed up the Java heterogeneous tree stuff and wonder of  
all wondes, I have added a TestASTFactory class in the examples area!   
My previous email + comments:

> 1. #[FOO] always builds an AST node of the default type because
>      the ASTFactory only knows about the default.  In future if you say
>
>      tokens {
>          PLUS<AST=PLUSNode>;
>          ...
>      }
>      then I'll make action #[PLUS] create the right node.  You can also
> say
>      #[ID,"foo","VarNode"] (3rd arg is the type of node to create).

I have modified action.g to handle the 3rd arg.

For 1 and 2 arg #[...] constructors, JavaCodeGenerator has been  
modified to handle snooping the first argument, ID in this case, and  
asking for any specified AST node type.  If a node type is known, it is  
passed as a third arg to create(...).

> 2. dup methods of ASTFactory don't respect the type of the nodes; it
>      uses default node type.  In future, i'll use
> t.getClass().newInstance()
>      to do the dup.

ASTFactory.dup(AST t) now makes sure that the resulting node has the  
same type, t.getClass(), as t.

> 3. hetero tree construction does not call the factory.  E.g.,
>
>      anIntRule : INT<AST=INTNode> ;
>
>      generates
>
>      INTNode v = new INTNode(LT(1));
>
>      but we need to instead generate:
>
>      AST v = (AST)astFactory.create(LT(1),"INTNode");
>
>      where the create(...) method is new and specifies the type to
>      create.  This will use newInstance() instead of "new" by default.

Every reference to INT now generates

INTNode tmp1_AST = null;
tmp1_AST = (INTNode)astFactory.create(LT(1),"INTNode");

not

INTNode tmp1_AST = null;
tmp1_AST = new INTNode(LT(1));

and so does stuff like X<AST=XNode>.

> 4. If you define ID<AST=T> in tokens section then all code in grammar
> "id:ID" should
>      define labels as "T id" not "AST id" nor labelASTType id.

I think it was this way before, but ANTLR definitely uses the most  
specific type it can now.

tokens {
     A<AST=X>;
}

a : A A<AST=Y> ;

generates

             X tmp1_AST = null;
             tmp1_AST = (X)astFactory.create(LT(1),"X");
             ...
             Y tmp2_AST = null;
             tmp2_AST = (Y)astFactory.create(LT(1),"Y");

I have modified the following files:  
----------------------------------------------

         //depot/code/org.antlr/main/main/antlr/ASTFactory.java  # edit

Added mapping from token type to classname for dynamic node  
specification.
ANTLR doesn't use this as it only handles static stuff like #[ID] not  
#[LT(1)].

        public void setTokenTypeASTNodeType(int tokenType, String  
className)

modified all of the create/dup routines to respect type.  Added

        public AST create(int type, String txt, String className)

to handle #[ID,"v","MyNodeType"].

         //depot/code/org.antlr/main/main/antlr/BaseAST.java     # edit

Just edited comment.

         //depot/code/org.antlr/main/main/antlr/JavaCodeGenerator.java    
# edit

Altered the getASTCreateString(...) routines to check for AST node type  
stuff

         //depot/code/org.antlr/main/main/antlr/Parser.java      # edit

Changed a comment

         //depot/code/org.antlr/main/main/antlr/Version.java     # edit
          
//depot/code/org.antlr/main/main/antlr/actions/java/ActionLexer.java     
# edit
          
//depot/code/org.antlr/main/main/antlr/actions/java/ 
ActionLexerTokenTypes.java  # edit
         //depot/code/org.antlr/main/main/antlr/actions/java/action.g     
# edit

Added #[a,b,c] format

          
//depot/code/org.antlr/main/main/examples/java/ASTsupport/ 
ASTType49.java        # add
          
//depot/code/org.antlr/main/main/examples/java/ASTsupport/MyAST.java     
# add
          
//depot/code/org.antlr/main/main/examples/java/ASTsupport/ 
TestASTFactory.java   # add

unit test

          
//depot/code/org.antlr/main/main/examples/java/heteroAST/ 
BinaryOperatorAST.java # edit
          
//depot/code/org.antlr/main/main/examples/java/heteroAST/CalcAST.java    
# edit
          
//depot/code/org.antlr/main/main/examples/java/heteroAST/INTNode.java    
# edit
          
//depot/code/org.antlr/main/main/examples/java/heteroAST/MULTNode.java   
# edit
          
//depot/code/org.antlr/main/main/examples/java/heteroAST/Main.java       
# edit
          
//depot/code/org.antlr/main/main/examples/java/heteroAST/PLUSNode.java   
# edit

Factored some init methods out of specific AST types.

I have uploaded this antlr-2.7.2a6_11-29-2002.jar file to the download  
area if you want to check it out on your grammars.  I think C++, C#,  
Java are now more similar in their treatment of hetero trees; for sure,  
the java hetero tree stuff was just plain broken before.

Next step is C# integration into the main line.

Ter
--
Co-founder, http://www.jguru.com
Creator, ANTLR Parser Generator: http://www.antlr.org
Lecturer in Comp. Sci., University of San Francisco


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/