[antlr-interest] confounded exceptions on predicate

Mon Apr 2 15:14:53 PDT 2012

Hi,

Kind of fishing for some suggestions on where to look for the cause of this
problem.
Currently I'm working on converting a set of grammar files from ANTLR 2 to
ANTLR 3.

Descrpition:
>From within Eclipse 3.6.2, using
	Java
	java version "1.6.0"
	Java(TM) SE Runtime Environment (build pwi3260sr10-20111208_01(SR10))
	IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Windows 7 x86-32
jvmwi3260sr10-201112

	ANTLR IDE- ANTLR Tools
	2.1.1.201011141044

	ANTLR
	antlr-3.4-complete.jar

Code snippet:

primary
	: (CURSOR LPAREN SELECT) => c=CURSOR^ LPAREN! s=subquery1 r=RPAREN!
	     {$c.setText (SrcAST.getText($c) + "(" + SrcAST.getText($s) + ")"
) ;
	      SrcAST.setSpan ($c, $c, $r);  }

  | CAST LPAREN => cast_expr	      // problematic line
	| (simple_case_expr) => simple_case_expr
	| (searched_case_expr)  => searched_case_expr
	//functions with special syntax:
	| (trim_function) => trim_function
	| (decompose_function) => decompose_function
	| (extract_function) => extract_function

	| (treat_function)	=> treat_function
	| (tz_offset_function)	=>	tz_offset_function
	| (percentile_functions) => percentile_functions
	| (xmlcolattval_function)=> xmlcolattval_function
	| (xmlelement_function)=> xmlelement_function
	| (xmlforest_function)=> xmlforest_function
	| literal
	| t=TIMESTAMP^ st=STRING_LITERAL
		{ $primary.setText("TIMESTAMP "+SrcAST.getText($st));
		  SrcAST.setSpan($primary,$t,$st); }
	| d=DATE^ sd=STRING_LITERAL
		{ $primary.setText("DATE "+SrcAST.getText($st));
		  SrcAST.setSpan($primary,$t,$st); }
	| interval_literal

	| ((NEW)=> NEW)? v=variable_expression
	  ->  // build no AST for this alternative
	    {$primary = $v;}
	| (LPAREN subquery2) =>   l3=LPAREN! su=subquery2 r3=RPAREN!
	     { $primary.setText ( "(" + SrcAST.getText($su) + ")" ) ;
	       SrcAST.setSpan ($primary, $l3, $r3); }
	| expr_list_p

	;

cast_expr
	: c=CAST^ LPAREN! e=expr_or_multiset AS! t=datatype r=RPAREN!
	     { $c.setText ( "CAST(" + SrcAST.getText($e) + " AS " +
SrcAST.getText($t) + ")" ) ;
	       SrcAST.setSpan ($c, $c, $r); }
	;

If the problem line, CAST LPAREN ... , is not commented out, from the
Eclipse IDE and ANTLRWorks, I get the errors below.  I get other errors if
I comment it out, but the processing makes it past the point where this
error occurs.

   ANTLR Parser Generator 3.4 Jul 19, 2011 11:35:12.
   Using project classpath: Yes.
   Grammar: [snipped]Parser.g
   java.lang.NoSuchFieldError:
   org/antlr/tool/GrammarSyntaxMessage.offendingToken
   	at org.deved.antlride.runtime.AntlrErrorListener.extractToken
   (AntlrErrorListener.java:108)
   	at org.deved.antlride.runtime.AntlrErrorListener.report
   (AntlrErrorListener.java:79)
   	at org.deved.antlride.runtime.AntlrErrorListener.message
   (AntlrErrorListener.java:63)
   	at org.deved.antlride.runtime.AntlrErrorListener.error
   (AntlrErrorListener.java:53)
   	at org.antlr.tool.ErrorManager.syntaxError(ErrorManager.java:768)
   	at org.antlr.grammar.v3.ANTLRParser.reportError(ANTLRParser.java:443)
   	at org.antlr.grammar.v3.ANTLRParser.elementNoOptionSpec
   (ANTLRParser.java:4909)
   	at org.antlr.grammar.v3.ANTLRParser.element(ANTLRParser.java:4533)
   	at org.antlr.grammar.v3.ANTLRParser.alternative
   (ANTLRParser.java:4120)
   	at org.antlr.grammar.v3.ANTLRParser.ruleAltList
   (ANTLRParser.java:3580)
   	at org.antlr.grammar.v3.ANTLRParser.rule(ANTLRParser.java:2787)
   	at org.antlr.grammar.v3.ANTLRParser.rules(ANTLRParser.java:2405)
   	at org.antlr.grammar.v3.ANTLRParser.grammar_(ANTLRParser.java:702)
   	at org.antlr.tool.Grammar.parseAndBuildAST(Grammar.java:644)
   	at org.antlr.Tool.getRootGrammar(Tool.java:626)
   	at org.antlr.Tool.process(Tool.java:459)
   	at org.deved.antlride.runtime.Tool2.main(Tool2.java:24)
   BUILD SUCCESSFUL
   Total time: 1 second

This is confusing for a couple of reasons:  the build is reported to be
successful when it isn't, and the field offendingToken appears to actually
be a member GrammarSyntaxMessage.

I wasn't getting anywhere trying to solve this with the plugin; so, I tried
using ANTLRWorks 1.4.3.  This told me a bit more:

   [16:35:27] error(100): [snipped]Parser.g:3825:17: syntax error: antlr:
   NoViableAltException(72@[722:4: ( ( id ( ASSIGN | PLUS_ASSIGN ) ( atom |
   block ) ) (sub= ebnfSuffix[root_0,false] )? | a= atom (sub2= ebnfSuffix
   [$a.tree,false] )? | ebnf | FORCED_ACTION | ACTION | p= SEMPRED
   ( IMPLIES )? | t3= tree_ )])
   [16:35:27] error(100): [snipped]Parser.g:3825:17: syntax error: antlr:
   NoViableAltException(72@[722:4: ( ( id ( ASSIGN | PLUS_ASSIGN ) ( atom |
   block ) ) (sub= ebnfSuffix[root_0,false] )? | a= atom (sub2= ebnfSuffix
   [$a.tree,false] )? | ebnf | FORCED_ACTION | ACTION | p= SEMPRED
   ( IMPLIES )? | t3= tree_ )])
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(0!=29)
   [16:35:27] error(100): [snipped]Parser.g:0:: syntax error: assign.types:
   NoViableAltException(3@[])
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(3!=28)
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(3!=27)
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(26!=3)
   [16:35:27] java.lang.ClassCastException:
   org.antlr.runtime.tree.CommonTree incompatible with
   org.antlr.tool.GrammarAST
   	at org.antlr.grammar.v3.DefineGrammarItemsWalker.rewrite
   (DefineGrammarItemsWalker.java:3463)
   	at org.antlr.grammar.v3.DefineGrammarItemsWalker.block
   (DefineGrammarItemsWalker.java:2072)
   	at org.antlr.grammar.v3.DefineGrammarItemsWalker.rule
   (DefineGrammarItemsWalker.java:1627)
   	at org.antlr.grammar.v3.DefineGrammarItemsWalker.rules
   (DefineGrammarItemsWalker.java:1190)
   	at org.antlr.grammar.v3.DefineGrammarItemsWalker.grammarSpec
   (DefineGrammarItemsWalker.java:623)
   	at
   org.antlr.grammar.v3.DefineGrammarItemsWalker.grammar_(DefineGrammarItemsWalker.java:275)
   	at org.antlr.tool.Grammar.defineGrammarSymbols(Grammar.java:748)
   	at org.antlr.tool.CompositeGrammar.defineGrammarSymbols
   (CompositeGrammar.java:369)
   	at org.antlr.tool.Grammar.setGrammarContent(Grammar.java:606)
   	at
   org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.createNewGrammar
   (ANTLRGrammarEngineImpl.java:192)
   	at
   org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.createParserGrammar
   (ANTLRGrammarEngineImpl.java:225)
   	at
   org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.createGrammars
   (ANTLRGrammarEngineImpl.java:169)
   	at org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.analyze
   (ANTLRGrammarEngineImpl.java:272)
   	at org.antlr.works.grammar.engine.GrammarEngineImpl.analyze
   (GrammarEngineImpl.java:325)
   	at org.antlr.works.grammar.CheckGrammar.run(CheckGrammar.java:70)
   	at java.lang.Thread.run(Thread.java:736)

   [16:35:27] error(100): [snipped]Parser.g:3825:17: syntax error: antlr:
   NoViableAltException(72@[722:4: ( ( id ( ASSIGN | PLUS_ASSIGN ) ( atom |
   block ) ) (sub= ebnfSuffix[root_0,false] )? | a= atom (sub2= ebnfSuffix
   [$a.tree,false] )? | ebnf | FORCED_ACTION | ACTION | p= SEMPRED
   ( IMPLIES )? | t3= tree_ )])
   [16:35:27] error(100): [snipped]Parser.g:3825:17: syntax error: antlr:
   NoViableAltException(72@[722:4: ( ( id ( ASSIGN | PLUS_ASSIGN ) ( atom |
   block ) ) (sub= ebnfSuffix[root_0,false] )? | a= atom (sub2= ebnfSuffix
   [$a.tree,false] )? | ebnf | FORCED_ACTION | ACTION | p= SEMPRED
   ( IMPLIES )? | t3= tree_ )])
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(0!=29)
   [16:35:27] error(100):[snipped]Parser.g:0:: syntax error: assign.types:
   NoViableAltException(3@[])
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(3!=28)
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(3!=27)
   [16:35:27] error(100): [snipped]Parser.g:0:1: syntax error:
   assign.types: MismatchedTreeNodeException(26!=3)
   [16:35:27] error(10):  internal error: [snipped]Parser.g :
   java.lang.ClassCastException: org.antlr.runtime.tree.CommonTree
   incompatible with org.antlr.tool.GrammarAST
   org.antlr.grammar.v3.DefineGrammarItemsWalker.rewrite
   (DefineGrammarItemsWalker.java:3463)
   org.antlr.grammar.v3.DefineGrammarItemsWalker.block
   (DefineGrammarItemsWalker.java:2072)
   org.antlr.grammar.v3.DefineGrammarItemsWalker.rule
   (DefineGrammarItemsWalker.java:1627)
   org.antlr.grammar.v3.DefineGrammarItemsWalker.rules
   (DefineGrammarItemsWalker.java:1190)
   org.antlr.grammar.v3.DefineGrammarItemsWalker.grammarSpec
   (DefineGrammarItemsWalker.java:623)
   org.antlr.grammar.v3.DefineGrammarItemsWalker.grammar_(DefineGrammarItemsWalker.java:275)
   org.antlr.tool.Grammar.defineGrammarSymbols(Grammar.java:748)
   org.antlr.tool.CompositeGrammar.defineGrammarSymbols
   (CompositeGrammar.java:369)
   org.antlr.Tool.process(Tool.java:465)
   org.antlr.works.generate.CodeGenerate.generate(CodeGenerate.java:104)
   org.antlr.works.generate.CodeGenerate.run(CodeGenerate.java:185)
   java.lang.Thread.run(Thread.java:736)

It is not very clear to me if I need to pursue this as a
NoViableAltException, a  MismatchedTreeNodeException, or a
ClassCastException.

I'm looking at the existing code, and I'm not sure I would have handled
(expression) in the same way, but I don't have time to change everything I
don't like.

I'm looking at some potential matches in the problem space:

________________
Why do I get a ClassCastException when parsing a tree?
http://www.antlr.org/wiki/pages/viewpage.action?pageId=23232571
// but this appears to be happening at runtime and mine is a generation
time problem.
________________
Tree Rewrite Rules Replace Rewrite Actions

While you can still use the ^ and ! tree construction operators to build
trees, v3 introduces an entirely new syntax for Tree construction that
avoids the special syntax was used in ANTLR 2 actions:

                               ANTLR 2                                                    ANTLR 3                      

 arrayLiteral                                                        arrayLiteral                                      
     :    LBRACK! (elementList)? RBRACK!                                 :    LBRACK (elementList)? RBRACK             
         {## = #([ARRAY_LITERAL, "ARRAY_LITERAL"],##);}                      -> ^(ARRAY_LITERAL elementList)           
     ;                                                                   ;                                             

http://www.antlr.org/wiki/display/ANTLR3/Migrating+from+ANTLR+2+to+ANTLR+3

// But, while this style of grammar appears in the first alt, it does not
appear on the problematic line.
_____________________

This looks promising, but 722:4 does not have any errors in it according to
ANTLRWorks, and there should be no relationship between the rules for CAST
and what is defined there.
http://www.antlr.org/pipermail/antlr-interest/2011-June/041789.html

Any suggestions?

Chris

"Ah, because I have learned something since last week."  - Gandhi