[antlr-interest] 3.1b1 trouble with a CSharp2/AST example

Johannes Luber JALuber at gmx.de
Thu May 22 14:53:24 PDT 2008


Hi!

My apologies: Right now, I do not have access to a machine with the required development tools, so I can't go bug hunting. Also I don't know how long that state will continue. :( I will add the report to JIRA, so I don't forget it. Maybe you or someone else hunt the bug down?

Johannes

> I'm having trouble producing an abstract syntax tree with ANTLR.  Lexing
> is working fine, parsing without any translation is working as far as I can
> tell as well.  Parsing using a grammer with rewrite rules in it dies.
> 
> Here's what I'm doing.
> 
> I scarfed the polynomial example from the web site and stuck it in a
> SimpleExpressions.g test file:
> 
> ------
> 
> grammar SimpleExpressions;
> 
> options { language = CSharp2; output = AST; }
> 
> tokens { MULT; } // imaginary token
> 
> start: term ('+'^ term)*
> 
> ;
> 
> term: INT ID -> ^(MULT["*"] INT ID)
> 
> | INT exp -> ^(MULT["*"] INT exp)
> 
> | exp
> 
> | INT
> 
> | ID
> 
> ;
> 
> exp : ID '^'^ INT
> 
> ;
> 
> ID : 'a'..'z'+ ;
> 
> INT : '0'..'9'+ ;
> 
> WS : (' '|'\t'|'\r'|'\n')+ {$channel = HIDDEN;} ;
> 
> ------
> 
> 
> 
> I ran the 3.1b1 version of antlr on it, from the command line, and it
> produced my .cs files for the lexer and parser without any errors:
> 
> 
> 
> $ java -cp
> "./antlr-3.1b1.jar;./stringtemplate-3.1.jar;./antlr-runtime-3.1b1.ja
> r;./antlr-2.7.7.jar;./gunit-1.0.2.jar" org.antlr.Tool "C:\Documents and
> Setting
> s\helink\My Documents\Visual Studio
> 2008\Projects\Examples\ANTLRTest\TestParser
> \SimpleExpressions.g"
> ANTLR Parser Generator  Version 3.1b1 (May 20, 2008)  1989-2008
> 
> $
> 
> 
> 
> I built a sample C# application and it compiles just fine (using
> antlr.runtime, Antlr3.Runtime, Antlr3.Utility, and StringTemplate as references). 
> My main program.cs file is as follows, and it is built with the
> SimpleExpressionsLexer.cs and SimpleExpressionsParser.cs.
> 
> 
> 
> ------
> 
> using System;
> 
> using System.Collections.Generic;
> 
> using System.Linq;
> 
> using System.Text;
> 
> using Antlr.Runtime;
> 
> using Antlr.Runtime.Tree;
> 
> using Antlr.Runtime.Debug;
> 
> using Antlr.Utility.Tree;
> 
> namespace TestParser
> 
> {
> 
> class Program
> 
> {
> 
> static void Main(string[] args)
> 
> {
> 
> string testString = "2x+3x^5";
> 
> SimpleExpressionsLexer lexer = new SimpleExpressionsLexer(new
> ANTLRStringStream(testString));
> 
> CommonTokenStream tokens = new CommonTokenStream(lexer);
> 
> SimpleExpressionsParser parser = new SimpleExpressionsParser(tokens);
> 
> SimpleExpressionsParser.start_return bar = parser.start();
> 
> CommonTree t = (CommonTree)bar.Tree;
> 
> String result = (t != null) ? t.ToStringTree() : null;
> 
> }
> 
> }
> 
> }
> 
> ------
> 
> 
> 
> When I run it, I get a "RewriteEmptyStreamException was unhandled" error
> with the message "token INT".  Here is the exception detail:
> 
> 
> 
> Antlr.Runtime.Tree.RewriteEmptyStreamException was unhandled
>   Message="token INT"
>   Source="Antlr3.Runtime"
>   StackTrace:
>        at Antlr.Runtime.Tree.RewriteRuleElementStream`1._Next()
>        at Antlr.Runtime.Tree.RewriteRuleTokenStream.NextNode()
>        at SimpleExpressionsParser.term() in C:\Documents and
> Settings\helink\My Documents\Visual Studio
> 2008\Projects\Examples\ANTLRTest\TestParser\SimpleExpressionsParser.cs:line 305
>        at SimpleExpressionsParser.start() in C:\Documents and
> Settings\helink\My Documents\Visual Studio
> 2008\Projects\Examples\ANTLRTest\TestParser\SimpleExpressionsParser.cs:line 104
>        at TestParser.Program.Main(String[] args) in C:\Documents and
> Settings\helink\My Documents\Visual Studio
> 2008\Projects\Examples\ANTLRTest\TestParser\Program.cs:line 20
>        at System.AppDomain._nExecuteAssembly(Assembly assembly, String[]
> args)
>        at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence
> assemblySecurity, String[] args)
>        at
> Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
>        at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
>        at System.Threading.ExecutionContext.Run(ExecutionContext
> executionContext, ContextCallback callback, Object state)
>        at System.Threading.ThreadHelper.ThreadStart()
>   InnerException:
> 
> in other words it's dying in term() as called by start() as called by my
> main program, i.e. it's dying trying to parse.  The code around the error
> (at the break point) in Visual Studio is....
> 
> 
> 
> // 10:15: -> ^( MULT[\"*\"] INT ID )
> 
> {
> 
> // C:\\Documents and Settings\\helink\\My Documents\\Visual Studio
> 2008\\Projects\\Examples\\ANTLRTest\\TestParser\\SimpleExpressions.g:10:18: ^(
> MULT[\"*\"] INT ID )
> 
> {
> 
> object root_1 = (object)adaptor.GetNilNode();
> 
> root_1 = (object)adaptor.BecomeRoot((object)adaptor.Create(MULT, "*"),
> root_1);
> 
> adaptor.AddChild(root_1, stream_INT.NextNode());         // <--- It's
> dying here, in SimpleExpressionsParser.cs
> 
> adaptor.AddChild(root_1, stream_ID.NextNode());
> 
> adaptor.AddChild(root_0, root_1);
> 
> }
> 
> 
> 
> If I get rid of the rewrite rules or whatever you call them, the -> and ^
> etc. stuff, and regenerate and recompile, it runs fine but of course it
> doesn't produce a useful AST, it just produces t==null.
> 
> 
> 
> Any help getting this simple example to run would be greatly appreciated. 
> At the moment however I'm suspecting either a bug in the example or in the
> CSharp2 target, as there's nothing of my own making in the picture (even
> the main program is cribbed from the web site example code).
> 
> 
> 
> My immediate goal is to get parse trees and ASTs generated from fairly
> simple expression grammars.  My ultimate goal is to build heterogenous node
> trees and write code to evaluate them, i.e. my ultimate goal is to make a
> fairly simple expression interpreter.  I think if I can get the polynomial
> example working I'll be a great deal closer to that goal.
> 
> 
> 
> thanks,
> 
> hamilton
> 
> 
> 
> 
> 
> 

-- 
GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf@gmx


More information about the antlr-interest mailing list