[antlr-interest] 3.1b1 trouble with a CSharp2/AST example

Johannes Luber JALuber at gmx.de
Fri May 23 05:20:56 PDT 2008


> If you have any suggestions of where to look for the problem (and any 
> suggestions on what the problem might be) I am happy to follow up on 
> them, I need to get this working.

The first thing I'd check, if the Java code generated by the same grammar has the same defect. You may have to change actions, too, but if the cause isn't your own code, the content of them is ignorable. Indeed, it would be a good idea to remove all actions in C# variant first and see if a clean version still fails.

After that, if you determine that Java doesn't work, send Ter the full grammar, so he can fix the templates (maybe even the C# variants, otherwise you have to do it yourself). If the Java code works, then compare the generated code for differences. It may be possible, that I lost a line somewhere when syncing with Java. If you find a difference beyond syntactic Java/C# changes, then change the C# template file at the appropriate place(s) and recreate ANTLR, regenerate the files and test them. Usually that does work.

If you don't find a difference in the output files, then the mistake is in the runtime somewhere. Look for the crash, determine why at that place the code blew up and track the wrong state back to its origin. Then compare all Java runtime files touched by the wrong state from the beginning with the C# equivalents. Somewhere there has to be the difference.

I hope that should be enough advice to deal with the problem. :)
> 
> Oh, um... I do have ANT 1.7 and JDK 1.6 installed but when I tried to 
> "ant build" the 3.1b1 distribution I got a mess of errors.  I didn't 
> look very closely at them, I just re-unpacked the distro since mostly I 
> care about using antlr rather than building it.  If I start to track 
> down this one bug I may have to start messing with other build problems 
> and Java isn't my strong point (thus my use of the C# targets to begin 
> with), but c'est la vie.  If you're interested in _those_ error logs I 
> can post them.
> 
> I will also try this with the CSharp target (vs. CSharp2)... yep, same 
> error and raised in the same place.

CSharp and CSharp2 differ currently at only four tested places, so the templates don't have to be checked both, unless you want to test the .NET 1.1.-only parts of the runtime.

Johannes
> 
> thanks,
> hamilton
> 
> Johannes Luber wrote:
> > Hi!
> >
> > My apologies: Right now, I do not have access to a machine with the
> required development tools, so I can't go bug hunting. Also I don't know how
> long that state will continue. :( I will add the report to JIRA, so I don't
> forget it. Maybe you or someone else hunt the bug down?
> >
> > Johannes
> >
> >   
> >> I'm having trouble producing an abstract syntax tree with ANTLR. 
> Lexing
> >> is working fine, parsing without any translation is working as far as I
> can
> >> tell as well.  Parsing using a grammer with rewrite rules in it dies.
> >>
> >> Here's what I'm doing.
> >>
> >> I scarfed the polynomial example from the web site and stuck it in a
> >> SimpleExpressions.g test file:
> >>
> >> ------
> >>
> >> grammar SimpleExpressions;
> >>
> >> options { language = CSharp2; output = AST; }
> >>
> >> tokens { MULT; } // imaginary token
> >>
> >> start: term ('+'^ term)*
> >>
> >> ;
> >>
> >> term: INT ID -> ^(MULT["*"] INT ID)
> >>
> >> | INT exp -> ^(MULT["*"] INT exp)
> >>
> >> | exp
> >>
> >> | INT
> >>
> >> | ID
> >>
> >> ;
> >>
> >> exp : ID '^'^ INT
> >>
> >> ;
> >>
> >> ID : 'a'..'z'+ ;
> >>
> >> INT : '0'..'9'+ ;
> >>
> >> WS : (' '|'\t'|'\r'|'\n')+ {$channel = HIDDEN;} ;
> >>
> >> ------
> >>
> >>
> >>
> >> I ran the 3.1b1 version of antlr on it, from the command line, and it
> >> produced my .cs files for the lexer and parser without any errors:
> >>
> >>
> >>
> >> $ java -cp
> >> "./antlr-3.1b1.jar;./stringtemplate-3.1.jar;./antlr-runtime-3.1b1.ja
> >> r;./antlr-2.7.7.jar;./gunit-1.0.2.jar" org.antlr.Tool "C:\Documents and
> >> Setting
> >> s\helink\My Documents\Visual Studio
> >> 2008\Projects\Examples\ANTLRTest\TestParser
> >> \SimpleExpressions.g"
> >> ANTLR Parser Generator  Version 3.1b1 (May 20, 2008)  1989-2008
> >>
> >> $
> >>
> >>
> >>
> >> I built a sample C# application and it compiles just fine (using
> >> antlr.runtime, Antlr3.Runtime, Antlr3.Utility, and StringTemplate as
> references).
> >> My main program.cs file is as follows, and it is built with the
> >> SimpleExpressionsLexer.cs and SimpleExpressionsParser.cs.
> >>
> >>
> >>
> >> ------
> >>
> >> using System;
> >>
> >> using System.Collections.Generic;
> >>
> >> using System.Linq;
> >>
> >> using System.Text;
> >>
> >> using Antlr.Runtime;
> >>
> >> using Antlr.Runtime.Tree;
> >>
> >> using Antlr.Runtime.Debug;
> >>
> >> using Antlr.Utility.Tree;
> >>
> >> namespace TestParser
> >>
> >> {
> >>
> >> class Program
> >>
> >> {
> >>
> >> static void Main(string[] args)
> >>
> >> {
> >>
> >> string testString = "2x+3x^5";
> >>
> >> SimpleExpressionsLexer lexer = new SimpleExpressionsLexer(new
> >> ANTLRStringStream(testString));
> >>
> >> CommonTokenStream tokens = new CommonTokenStream(lexer);
> >>
> >> SimpleExpressionsParser parser = new SimpleExpressionsParser(tokens);
> >>
> >> SimpleExpressionsParser.start_return bar = parser.start();
> >>
> >> CommonTree t = (CommonTree)bar.Tree;
> >>
> >> String result = (t != null) ? t.ToStringTree() : null;
> >>
> >> }
> >>
> >> }
> >>
> >> }
> >>
> >> ------
> >>
> >>
> >>
> >> When I run it, I get a "RewriteEmptyStreamException was unhandled"
> error
> >> with the message "token INT".  Here is the exception detail:
> >>
> >>
> >>
> >> Antlr.Runtime.Tree.RewriteEmptyStreamException was unhandled
> >>   Message="token INT"
> >>   Source="Antlr3.Runtime"
> >>   StackTrace:
> >>        at Antlr.Runtime.Tree.RewriteRuleElementStream`1._Next()
> >>        at Antlr.Runtime.Tree.RewriteRuleTokenStream.NextNode()
> >>        at SimpleExpressionsParser.term() in C:\Documents and
> >> Settings\helink\My Documents\Visual Studio
> >>
> 2008\Projects\Examples\ANTLRTest\TestParser\SimpleExpressionsParser.cs:line 305
> >>        at SimpleExpressionsParser.start() in C:\Documents and
> >> Settings\helink\My Documents\Visual Studio
> >>
> 2008\Projects\Examples\ANTLRTest\TestParser\SimpleExpressionsParser.cs:line 104
> >>        at TestParser.Program.Main(String[] args) in C:\Documents and
> >> Settings\helink\My Documents\Visual Studio
> >> 2008\Projects\Examples\ANTLRTest\TestParser\Program.cs:line 20
> >>        at System.AppDomain._nExecuteAssembly(Assembly assembly,
> String[]
> >> args)
> >>        at System.AppDomain.ExecuteAssembly(String assemblyFile,
> Evidence
> >> assemblySecurity, String[] args)
> >>        at
> >> Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
> >>        at System.Threading.ThreadHelper.ThreadStart_Context(Object
> state)
> >>        at System.Threading.ExecutionContext.Run(ExecutionContext
> >> executionContext, ContextCallback callback, Object state)
> >>        at System.Threading.ThreadHelper.ThreadStart()
> >>   InnerException:
> >>
> >> in other words it's dying in term() as called by start() as called by
> my
> >> main program, i.e. it's dying trying to parse.  The code around the
> error
> >> (at the break point) in Visual Studio is....
> >>
> >>
> >>
> >> // 10:15: -> ^( MULT[\"*\"] INT ID )
> >>
> >> {
> >>
> >> // C:\\Documents and Settings\\helink\\My Documents\\Visual Studio
> >>
> 2008\\Projects\\Examples\\ANTLRTest\\TestParser\\SimpleExpressions.g:10:18: ^(
> >> MULT[\"*\"] INT ID )
> >>
> >> {
> >>
> >> object root_1 = (object)adaptor.GetNilNode();
> >>
> >> root_1 = (object)adaptor.BecomeRoot((object)adaptor.Create(MULT, "*"),
> >> root_1);
> >>
> >> adaptor.AddChild(root_1, stream_INT.NextNode());         // <--- It's
> >> dying here, in SimpleExpressionsParser.cs
> >>
> >> adaptor.AddChild(root_1, stream_ID.NextNode());
> >>
> >> adaptor.AddChild(root_0, root_1);
> >>
> >> }
> >>
> >>
> >>
> >> If I get rid of the rewrite rules or whatever you call them, the -> and
> ^
> >> etc. stuff, and regenerate and recompile, it runs fine but of course it
> >> doesn't produce a useful AST, it just produces t==null.
> >>
> >>
> >>
> >> Any help getting this simple example to run would be greatly
> appreciated.
> >> At the moment however I'm suspecting either a bug in the example or in
> the
> >> CSharp2 target, as there's nothing of my own making in the picture
> (even
> >> the main program is cribbed from the web site example code).
> >>
> >>
> >>
> >> My immediate goal is to get parse trees and ASTs generated from fairly
> >> simple expression grammars.  My ultimate goal is to build heterogenous
> node
> >> trees and write code to evaluate them, i.e. my ultimate goal is to make
> a
> >> fairly simple expression interpreter.  I think if I can get the
> polynomial
> >> example working I'll be a great deal closer to that goal.
> >>
> >>
> >>
> >> thanks,
> >>
> >> hamilton
> >>
> >>
> >>
> >>
> >>
> >>
> >>     
> >
> > --
> > GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
> > Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf@gmx
> >
> >
> >   
> 

-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger


More information about the antlr-interest mailing list