[antlr-interest] 3.1b1 trouble with a CSharp2/AST example

Hamilton Link helink at sandia.gov
Thu May 22 15:04:49 PDT 2008


If you have any suggestions of where to look for the problem (and any 
suggestions on what the problem might be) I am happy to follow up on 
them, I need to get this working.

Oh, um... I do have ANT 1.7 and JDK 1.6 installed but when I tried to 
"ant build" the 3.1b1 distribution I got a mess of errors.  I didn't 
look very closely at them, I just re-unpacked the distro since mostly I 
care about using antlr rather than building it.  If I start to track 
down this one bug I may have to start messing with other build problems 
and Java isn't my strong point (thus my use of the C# targets to begin 
with), but c'est la vie.  If you're interested in _those_ error logs I 
can post them.

I will also try this with the CSharp target (vs. CSharp2)... yep, same 
error and raised in the same place.

thanks,
hamilton

Johannes Luber wrote:
> Hi!
>
> My apologies: Right now, I do not have access to a machine with the required development tools, so I can't go bug hunting. Also I don't know how long that state will continue. :( I will add the report to JIRA, so I don't forget it. Maybe you or someone else hunt the bug down?
>
> Johannes
>
>   
>> I'm having trouble producing an abstract syntax tree with ANTLR.  Lexing
>> is working fine, parsing without any translation is working as far as I can
>> tell as well.  Parsing using a grammer with rewrite rules in it dies.
>>
>> Here's what I'm doing.
>>
>> I scarfed the polynomial example from the web site and stuck it in a
>> SimpleExpressions.g test file:
>>
>> ------
>>
>> grammar SimpleExpressions;
>>
>> options { language = CSharp2; output = AST; }
>>
>> tokens { MULT; } // imaginary token
>>
>> start: term ('+'^ term)*
>>
>> ;
>>
>> term: INT ID -> ^(MULT["*"] INT ID)
>>
>> | INT exp -> ^(MULT["*"] INT exp)
>>
>> | exp
>>
>> | INT
>>
>> | ID
>>
>> ;
>>
>> exp : ID '^'^ INT
>>
>> ;
>>
>> ID : 'a'..'z'+ ;
>>
>> INT : '0'..'9'+ ;
>>
>> WS : (' '|'\t'|'\r'|'\n')+ {$channel = HIDDEN;} ;
>>
>> ------
>>
>>
>>
>> I ran the 3.1b1 version of antlr on it, from the command line, and it
>> produced my .cs files for the lexer and parser without any errors:
>>
>>
>>
>> $ java -cp
>> "./antlr-3.1b1.jar;./stringtemplate-3.1.jar;./antlr-runtime-3.1b1.ja
>> r;./antlr-2.7.7.jar;./gunit-1.0.2.jar" org.antlr.Tool "C:\Documents and
>> Setting
>> s\helink\My Documents\Visual Studio
>> 2008\Projects\Examples\ANTLRTest\TestParser
>> \SimpleExpressions.g"
>> ANTLR Parser Generator  Version 3.1b1 (May 20, 2008)  1989-2008
>>
>> $
>>
>>
>>
>> I built a sample C# application and it compiles just fine (using
>> antlr.runtime, Antlr3.Runtime, Antlr3.Utility, and StringTemplate as references).
>> My main program.cs file is as follows, and it is built with the
>> SimpleExpressionsLexer.cs and SimpleExpressionsParser.cs.
>>
>>
>>
>> ------
>>
>> using System;
>>
>> using System.Collections.Generic;
>>
>> using System.Linq;
>>
>> using System.Text;
>>
>> using Antlr.Runtime;
>>
>> using Antlr.Runtime.Tree;
>>
>> using Antlr.Runtime.Debug;
>>
>> using Antlr.Utility.Tree;
>>
>> namespace TestParser
>>
>> {
>>
>> class Program
>>
>> {
>>
>> static void Main(string[] args)
>>
>> {
>>
>> string testString = "2x+3x^5";
>>
>> SimpleExpressionsLexer lexer = new SimpleExpressionsLexer(new
>> ANTLRStringStream(testString));
>>
>> CommonTokenStream tokens = new CommonTokenStream(lexer);
>>
>> SimpleExpressionsParser parser = new SimpleExpressionsParser(tokens);
>>
>> SimpleExpressionsParser.start_return bar = parser.start();
>>
>> CommonTree t = (CommonTree)bar.Tree;
>>
>> String result = (t != null) ? t.ToStringTree() : null;
>>
>> }
>>
>> }
>>
>> }
>>
>> ------
>>
>>
>>
>> When I run it, I get a "RewriteEmptyStreamException was unhandled" error
>> with the message "token INT".  Here is the exception detail:
>>
>>
>>
>> Antlr.Runtime.Tree.RewriteEmptyStreamException was unhandled
>>   Message="token INT"
>>   Source="Antlr3.Runtime"
>>   StackTrace:
>>        at Antlr.Runtime.Tree.RewriteRuleElementStream`1._Next()
>>        at Antlr.Runtime.Tree.RewriteRuleTokenStream.NextNode()
>>        at SimpleExpressionsParser.term() in C:\Documents and
>> Settings\helink\My Documents\Visual Studio
>> 2008\Projects\Examples\ANTLRTest\TestParser\SimpleExpressionsParser.cs:line 305
>>        at SimpleExpressionsParser.start() in C:\Documents and
>> Settings\helink\My Documents\Visual Studio
>> 2008\Projects\Examples\ANTLRTest\TestParser\SimpleExpressionsParser.cs:line 104
>>        at TestParser.Program.Main(String[] args) in C:\Documents and
>> Settings\helink\My Documents\Visual Studio
>> 2008\Projects\Examples\ANTLRTest\TestParser\Program.cs:line 20
>>        at System.AppDomain._nExecuteAssembly(Assembly assembly, String[]
>> args)
>>        at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence
>> assemblySecurity, String[] args)
>>        at
>> Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
>>        at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
>>        at System.Threading.ExecutionContext.Run(ExecutionContext
>> executionContext, ContextCallback callback, Object state)
>>        at System.Threading.ThreadHelper.ThreadStart()
>>   InnerException:
>>
>> in other words it's dying in term() as called by start() as called by my
>> main program, i.e. it's dying trying to parse.  The code around the error
>> (at the break point) in Visual Studio is....
>>
>>
>>
>> // 10:15: -> ^( MULT[\"*\"] INT ID )
>>
>> {
>>
>> // C:\\Documents and Settings\\helink\\My Documents\\Visual Studio
>> 2008\\Projects\\Examples\\ANTLRTest\\TestParser\\SimpleExpressions.g:10:18: ^(
>> MULT[\"*\"] INT ID )
>>
>> {
>>
>> object root_1 = (object)adaptor.GetNilNode();
>>
>> root_1 = (object)adaptor.BecomeRoot((object)adaptor.Create(MULT, "*"),
>> root_1);
>>
>> adaptor.AddChild(root_1, stream_INT.NextNode());         // <--- It's
>> dying here, in SimpleExpressionsParser.cs
>>
>> adaptor.AddChild(root_1, stream_ID.NextNode());
>>
>> adaptor.AddChild(root_0, root_1);
>>
>> }
>>
>>
>>
>> If I get rid of the rewrite rules or whatever you call them, the -> and ^
>> etc. stuff, and regenerate and recompile, it runs fine but of course it
>> doesn't produce a useful AST, it just produces t==null.
>>
>>
>>
>> Any help getting this simple example to run would be greatly appreciated.
>> At the moment however I'm suspecting either a bug in the example or in the
>> CSharp2 target, as there's nothing of my own making in the picture (even
>> the main program is cribbed from the web site example code).
>>
>>
>>
>> My immediate goal is to get parse trees and ASTs generated from fairly
>> simple expression grammars.  My ultimate goal is to build heterogenous node
>> trees and write code to evaluate them, i.e. my ultimate goal is to make a
>> fairly simple expression interpreter.  I think if I can get the polynomial
>> example working I'll be a great deal closer to that goal.
>>
>>
>>
>> thanks,
>>
>> hamilton
>>
>>
>>
>>
>>
>>
>>     
>
> --
> GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
> Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf@gmx
>
>
>   




More information about the antlr-interest mailing list