[antlr-interest] Addendum: Trying to get an AST in C#. Not working - could use a sanity check.

Kevin Carroll kcarroll at signmgmt.com
Mon May 7 09:14:45 PDT 2012


Peter,

Change HIDDEN to Hidden.  The C# target attempts to use C# idioms when possible.  Also, look at http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases.

Kevin

-----Original Message-----
From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Peter Bosch
Sent: Sunday, May 06, 2012 7:53 PM
To: antlr-interest at antlr.org
Subject: [antlr-interest] Addendum: Trying to get an AST in C#. Not working - could use a sanity check.

Also, I tried only libraries and tools from antlr-dotnet-csharpbootstrap-3.4.1.9004.7z, per advice from http://www.antlr.org/pipermail/antlr-interest/2012-April/044370.html and got the following:

 

[With language=CSharp2;]

C:\Users\pbosch\Desktop\antlr-dotnet-csharpbootstrap-3.4.1.9004>Antlr3
Marbles.g

AST.stg 54:23: premature EOF

 

[With language=CSharp3;]

C:\Users\pbosch\Desktop\antlr-dotnet-csharpbootstrap-3.4.1.9004>Antlr3
Marbles.g

 

C:\Users\pbosch\Desktop\antlr-dotnet-csharpbootstrap-3.4.1.9004>

 

For both versions, I used the generated parser & lexer classes, and got the following errors both times:

The type name 'parse_return' does not exist in the type 'Marbles.MarblesParser'

'Marbles.MarblesParser.parse()' is inaccessible due to its protection level

The name 'HIDDEN' does not exist in the current context ß This was in generated code.

The name 'HIDDEN' does not exist in the current context ß This was in generated code.

 

This was with 

 

 

From: Peter Bosch [mailto:pbosch at highpointsoftware.com]
Sent: Sunday, May 06, 2012 7:19 PM
To: 'antlr-interest at antlr.org'
Subject: Trying to get an AST in C#. Not working - could use a sanity check.

 

Hi;

 

I'll preface with "I am a first time Lexer/Parser generator user, but a long time software guy." That, of course, makes me a "noob."

 

I'm preparing to build an analysis and round-trip engineering tool in C# for a 3rd party system with its own syntax. I believe I can create the foundation of a persistence layer to and from their custom format through an ANTLR grammar. In order to work out the kinks, I've begun a proof-of-concept on a simple analogue. One thing it needs to do is support an N-ary AST where
N>2, but I think I have reasonable rewrite rules in place to bring that
about.

 

The challenge has been in getting an AST generated from even that simple grammar. I have a grammar that passes code generation from ANTLRWorks, and code (see attached) that seems to have worked for others, in regards to generating an AST.

 

It may have to do with the versions of ANTLRWorks and the ANTLR C# runtimes that I'm linking to. I've tried:

ANTLR 3.1.0 C# Runtimes version (3.1.0.39271) & ANTLRWorks 1.4.3 : Odd partial parsing - skips or duplicates some children.

ANTLR 3.4.1 C# Runtimes version (3.4.1.9004) & ANTLRWorks 1.4.3 : Generated lexer issues (_channel=HIDDEN ßnot defined) and program issues
(MarblesParser.parse() inaccessible and MarblesParser.parse_return not
defined.)

 

So my question is - *Should* either of these configurations work? (I've seen posts hinting that certain builds of ANTLR C# runtime are not fully-featured, and I'm also wondering if there might have been breaking changes between 3.1.0.x and 3.4.1.x such that my source needs to change to support a different model.)

 

If things *should* work as I've got 'em, I've inlined the grammar, the sample input and the Program.cs below in case anyone cares to beat me about the head and shoulders for some noob-ish-nesses. J 

 

[Note: A sack of marbles was the initial concept. You'll see below, we've now graduated to bags of hammers & boxes of rocks. A subject of which I am currently feeling quite exemplary.]

 

In any case, thanks in advance for whatever help you can offer.

 

Regards, (code after the .sig) 

 

Peter Bosch
Highpoint Software Systems
E-mail : pbosch at highpointsoftware.com
Phone : 262-893-5400
P-Mail : S42 W27451 Oak Grove Lane Waukesha, WI, 53189 Useful and Usable Decision Support & Technical Software

 

 

////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////

Marbles.g

////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////

 

grammar Marbles;

 

// options, then tokens, then @header then @members.

options {

  language=CSharp2;

  output=AST;

}

 

tokens {

    ROOT;

}

 

@header {

    //using System;

}

 

@parser::namespace { Marbles }

@lexer::namespace { Marbles }

 

@parser::members { protected void Print(string s){
/*Console.WriteLine("Token: " + s);*/ } }

 

// Parser Rules.

 

parse

    :  r=box* EOF -> ^(ROOT $r*)

    ;

 

color : K_COLOR '=' s=STRINGLITERAL {Print("C");} ->^(K_COLOR $s) 

    ;

 

weight     : K_WEIGHT '=' w=INTEGER -> ^(K_WEIGHT $w)

    ;

 

htype : K_HTYPE '=' t=STRINGLITERAL -> ^(K_HTYPE $t)

    ;

 

rtype : K_RTYPE '=' t=STRINGLITERAL -> ^(K_RTYPE $t)

    ;

    

rock : 

    '{' K_ROCK c=color w=weight t=rtype '}' {Print("R");} ->^(K_ROCK $c $w
$t)

    ;

    

hammer     :

    '{' K_HAMMER t=htype? w=weight? '}'  ->^(K_HAMMER $t? $w?)

    ;

    

box  :

    '[' K_BOX r=rock* ']' {Print("B");} -> ^(K_BOX $r*)

    ;

    

bag  :

    '[' K_HAMMER h=(hammer)* ']' -> ^(K_HAMMER $h*)

    ;

    

 

 

// Lexer Rules

EOL  : ('\r'? '\n') {$channel=HIDDEN;}

    ;

    

WS      : (' '|'\t'|'\f'| EOL)+ {$channel=HIDDEN;}

    ;

    

K_BOX : 'BOX'

    ;

K_BAG : 'BAG'

    ;

K_ROCK     : 'ROCK'

    ;

K_COLOR    : 'COLOR'

    ;

K_HAMMER

    : 'HAMMER'

    ;

K_HTYPE    : 'H_TYPE'

    ;

K_RTYPE    : 'R_TYPE'

    ;

K_WEIGHT

    : 'WEIGHT'

    ;

    

STRINGLITERAL

    : '"' (~('"' | '\\'))* '"'

    ;

    

INTEGER    : ('0'..'9')+

    ;

 

COMMENT

    : '//' ~('\n'|'\r')* '\r'? '\n' {Skip();} //{$channel=HIDDEN;}

    | '/*' ( options {greedy=false;} : . )* '*/' {Skip();} //{$channel=HIDDEN;}

    ;

 

    

////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////

SampleMarbles.txt

////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////

/*[BAG 

{ HAMMER H_TYPE="Claw";} // Bob's your uncle.

{ HAMMER H_TYPE="Ball Peen" WEIGHT=12 ; }

]*/

 

[BOX

{ ROCK COLOR="Blue" WEIGHT=41 R_TYPE="Schist" }

{ ROCK COLOR="Yellow" WEIGHT=31 R_TYPE="Sulfur" }

{ ROCK COLOR="Black" WEIGHT=21 R_TYPE="Quartz" }

]

[BOX

{ ROCK COLOR="Blue" WEIGHT=42 R_TYPE="Schist" }

{ ROCK COLOR="Yellow" WEIGHT=32 R_TYPE="Sulfur" }

{ ROCK COLOR="Black" WEIGHT=22 R_TYPE="Quartz" }

]

[BOX

{ ROCK COLOR="Blue" WEIGHT=43 R_TYPE="Schist" }

{ ROCK COLOR="Yellow" WEIGHT=33 R_TYPE="Sulfur" }

{ ROCK COLOR="Black" WEIGHT=23 R_TYPE="Quartz" }

]

 

 

////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////

Program.cs

////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////

using System;

using Antlr.Runtime;

using Antlr.Runtime.Tree;

//using Antlr.StringTemplate;

using System.IO;

 

// http://stackoverflow.com/questions/4396080/antlr-3-3-c-sharp-tutorials

//
http://stackoverflow.com/questions/4396080/antlr-3-3-c-sharp-tutorials/43977
99#4397799

namespace Marbles {

    class MainClass {

        public static void Preorder(ITree Tree, int Depth) {

            if (Tree == null) {

                return;

            }

 

            for (int i = 0; i < Depth; i++) {

                Console.Write("  ");

            }

 

            Console.WriteLine(Tree);

 

            for (int i = 0; i < Tree.ChildCount; i++) {

                Preorder(Tree.GetChild(i), Depth + 1);

            }

        }

 

        public static void Main(string[] args) {

            string s = File.ReadAllText(args[0]);

            ANTLRStringStream Input = new ANTLRStringStream(s);

            MarblesLexer Lexer = new MarblesLexer(Input);

            CommonTokenStream Tokens = new CommonTokenStream(Lexer);

            MarblesParser Parser = new MarblesParser(Tokens);

            MarblesParser.parse_return ParseReturn = Parser.parse();

            CommonTree Tree = (CommonTree)ParseReturn.Tree;

            Preorder(Tree, 0); 

        }

    }

}


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address


More information about the antlr-interest mailing list