[antlr-interest] [CSharp3] rule visibility in composite grammars

Wed Jun 8 22:53:18 PDT 2011

I'm also very interested in ways to make ANTLR grammars
target-language-agnostic. I know ANTLR version 4 will provide more
consistent semantics across a number of language features, but I'm not sure
if a specific symbol table support like you mention will be included. The
problem with such a feature is it increases the complexity of the grammar
language specification, increases the size of the runtime and/or the
complexity of the code generation templates (making things more difficult
for target developers), and only meets the semantic language features of a
small number of users.

One idea I had is for a target-agnostic "call" syntax that could be used in
semantic predicates and actions. It could use a form such as the following:

@{FunctionName(arg1,arg2,arg3)}

Where an argument can be one of the following:

.         A reference to a token, rule return value, or label in the rule.

.         A reference to an argument passed to the rule.

.         A reference to a value in an attribute scope.

The target would then declare a user-definable method with the appropriate
parameter types. The Java and CSharp2 targets could create an abstract
method, the CSharp3 target could create a partial method, and a C/C++
targets could declare the method in a generated header.

Sam

From: Douglas Godfrey [mailto:douglasgodfrey at gmail.com] 
Sent: Tuesday, June 07, 2011 8:09 AM
To: Ranco Marcus
Cc: Sam Harwell; antlr-interest at antlr.org
Subject: Re: [antlr-interest] [CSharp3] rule visibility in composite
grammars

2 months ago I submitted a feature request for an Antlr built-in symbol
table to support the common 
requirements of the majority of block-structured languages. By making the
SymbolTable part of the
Antlr grammar language the interface can be much cleaner. The implementation
of the SymbolTable
classes would be part of the target runtime(s).

i.e. 

new_variable_name:
    (Identifier.IsNewSymbol()) => Identifier.AddSymbol();

SymbolTable -> NameSpace -> SymbolScope -> Symbol -> Attribute-List {
optional for structs -> NameSpace }  

On Tue, Jun 7, 2011 at 5:56 AM, Ranco Marcus <ranco.marcus at epirion.nl>
wrote:

Yes, that sounds like a good idea. I would definitely be in favour of
merging the grammars before generating the code. My only concern is that
this approach would deviate from the general ANTLR approach.

In general, I have found ANTLR to be a great tool for parser generation, but
never really liked the way target specific actions are mixed with the
grammar definition. Ideally, I would like my grammars to be _completely_
target agnostic (no actions, no visibility modifiers, members, headers,
superClass definitions, etc.) and have an abstract mechanism that we can use
to hookup actions and implementation specific stuff to the generated
grammar.

Do you know if there are plans to redesign the composite grammar feature in
v4?

Best regards,

Ranco

> -----Original Message-----
> From: Sam Harwell [mailto:sharwell at pixelminegames.com]
> Sent: Sunday, May 29, 2011 11:08 AM
> To: Ranco Marcus; antlr-interest at antlr.org
> Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
>
> I'm not going to be able to address this issue until the second week of
June.
>
> That said, it seems the best way to handle all these issues with delegate
> grammars is to inline their rules before code generation. Suppose you have
> grammar C importing A and B, and you also have D importing A and B. The
> code generation will result in classes C, C_A, C_B, D, D_A, and D_B.
Clearly
> the independent generation of C_A and D_A during code generation does
> not allow a single instance of the imported A grammar to be shared by C
and
> D. If we instead "flatten" the imported grammar hierarchy and only
generate
> classes C and D, then everything behaves like it was written in a single
> grammar. Do you see any immediate problems with this potential approach?
>
> -----Original Message-----
> From: Ranco Marcus [mailto:ranco.marcus at epirion.nl]
> Sent: Wednesday, May 25, 2011 4:26 PM
> To: Sam Harwell; antlr-interest at antlr.org
> Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
>
> Hi Sam/all,
>
> When a (tree) grammar C imports (tree) grammars A and B, where grammar
> A calls a rule R from grammar B, a call is being made from delegate parser
C_A
> to a delegate rule R (targeting C_B) in the composite parser C (its
parent).
>
> Now that the visibility of the delegate rules in C match the visibility of
the
> imported grammar, the rule R has to be made public for the above to work.
> In our grammars, we build up internal structures that are subsequently
> processed. In our case, that means that all those internal structures have
to
> be made public as well. This could be solved by allowing ANTLR rules to
have
> 'internal' visibility. Also, imported grammars can probably remain
internal as
> well.
>
> What are your thoughts on this?
>
> Thanks, Ranco
>
> > -----Original Message-----
> > From: Sam Harwell [mailto:sharwell at pixelminegames.com]
> > Sent: Thursday, April 07, 2011 3:57 PM
> > To: Ranco Marcus; antlr-interest at antlr.org
> > Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
> > grammars
> >
> > The visibility of delegated rules now matches the way they're declared
> > in the imported grammar. Since some of them return nested classes
> > declared in the delegate grammar, I had to make the delegate grammar
> > 'public' and all its rules 'internal'.
> >
> > I fixed the reserved names issue right after releasing 3.3.1, so it'll
> > be included when I release 3.3.2. Note that rule parameters and return
> > values need to include an explicit '@' in their declarations in the
> > grammar, as follows. Labels and rule names do not need the '@'.
> >
> > namespace[int @in] returns [int @out] : as=A B;
> >
> > Sam
> >
> > -----Original Message-----
> > From: Ranco Marcus [mailto:ranco.marcus at epirion.nl]
> > Sent: Thursday, April 07, 2011 6:49 AM
> > To: Sam Harwell; antlr-interest at antlr.org
> > Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
> > grammars
> >
> > Sounds great, Sam!
> >
> > What will be the visibility of the methods in the 'Delegated rules'
> > region now that the class of the imported grammar is internal? Private
> > or
> internal?
> >
> > Another small suggestion is to prefix variable names (at least those
> > that are reserved in C#) with an at-sign. The generated class for the
> > grammar below has a compile error since 'as' is not a valid variable
> > name
> while '@as' is.
> >
> > name : as=firstName lastName;
> >
> > Are you planning to release new binaries any time soon?
> >
> > Thanks again,
> >
> > Ranco Marcus
> >
> > > -----Original Message-----
> > > From: Sam Harwell [mailto:sharwell at pixelminegames.com]
> > > Sent: woensdag 6 april 2011 17:20
> > > To: Ranco Marcus; antlr-interest at antlr.org
> > > Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
> > > grammars
> > >
> > > Hi Ranco,
> > >
> > > I made the following changes so far:
> > >
> > > * An imported grammar is always generated as an internal class
> > > instead of a public class.
> > > * Rules within the imported grammar are always declared public since
> > > they are only called by the root grammar.
> > >
> > > How does that sound?
> > >
> > > Sam
> > >
> > > -----Original Message-----
> > > From: antlr-interest-bounces at antlr.org
> > > [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Ranco Marcus
> > > Sent: Wednesday, March 23, 2011 6:40 AM
> > > To: antlr-interest at antlr.org
> > > Subject: [antlr-interest] [CSharp3] rule visibility in composite
> > > grammars
> > >
> > > Hi all/Sam,
> > >
> > > While using the CSharp3 target on a composite grammar, I came across
> > > a few issues regarding rule visibility.
> > >
> > > If no modifier (private, protected, public, ...) is specified for a
> > > rule, the Java target generates public methods (since it is not
> > > implemented IIRC) and the
> > > CSharp3 target generates private methods.
> > >
> > > I like the idea to explicitly specify which rules are callable (by
> > > making them public). However, for composite grammars the generated
> > > parser contains delegate rules for every rule of the imported
> > > grammar, even for rules in the imported grammar that aren't public
> > > (causing compile errors). This can solved this adding a public
> > > modifier to every rule of the imported grammar, but I was wondering
> > > if there's a more
> > convenient way of doing this.
> > >
> > > Furthermore, the methods for the delegate rules in the composite
> > > grammars are all public, regardless of the visibility of the rule in
> > > the
> > imported grammar.
> > > IMHO, it would be better to make them private and use the composite
> > > grammar to explicitly specify the rules that are externally visible.
> > > What are your thoughts on this?
> > >
> > > Btw, the CSharp3 templates are also included in the original Java
> > > tool. Does that mean that we don't need the .NET version of the tool
> > anymore?
> > >
> > >
> > > Best regards,
> > >
> > > Ranco Marcus
> > > Epirion Knowledge Solutions B.V.
> > >
> > >
> > > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > > Unsubscribe:
> > > http://www.antlr.org/mailman/options/antlr-interest/your-email-addre
> > > ss
> >
>

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address