[antlr-interest] Aspects and ANTLR (was: Re: nice threads)

Bogdan Mitu bogdan_mt at yahoo.com
Mon Jun 17 08:21:11 PDT 2002


I'm happy to see Monty's thread, because I am working on a preprocessing
tool that goes beyond the current grammar inheritance mechanism, using an
Aspect-like style of programming. For the moment, I am able to modify the
header and the main "action" (the additional Constructors, declarations and
methods) of a grammar. I'm still looking for a convenient syntax. I am
thinking on something like:


class MyXxxParser extends BasicXxxParser;

header { // new header

options { buildAST = true; }  // overwrite default value = false

{  // start of top-level inserted Java code

   ArrayList methodList;

   public MyXxxParser( Object arg) { ... }


If I want to preserve the original header, and just add something to it (a
new import, for instance, because I want to add some new auxiliary methods),
I can write something like:

header {
   super.header();           // preserve the default header
   import com.extra.*;       // add a new import

Alternatively, in an Aspect style:

XxxParser.header: last() { import com.extra.*;      // add a new import }

Of course, it will allow rule inheritance, like the current supergrammar
stuff, but it will also let you modify the init and final actions of the

Using Aspects, you can modify a group of related files. Let's say you want
to parse single line comments, looking for TODOs, FIXMEs or stuff like this.

aspect InterpretComments {

   // Make the lexer switch to an embedded comment lexer, 
   // instead of returning a monolithic token
      :  "//" { switchToCommentLexer(); }

   // Add a rule in the parser to interpret the comments ...
      :   COMMENT_START! ("TODO"^ | "FIXME"^ | ...) ...

   // .. and new rules in the treeParser
      :   #( "TODO" ... )
      :   #( "FIXME" ... )

Using Aspects can also bring the benefits of literate programming. For
instance, you can write the parser and treeParser rules interleaved in the
same Aspect, so that when you modify a parser rule, you have the
corresponding treeParser rule just bellow, being less tempted to say "I'll
update it later".

I would like to go down to token (AST) level, and I'm thinking on a
XPath-like syntax:


(to reffer the third ID of the parser rule named "expression"). Although I
would prefer '.' instead of '/' (more readable, to me).

I need to think more on this; what do you think? Any feedback from you will
help me a lot.

Best regards,

--- Terence Parr <parrt at jguru.com> wrote:
> Folks, Monty's email is not getting thru to the list...so here is a side 
> thread to be weaved back in ;)
> Ter
> On Friday, June 14, 2002, at 07:15  AM, mzukowski at yci.com wrote:
> > Tree grammars for analysis.  Why wouldn't that work?  Do you really 
> > need a
> > cyclic graph/network structure?
> Well, yes/no.  Turns out it is a tree now loring says ;)  You really 
> need to convert a grammar to an NFA and then run a bounded k level 
> NFA->DFA conversion to do the analysis (that is the algorithm I designed 
> during my thesis).
> >
> > Is grammar subclassing still appropriate?
> I think it should be a function of the environment.  I.e, have a library 
> of rules/grammars in your repository and then pick and choose stuff to 
> grab to begin a new grammar.  A live push-forward-changes sort of thing 
> is the same as inheritance ;) (I think they call that RCS) ;) ;)
> >
> > When I think about grammars and aspects I think actions--AspectANTLR 
> > would
> > be the tool to weave your actions into your pure antlr grammars.  The 
> > trick
> Oooooooohhhhhh.  Now THAT is the perfect explanation of what an Aspect 
> is.  You don't want to modify a grammar physically just to add actions 
> into the right spot!.  Oohhhh.
> > is how to specify the join points and the answer is???  Note that your 
> > IDE
> > could still let you edit the actions as if they were attached to the .g
> > file, but in reality they don't have to be.  Nice for multipass tree
> > transforms and mixing C++ and Java output languages.
> Oh man...i think we're on to something.  AspectANTLR :)
> > I like the separation of antlr phases concept.  When I was digging into
> > lookahead to generate a lookahead dependency graph I was frustrated by 
> > the
> > caching of lookahead info.  I don't remember exactly now but it seems 
> > like
> > maybe I was trying to find the specific rules contributing to the 
> > ambiguity
> > but sometimes when I got to the ambiguous rule the lookahead was already
> > computed without the backlinks to what rules contributed to the 
> > lookahead
> > set.  Because you don't just want to know that a rule is ambiguous.  
> > 50% of
> > the time it's because another rule called it and I was automating that
> > manual process of figuring out which caller rules were to blame.
> Yes, the cached lookahead information is useless for determining the 
> paths in the grammar that are nondeterministic.  I would really like to 
> have ANTLREclipse or whatever be able to highlight the paths for you.
> The caching is absolutely necessary for efficiency of this algorithm.  
> Without it I cannot claim it is O(nk) where n is size of grammar.  But, 
> a new version of antlr could avoid the caching when finding the problem 
> spots.
> Ter
> --
> Co-founder, http://www.jguru.com
> Creator, ANTLR Parser Generator: http://www.antlr.org
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 

Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 

More information about the antlr-interest mailing list