[antlr-interest] Request:External action definitions
Tiller, Michael (M.M.)
mtiller at ford.com
Tue Nov 26 12:34:31 PST 2002
I like this discussion of separating the grammar from the actions. I've often wished for this. One other thing worth mentioning. I find it quite tedious to work with new grammars, lexers and treewalkers because they generate so much stuff that needs to be compiled, etc.
Assume, for the sake of argument, that we complete decoupled the grammar from the actions. What would be very nice would be a sort of "virtual machine tree walker" system whereby the grammar could be broken down into a special language neutral "byte code" (conceptually a binary representation of the grammar, look-ahead, alternatives, etc) and then you could just create tree walkers in whatever language you wanted.
The "ideal situation" I envision would be have a compiled executable of ANTLR that could take a grammar and generate a machine representation of the recursive descent tree (i.e. the "byte code"). As much of the analysis as possible would be done in this phase. My hope is that I could then implement a relatively lightweight Python parser that would allow me to parse data files and walk the resulting tree while not having to compile anything or integrate this Python "back end" with any of the grammar analysis capabilities.
This would avoid the need to compile anything.
At first glance, it might appear that this would break all the existing code written in ANTLR. I have some comments on this, but I'll save the time of typing them because I'm guessing most people think this is completely out to lunch. :-)
The main point here, for me, is to promote language independence, separate the grammar from the actions and avoid the long tool chain (Java to run ANTLR, C++ compiler to compile the resulting parser, SWIG or BOOST to map the C++ code to Python and Python to try out tree walking ideas).
--
Mike
> -----Original Message-----
> From: Terence Parr [mailto:parrt at jguru.com]
> Sent: Tuesday, November 26, 2002 3:02 PM
> To: antlr-interest at yahoogroups.com
> Subject: Re: [antlr-interest] Request:External action definitions
>
>
> Hi. yes, that method works but has two problems:
>
> 1 you need to modify the grammar to have actions that
> reference methods
> all over or you won't be able to handle new situations
>
> 2 you can't define local variables in the rules, which are REALLY
> useful as you get a new copy per rule invocation.
>
> Good summary, ram.
>
> Terence
>
> On Tuesday, November 26, 2002, at 10:25 AM, Sriram Durbha wrote:
>
> > This looks alot like virtual functiuns
> > so we write an abstract grammar first, with virtualfunctions for
> > actions,
> > and then override them in the derived grammars ;
> >
> > since i have not used the inheritance feature even now i am at aloss
> > here, but the general idea looks like this :
> >
> >
> > if i write a grammar now which might be used with different
> actions at
> > a later stage, i want to put place holders for those
> actions and fill
> > them later;
> > so even now if in the action part we simply invoke a
> virtual function
> > from a calss called actions, declared in the first part of
> the grammar,
> > and defined in a source file which is also included in the project
> > so all that the actions part will have is
> >
> > rule : ...
> > {
> > action_object_ptr->method_for_this_rule() ;
> > }
> > ;
> >
> >
> > the action_object_ptr has to be assigned a new object of
> the required
> > sub-type ; and its visibility , local ness etc have to be
> taken care of
> > ..
> >
> > so now i can change the action independently of the grammar ;
> > also i can override only some of the actions if i have to ;
> >
> > but when grammar has to change, new situations might arise :)
> >
> > cheers
> > ram
> >
> >
> >
> >
> > --- Terence Parr <parrt at jguru.com> wrote:
> >> Hi. Yes, great idea! I would call this "aspect oriented
> >> programming"
> >> and is one future enhancement we've considered. I have
> been working
> >> with Monty and Loring on my new ideas for "how to keep
> those actions
> >> from screwing up the grammar and making the grammar specific to one
> >> problem." The basic idea, assuming we had good tools for
> this, would
> >>
> >> be to let you "derive" a new grammar from an old grammar and then
> >> have
> >> changes to the original grammar be "pushed" into your modified
> >> grammar.
> >> Kind of like a "live" cut-n-paste. In this way, you
> keep one clean
> >>
> >> grammar that is just the grammar; no actions. Then to make that
> >> grammar do something "derive" a new one and add your actions.
> >> Changes
> >> can be rolled forward to any derived grammar. Currently we use
> >> inheritance which is really just compile-time cut-n-paste for
> >> grammars;
> >> but this is less flexible as you have to rewrite every rule just to
> >> get
> >> actions in there. This strategy is what you do now,
> right? You grab
> >> a
> >> grammar and modify it and then curse when the original grammar gets
> >> updated (such as what happens for any of you with a modified Java
> >> grammar). ;)
> >>
> >> You might call this the
> >> RCS-based-grammar-derivation-and-composition-method.
> <snicker> Howz
> >>
> >> that for a buzzword!?
> >>
> >> Sound interesting?
> >>
> >> Terence
> >> PS Just finishing up my semester here at USF and should
> get some good
> >>
> >> ANTLR time in now for next month or so to push out 2.7.2 for real!
> >>
> >> On Tuesday, November 26, 2002, at 08:10 AM,
> mzukowski at yci.com wrote:
> >>
> >>> Someone has recently done just this, but I forget who. I'm sure
> >> they
> >>> will
> >>> pipe up. Look at the files section on the yahoo list
> site, I think
> >> it
> >>> is
> >>> there.
> >>>
> >>> Also noweb can be used for this type of thing too. Searching for
> >>> noweb in
> >>> the list archives may uncover the one I was thinking of.
> >>>
> >>> Monty
> >>>
> >>> -----Original Message-----
> >>> From: Anakreon Mejdi [mailto:amejdi at ertonline.gr]
> >>> Sent: Tuesday, November 26, 2002 3:41 AM
> >>> To: antlr-interest at yahoogroups.com
> >>> Subject: [antlr-interest] Request:External action definitions
> >>>
> >>>
> >>> I have an idea which might make antlr better.
> >>> Instead of defining the actions in the same file with the grammar
> >>> they could be defined in other file(s).
> >>>
> >>> A way this could be achived(not necessarily the best) would be:
> >>>
> >>> RULE:
> >>> 'a' .. 'z' { %id; }
> >>> ;
> >>>
> >>>
> >>> rule
> >>> :
> >>> RULE { %some_id; }
> >>> ;
> >>>
> >>>
> >>> tree_rule
> >>> :
> >>> #(r:RULE { %tree_id; System.out.println("RULE
> matched"); }
> >>> ;
> >>>
> >>>
> >>> In this case antlr.Tool will be fed with the grammar file and the
> >>> file(s) where actions are defined.
> >>>
> >>> The actions identified with a label (like %id) will be searched
> >> among
> >>> the action files and the label should be replaced by the content
> >> of
> >>> the action.
> >>>
> >>> Example:
> >>> id = {
> >>> System.out.println("Externaly defined action");
> >>> }
> >>>
> >>> tree_id = {
> >>> System.out.println(r.getText());
> >>> }
> >>>
> >>> This could be achieved with regular expressions but would be a
> >>> primitive
> >>> implementation.
> >>>
> >>> I'd like to know what you think about it.
> >>> Anakreon
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> Your use of Yahoo! Groups is subject to
> >>> http://docs.yahoo.com/info/terms/
> >>>
> >>>
> >>>
> >>>
> >>> Your use of Yahoo! Groups is subject to
> >>> http://docs.yahoo.com/info/terms/
> >>>
> >>>
> >> --
> >> Co-founder, http://www.jguru.com
> >> Creator, ANTLR Parser Generator: http://www.antlr.org
> >> Lecturer in Comp. Sci., University of San Francisco
> >>
> >>
> >>
> >>
> >> Your use of Yahoo! Groups is subject to
> >> http://docs.yahoo.com/info/terms/
> >>
> >>
> >
> >
> > __________________________________________________
> > Do you Yahoo!?
> > Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
> > http://mailplus.yahoo.com
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
> > http://docs.yahoo.com/info/terms/
> >
> >
> --
> Co-founder, http://www.jguru.com
> Creator, ANTLR Parser Generator: http://www.antlr.org
> Lecturer in Comp. Sci., University of San Francisco
>
>
>
>
> Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list