[stringtemplate-interest] Group Syntax extension for ModelAdapter and Renderer

Udo Borkowski ub at abego-software.de
Mon Jun 27 14:27:27 PDT 2011 

> The problem with that idea is that ST was always meant (afaict) to be an implementation-language-agnostic syntax. I don't think it's worth sacrificing that in order to do what you seem to be suggesting.

I think the "implementation-language-agnostic" syntax of StringTemplate may have been a goal (I was not aware of this, btw). 

However in its current implementation StringTemplate (ST4) is implementation language dependent. The feature that directly comes into my mind is ("surprise") the StringRenderer, as delivered in the Java reference implementation. If the format specified for a String attribute is not "upper", "lower", "cap", "url-encode", or "xml-encode" the current implementation will pass the format string to the Java String.format method. This is a very powerful method with a lot of nice features. However other implementations of ST4 do not implement exactly this functionality. E.g. the C-Sharp implementation calls

	string.Format(formatString, s);

Even though both methods sound similar they use a completely different syntax (%... vs. {…}). So when you write a template (and used the format option in its free form) you must know what implementation language is used for your engine, even when you restrict yourself to the build-in StringRenderer.

So ST4 is has no implementation language agnostic syntax, at least if you include the syntax of the format string into the ST4 syntax.


If you don't want to include the format syntax into the ST4 syntax you also should not do so for the suggested "renderer" feature. Then the renderer feature has the following implementation agnostic syntax:

renderer : 'renderer' STRING STRING

The meaning of the strings depends on the engine you are using and for Java this may look like

	renderer "org.w3c.dom.Node" "com.collinfagan.strum.adapters.xml.NodeRenderer"

and for C-Sharp 

	renderer "some text to identify a class in CS" "some text to identify a class in CS"



Udo


On 27.06.2011, at 18:42, Sam Barnett-Cormack wrote:

> On 27/06/2011 16:58, Udo Borkowski wrote:
>> 
>>> Maybe the sensible thing to do, if you really run into things like this,
>> I am really running into things like this :)
>> 
>>> is define a dynamic renderer for strings, that you can then register format-specific subrenderers with.
>> This is what I am currently doing. However this does not seem to be the right way. Now I am adding more and more "subrenderers" to the system and I am either getting this gigantic "Super StringRenderer" I need to use in all my projects, or I need to individually add the subrenderers for each project and need to maintain these individual lists as the templates evolve over time and need more/other "subrenderers".
>> 
>> Putting the renderers next to the place where their are used (the groups) seem to lead to a more maintainable structure.
> 
> The problem with that idea is that ST was always meant (afaict) to be an implementation-language-agnostic syntax. I don't think it's worth sacrificing that in order to do what you seem to be suggesting.
> 
> Regarding the problem of escapes that are clearly the common point in your two examples, maybe there is another way to handle this... it might be seen as beyond the scope of ST itself, but it's a well-defined enough task that it might be worth it being support in a default StringRenderer, with flags to indicate what escapes are needed, including a flexible enough one to work in cases like the javadoc-comment one. The other option there is to make it the responsibility of the model to provide data that's already 'suitable' to be used as a javadoc-comment.
> 
> Sam
> 
>> On 27.06.2011, at 16:52, Sam Barnett-Cormack wrote:
>> 
>>> On 27/06/2011 14:54, Udo Borkowski wrote:
>>>> CASE B
>>>> ======
>>>> 
>>>> But now assume we have a slightly different main template:
>>>> 
>>>> --- template main.stg ---------
>>>> main(s,cmt) ::=<<
>>>> <*javadoc*(cmt)>
>>>> String f() {
>>>> return<s;*format="quote"*>;
>>>> }
>>>>>> 
>>>> -----------------------------------------
>>>> 
>>>> Here template main needs an attribute renderer for the "quote" format
>>>> (QuotedStringRenderer). Therefore we add this:
>>>> 
>>>> group.registerRenderer(String.class, new QuotedStringRenderer());
>>>> 
>>>> Now the "quote" renders fine, however the JavaDoc is not ok.
>>>> 
>>>> Note we cannot just add
>>>> 
>>>> group.registerRenderer(String.class, new JavaDocRenderer());
>>>> 
>>>> to the main group as we have already registered a renderer for String.class.
>>>> 
>>>> Also the "inheritance" mechanism will not help here, as the lookup for a
>>>> renderer for String.class will find the QuotedStringRenderer in group
>>>> main and also use this one for the "javadoc" case.
>>> 
>>> This sounds like a situation that the renderer system isn't design to cater to; rather, it sounds like you should be using a template itself for the javadoccing case, for instance, or having a single renderer that does both sensitive to the template; perhaps a meta-renderer that checks to see whether formats are recognised. In any case, it's not a case that ST seems to try to cater to at the moment, renderers instead being based principally on the type being passed, rather than the format. I'm not sure how generally usefula a fix for it would be, as your examples seem to be generally better served by templates than by renderers (okay, the quote one presumeably does escaping and suchlike, making it possibly appropriate to the renderer). Maybe the sensible thing to do, if you really run into things like this, is define a dynamic renderer for strings, that you can then register format-specific subrenderers with.
>>> 
>>> Sam
>>> 
>>>> PROPOSAL
>>>> =========
>>>> 
>>>> Instead of using inheritance we could make the attributeRender lookup a
>>>> simple two step approach:
>>>> 
>>>> Before using the renderer as provided by the current implementation
>>>> check if the group containing the template using the "format" option
>>>> defines an appropriate renderer. If yes, use that, otherwise use the
>>>> "old" one.
>>>> 
>>>> protectedintwritePOJO(STWriter out, ST self, Object o, String[] options)
>>>> throwsIOException {
>>>> String formatString = null;
>>>> if( options!=null) formatString = options[Option.FORMAT.ordinal()];
>>>> 
>>>> 
>>>> AttributeRenderer r =
>>>> *self.**impl**.**nativeGroup**.getAttributeRenderer(o.getClass());*
>>>> *****if**(r == **null**) {*
>>>> r = group.getAttributeRenderer(o.getClass());
>>>> }
>>>> ...
>>>> 
>>>> This will work fine in both case A and case B.
>>>> 
>>>> This is a simple solution, simpler than any "inheritance" or
>>>> "propagation" approach. Actually I currently cannot think of a use case
>>>> I would need anything more sophisticated.
>>>> 
>>>> Note if someone prefers to define all renderers in the root group (and
>>>> not in imported groups) he is still free to do so. The proposed solution
>>>> will then automatically fall back to the current behavior (as no
>>>> attributeRenderers are found in the imported groups).
>>>> 
>>>> 
>>>> BTW: this suggested extension to the attributeRenderer lookup is not
>>>> directly related to the "syntax extension" I proposed. Actually the
>>>> "programmatic" approach to add renderers and do imports will benefit
>>>> from the new lookup rules as well.
>>>> 
>>>> 
>>>> 
>>>> Udo
>>>> 
>>>> 
>>>> 
>>>> On 25.06.2011, at 20:08, Terence Parr wrote:
>>>> 
>>>>> hiya. Inheritance will work on this. it's the only way polymorphism
>>>>> works with template instantiation; same for adaptors/renderers. If A
>>>>> imports B we created templates relative to A even if defined in B.
>>>>> Therefore, a renderer for A works for B. A's renderer overrides any
>>>>> you set in B as well. No need to think about propagation or whatever.
>>>>> 
>>>>> I'll go look at inheritance mech. i see on my list "should adaptors
>>>>> get imported from super group?"
>>>>> 
>>>>> http://www.antlr.org/wiki/display/ST4/ST+v4+TODO+list
>>>>> 
>>>>> Ter
>>>>> On Jun 24, 2011, at 7:20 AM, Sam Barnett-Cormack wrote:
>>>>> 
>>>>>> On 24/06/2011 14:59, Sam Harwell wrote:
>>>>>>> The VM-wide setting approach seems like a possibility, but only if
>>>>>>> the "VM"
>>>>>>> is an instance object. There are a number of cases where I run several
>>>>>>> independent ST "sessions" within a single process, so I'm trying hard to
>>>>>>> remove every last mutable static variable from the entire [C# port
>>>>>>> of the]
>>>>>>> library. Perhaps an Interpreter object could be stored in an STGroup
>>>>>>> at the
>>>>>>> time when the group is constructed, and that interpreter used for all
>>>>>>> rendering of templates accessed through that group's getInstanceOf()
>>>>>>> method.
>>>>>>> That way, we have a "VM" which is a single interpreter used for that
>>>>>>> group's
>>>>>>> operations. I'd even go so far as to declare the Interpreter field of
>>>>>>> STGroup final, giving a fixed view of the VM.
>>>>>> 
>>>>>> I used the term VM based on my understanding of the Java version. I
>>>>>> would agree that it would be best not to have such things truly static -
>>>>>> I would support a (for example) STInterpreter class, an instance of
>>>>>> which can be used when creating a group, or it be specified with a
>>>>>> boolean parameter that a new one should be created (otherwise a truly
>>>>>> VM-wide default shared one would be used). I imagine implementation
>>>>>> would be awkward if you could import groups that use a different
>>>>>> STInterpreter from the one doing the important (depending on what
>>>>>> actually gets stored in the instance), but otherwise it's a good way to
>>>>>> share details of such things between multiple groups under control.
>>>>>> Perhaps allow STInterpreters to be defined as 'children' of another
>>>>>> STInterpreter, with any properties not specifically set in that child
>>>>>> being proxied to the parent (implementation details could vary, but I
>>>>>> can see that being easiest with a specialised subclass).
>>>>>> 
>>>>>> And yes, it would make sense for the interpreter of any given STGroup to
>>>>>> be final, to whatever extent is possible in any given implementation
>>>>>> language. You don't want to be changing back and forth all of those
>>>>>> things after a group is created, though you may want to manipulate the
>>>>>> interpreter itself.
>>>>>> 
>>>>>> This would also fit with the usual patterns used for such things in Java
>>>>>> these days, particularly EE, allowing it to fit better with Dependency
>>>>>> Injection and Contexts. It's definitely more common in my experience to
>>>>>> have some sort of context class to hold such things than it is to have
>>>>>> anything global/static.
>>>>>> 
>>>>>> Sam
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Sam Barnett-Cormack [mailto:s.barnett-cormack at lancaster.ac.uk]
>>>>>>> Sent: Friday, June 24, 2011 5:17 AM
>>>>>>> To: Udo Borkowski
>>>>>>> Cc: Sam Harwell; 'stringtemplate-interest Template'; 'Terence Parr'
>>>>>>> Subject: Re: [stringtemplate-interest] Group Syntax extension for
>>>>>>> ModelAdapter and Renderer
>>>>>>> 
>>>>>>> Sorry to top-post, but this is a very general statement/opinion...
>>>>>>> 
>>>>>>> Renderers and ModelAdapters need to be entirely in the programming side,
>>>>>>> surely, to keep the complete language-agnostic value of template code.
>>>>>>> Unlike Antlr grammars, for instance, there's nothing in a StringTemplate
>>>>>>> that is ever language-specific (in terms of implementation), that
>>>>>>> I'm aware
>>>>>>> of, and I devoutly hope it will remain so.
>>>>>>> 
>>>>>>> Furthermore, whether it happens at the programming level or in the group
>>>>>>> file (and what about people not using group files, but group dirs or a
>>>>>>> custom group type), whatever is chosen in terms of when to propagate
>>>>>>> and in
>>>>>>> which direction, there will always be situations where what happens
>>>>>>> isn't
>>>>>>> what makes sense for that use-case. Thus, what would really be ideal
>>>>>>> is a
>>>>>>> way of controlling it more finely, as to when it propagates or
>>>>>>> doesn't, some
>>>>>>> sort of configuration. This would, however, likely be a huge burden
>>>>>>> on the
>>>>>>> ST developers, and a pretty big burden on those using it.
>>>>>>> 
>>>>>>> One alternative that occurs to me as probably working in most cases,
>>>>>>> is to
>>>>>>> allow a VM-wide setting of model adapters and renderers, and then allow
>>>>>>> group-specific overriding of this. Whether the group-specific ones then
>>>>>>> propagate or not, and in which direction, I'm not sure. You
>>>>>>> obviously can't
>>>>>>> have them propagating in both directions without every change always
>>>>>>> affecting the whole constellation of groups involved, which would seem
>>>>>>> suboptimal to me.
>>>>>>> 
>>>>>>> Anyway, that's just some software engineering thoughts. To get a better
>>>>>>> decision, it would really make sense to drill down to some competing
>>>>>>> use-cases and see if there's a common denominator that would work in all
>>>>>>> cases that anyone can think of.
>>>>>>> 
>>>>>>> Sam
>>>>>>> 
>>>>>>> On 24/06/2011 10:41, Udo Borkowski wrote:
>>>>>>>> The "propagation" approach solves one half of the problem: with this
>>>>>>>> approach one could use renderers and adapters in groups that are
>>>>>>>> imported through the "import" statement (and not programmatically).
>>>>>>>> This is currently not possible.
>>>>>>>> 
>>>>>>>> But the other half of the problem still exists: assume I change a
>>>>>>>> group G and use some special renderer in the new version. The group G
>>>>>>>> is imported by many other groups (maybe indirectly). I now need to
>>>>>>>> find all the "roots" of imports to G and add the new renderer there.
>>>>>>>> This can really become a maintenance nightmare. Providing the
>>>>>>>> "renderer" syntax extension and defining the renderer in the group
>>>>>>>> text would solve this issue.
>>>>>>>> 
>>>>>>>> Regarding the implementation of the "propagation" approach:
>>>>>>>> registering a renderer will require to visit all directly and
>>>>>>>> indirectly imported groups and add the renderer to all of them,
>>>>>>>> possibly creating new maps etc.. Most of the times this will be extra
>>>>>>>> work as the imported groups don't reference that renderer. Propagating
>>>>>>>> a renderer to all imported groups may lead to a performance issue when
>>>>>>> using large systems.
>>>>>>>> 
>>>>>>>> I would also hesitate to use the "propagation" approach as it adds a
>>>>>>>> new concept to StringTemplate ("propagation of renderers") that in
>>>>>>>> addition seems to "reverse" an already existing concept
>>>>>>>> ("inheritance"). I guess this may to misunderstandings.
>>>>>>>> 
>>>>>>>> In contrast to this the "renderer" syntax extension does not use a new
>>>>>>>> concept but rather makes an existing feature ("registerRenderer")
>>>>>>>> accessible to those users who prefer to work on the group text than
>>>>>>>> coding in Java.
>>>>>>>> 
>>>>>>>> As the "propagation" approach does not solve the whole problem I still
>>>>>>>> think we should add the "renderer"/"adapter" feature I suggested.
>>>>>>>> 
>>>>>>>> Udo
>>>>>>>> 
>>>>>>>> P.S.: all stuff mentioned regarding "renderer" also applied "adapter".
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 23.06.2011, at 23:38, Sam Harwell wrote:
>>>>>>>> 
>>>>>>>>> This is an interesting problem. Normally when things are inherited,
>>>>>>>>> they start at the imported group and propagate to the group that
>>>>>>>>> imported them. In this case, it sounds like you want to propagate it
>>>>>>>>> from the topmost group to all the groups it imports.
>>>>>>>>> Perhaps the thing to do here is propagate renderers to groups
>>>>>>>>> imported with STGroup.importTemplates(Token) (those are the ones
>>>>>>>>> imported via the group file), but don't propagate them to groups
>>>>>>>>> imported with a direct call to importTemplates(STGroup)?
>>>>>>>>> If you look at the diff of TemplateGroup.cs in CL8734 (it's a small
>>>>>>>>> diff), you can see how I adjusted the import code to handle the
>>>>>>>>> unload() method in everyone's cases. If the java code is adjusted in
>>>>>>>>> the same way (I can make the change if you want), then it's easy to
>>>>>>>>> propagate renderers by the following two rules:
>>>>>>>>> 1.When STGroup.importTemplates(Token) is called, the current group's
>>>>>>>>> renderers are added to the group that just got imported. This handles
>>>>>>>>> the case where the renderer is registered before the group is
>>>>>>>>> imported (especially relevant in reloading a group file after calling
>>>>>>> unload()).
>>>>>>>>> 2.When a renderer is added to a group, it is automatically added (by
>>>>>>>>> calling registerRenderer) to all groups in the importsToClearOnUnload
>>>>>>>>> list. This handles the case where the group is imported before the
>>>>>>>>> renderer is registered.
>>>>>>>>> Sam
>>>>>>>>> *From:*stringtemplate-interest-bounces at antlr.org
>>>>>>>>> <mailto:stringtemplate-interest-bounces at antlr.org>[mailto:stringtempl
>>>>>>>>> ate-interest-bounces at antlr.org]*On
>>>>>>>>> Behalf Of*Terence Parr
>>>>>>>>> *Sent:*Thursday, June 23, 2011 1:44 PM *To:*Udo Borkowski
>>>>>>>>> *Cc:*stringtemplate-interest Template
>>>>>>>>> *Subject:*Re: [stringtemplate-interest] Group Syntax extension for
>>>>>>>>> ModelAdapter and Renderer I think that we should probably keep this
>>>>>>>>> at the programming level.
>>>>>>>>> should we make renderers inherited instead to solve your problem?
>>>>>>>>> Ter
>>>>>>>>> On Jun 21, 2011, at 1:17 AM, Udo Borkowski wrote:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> currently we programmatically register ModelAdapters and Renderers to
>>>>>>>>> an STGroup.
>>>>>>>>> What about providing an extension to the syntax of Group to also
>>>>>>>>> define these in a Group file?
>>>>>>>>> E.g. I could imagine to use something like
>>>>>>>>> 
>>>>>>>>> adapter "org.w3c.dom.Node"
>>>>>>>>> "com.collinfagan.strum.adapters.xml.NodeModelAdapter"
>>>>>>>>> renderer "org.w3c.dom.Node"
>>>>>>>>> "com.collinfagan.strum.adapters.xml.NodeRenderer"
>>>>>>>>> 
>>>>>>>>> at the top of an Group file. This would mean the same as running this
>>>>>>>>> Java code for the group:
>>>>>>>>> 
>>>>>>>>> group.registerRenderer(org.w3c.dom.Node.class, new
>>>>>>>>> com.collinfagan.strum.adapters.xml.NodeRenderer());
>>>>>>>>> 
>>>>>>>>> group.registerModelAdaptor(org.w3c.dom.Node.class, new
>>>>>>>>> com.collinfagan.strum.adapters.xml.NodeModelAdapter());
>>>>>>>>> 
>>>>>>>>> Especially when importing groups this feature comes in handy as I
>>>>>>>>> cannot register adapters/renderers when importing. In these cases I
>>>>>>>>> must rely on the root group. For this group R we must register ALL
>>>>>>>>> adapters/renders used in ANY group R imports. This make things hard
>>>>>>>>> to maintain as using a "new" renderer in some template T requires me
>>>>>>>>> to add the "registerRenderer" in EVERY code using T, maybe indirectly
>>>>>>>>> through imports.
>>>>>>>>> Similar to features discussed earlier this feature is easy to
>>>>>>>>> implement for the STGroupFile, but the STGroupDir currently has no
>>>>>>>>> proper place to hold this information. So we may also need to tackle
>>>>>>>>> this re-appearing topic, too.
>>>>>>>>> What do others think?
>>>>>>>>> Udo
>>>>>>>>> _______________________________________________
>>>>>>>>> stringtemplate-interest mailing list
>>>>>>>>> stringtemplate-interest at antlr.org
>>>>>>>>> <mailto:stringtemplate-interest at antlr.org>
>>>>>>>>> http://www.antlr.org/mailman/listinfo/stringtemplate-interest
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> stringtemplate-interest mailing list
>>>>>>>> stringtemplate-interest at antlr.org
>>>>>>>> http://www.antlr.org/mailman/listinfo/stringtemplate-interest
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> stringtemplate-interest mailing list
>>>>>> stringtemplate-interest at antlr.org
>>>>>> http://www.antlr.org/mailman/listinfo/stringtemplate-interest
>>>>> 
>>>>> _______________________________________________
>>>>> stringtemplate-interest mailing list
>>>>> stringtemplate-interest at antlr.org
>>>>> http://www.antlr.org/mailman/listinfo/stringtemplate-interest
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> stringtemplate-interest mailing list
>>>> stringtemplate-interest at antlr.org
>>>> http://www.antlr.org/mailman/listinfo/stringtemplate-interest
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/stringtemplate-interest/attachments/20110627/c6203e8e/attachment-0001.html

More information about the stringtemplate-interest mailing list