[stringtemplate-interest] what is the use for embeddedInstances ?

[Terence Parr]

Hi Jax,

ok, I have simply removed the list of embedded instances. I believe I  
used to have that in there so that during debugging we could walk from  
the top of the list downwards. As it stands, you cannot ask a template  
to list all of the embedded templates. Well, I wonder if we could get  
that from the list of chunks and attributes. perhaps.

I read your test, after making it compile on my machine, and then ran

time java MemoryLeakTestCase > /dev/null

it took 6m13s and then got an out of memory exception. I am rerunning  
it now to see if there is a speed improvement as you suggest. Surely  
there must be a memory improvement ;)   Actually, I guess it will keep  
going and take a very long time because it does not get an exception.

How noticeable was your performance improvement?

Ok, test also got a memory exception at 6m16s. Does your test throw  
anything away? in other words, should it run until completion?

All tests seem to pass, so I'm going to  submit this to the repository  
and make a new daily build.

http://www.antlr.org/download/build/stringtemplate-06-03-2008.12.tar.gz

Ter

On Jun 3, 2008, at 11:37 AM, Jax Law wrote:

> Hi Terence,
>
> Thanks for the prompt reply ! I've attached a simple test case where  
> the memory leak can be reproduced.
>
> Here is a summary of the issue:
>
> 1. A StringTemplateGroup is held from a singleton
> 2. One of the template ( the order_by, say ST1 ) has an anonymous  
> template ( say ST2 )
> 3. The cached copy of the "order_by" template in the  
> StringTemplateGroup has a chunk ( ASTExpr )
>   that has a parse tree that leads to ST2
> 4. A copy of the ST1 obtained from the group.getInstanceOf() method  
> return a new ST but the chunk
>   list is copied as-is.
> 5. Calling toString() on the copy leads to a series of unwrapping of  
> references of the AST and ultimately
>   ST2.write() is called, which then addEmbeddedInstance is called.
> 6. Since ST2 is reachable from the singleton via the templates  
> HashMap, embeddedInstances list is never
>   garbage collected.
>
> We have workaround the issue by simply making addEmbeddedInstnace()  
> a no-op. It seems to have
> also improved performance greatly. Another suggestion to avoid this  
> issue is to make toString() and
> write() method non-state-changing, or making dup() returns a true  
> deep copy.
>
> Thanks a lot !
>
> Jax
>
>
> Terence Parr wrote:
>> thanks! just added bug:
>>
>> http://www.antlr.org:8888/browse/ST-39
>>
>> Ter
>> On Jun 2, 2008, at 2:48 PM, Jax Law wrote:
>>
>>> Hi,
>>>
>>> In StringTemplate.java, what is the purpose of the LinkedList
>>> embeddedInstances ?
>>> I've searched the source tree and there is no usage of this  
>>> LinkedList.
>>> We've found
>>> an obscure memory leak issue with embeddedInstances where the list  
>>> has grown
>>> to 10 million entries. The issue is that when
>>> StringTemplate.getInstanceOf() method
>>> is called, the copy it returns is not a true deep copy. It has many
>>> references to the
>>> original StringTemplate via the chunks Expr,  
>>> StringTemplateAST ...etc.
>>> One of
>>> these references is accessible to a write() method call stack and  
>>> thus
>>> build up the
>>> embeddedInstances LinkedList.
>>>
>>> Since no usage can be found, I am assuming we can make  
>>> addEmbeddedInstance
>>> a No-Op in our own build ?
>>>
>>> Thanks !
>>>
>>> Jax
>>>
>>> _______________________________________________
>>> stringtemplate-interest mailing list
>>> stringtemplate-interest at antlr.org
>>> http://www.antlr.org:8080/mailman/listinfo/stringtemplate-interest
>>
>>
>
> <st39.jar>