Query plan cache memory leak after cleanup

Description

After a QueryPlanCache.cleanup() the cache still retains a lot of memory (~500MB), but it should be empty.
I empty the cache using the *entityManager *as follows:

After the cleanup I did a heapdump and notices that the segments tables were cleared but the evictions LIRS header was not, causing the memory leak issue.
From the LIRS source code, only the accessQueue is cleared:
org\hibernate\internal\util\collections\BoundedConcurrentHashMap.java (line 1096):

Heapdump screenshot in Eclipse MAT:

Attachments

Activity

Show:

Christian Beikov December 2, 2020 at 8:46 AM

Try using the hibernate.query.in_clause_parameter_padding parameter as documented here: https://docs.jboss.org/hibernate/stable/orm/userguide/html_single/Hibernate_User_Guide.html#configurations-query

Alex CD December 1, 2020 at 11:58 AM

My original use case was clearing the query plan cache after polluting with lots of varying parameter queries, thus reducing the memory footprint.

I’ve resolved to lowering the cache size by lowering config params (plan_cache_max_soft_references, plan_cache_max_strong_references, plan_cache_max_size) and reducing distinct query count by filling the param list with same value until a common size.

Sanne Grinovero December 1, 2020 at 9:26 AM

Ok yes I can see your point of view - and I agree that 500MB is not a trivial amount - I’ll see what can be done, but could you share more about the use case?

This cleanup method on QueryPlanCache is designed to be invoked on SessionFactory shutdown, but you both seem to want doing something different.

Also related:

Alex CD November 30, 2020 at 11:33 PM

Conceptually, clearing a cache should not leave a 500MB internal structure that MAY be reused, OR NOT…

When the user manually clears the cache, he expects all cache memory to be freed.

Sanne Grinovero November 30, 2020 at 6:46 PM

Hi , that sounds odd indeed. Woud you be able to share a reproducer of some sorts, or provide some more details? I don’t see how the leak is possible if you close the SessionFactory.

Regarding the issue described by , apologies I didn’t notice this earlier, but I don’t think this qualifies as a “leak”: many Map implementations (including the JDK’s HashMap) don’t actually free the internal structure - and there’s good reasons for that, among others it’s better to not release these as the application is going to use such entries soon again, so it would create unnecessary memory churn.

Is there a reason to actually need to reclaim that memory back? (Unless you’re shutting down the SessionFactory of course)