Query plan cache memory leak after cleanup

Description

After a QueryPlanCache.cleanup() the cache still retains a lot of memory (~500MB), but it should be empty.
I empty the cache using the *entityManager *as follows:

After the cleanup I did a heapdump and notices that the segments tables were cleared but the evictions LIRS header was not, causing the memory leak issue.
From the LIRS source code, only the accessQueue is cleared:
org\hibernate\internal\util\collections\BoundedConcurrentHashMap.java (line 1096):

Heapdump screenshot in Eclipse MAT:

Environment

Java 1.8.0_162, JVM: Java HotSpot(TM) 64-Bit Server VM (25.162-b12, mixed mode), Windows 10, Postgres 10

Activity

Show:
Sanne Grinovero
November 30, 2020, 6:46 PM

Hi , that sounds odd indeed. Woud you be able to share a reproducer of some sorts, or provide some more details? I don’t see how the leak is possible if you close the SessionFactory.

Regarding the issue described by , apologies I didn’t notice this earlier, but I don’t think this qualifies as a “leak”: many Map implementations (including the JDK’s HashMap) don’t actually free the internal structure - and there’s good reasons for that, among others it’s better to not release these as the application is going to use such entries soon again, so it would create unnecessary memory churn.

Is there a reason to actually need to reclaim that memory back? (Unless you’re shutting down the SessionFactory of course)

Alex CD
November 30, 2020, 11:33 PM

Conceptually, clearing a cache should not leave a 500MB internal structure that MAY be reused, OR NOT

When the user manually clears the cache, he expects all cache memory to be freed.

Sanne Grinovero
December 1, 2020, 9:26 AM

Ok yes I can see your point of view - and I agree that 500MB is not a trivial amount - I’ll see what can be done, but could you share more about the use case?

This cleanup method on QueryPlanCache is designed to be invoked on SessionFactory shutdown, but you both seem to want doing something different.

Also related:

Alex CD
December 1, 2020, 11:58 AM

My original use case was clearing the query plan cache after polluting with lots of varying parameter queries, thus reducing the memory footprint.

I’ve resolved to lowering the cache size by lowering config params (plan_cache_max_soft_references, plan_cache_max_strong_references, plan_cache_max_size) and reducing distinct query count by filling the param list with same value until a common size.

Christian Beikov
December 2, 2020, 8:46 AM

Try using the hibernate.query.in_clause_parameter_padding parameter as documented here: https://docs.jboss.org/hibernate/stable/orm/userguide/html_single/Hibernate_User_Guide.html#configurations-query

Assignee

Unassigned

Reporter

Alex CD

Fix versions

Labels

backPortable

None

Suitable for new contributors

None

Requires Release Note

None

Pull Request

None

backportDecision

None

Components

Affects versions

Priority

Major
Configure