Natural id cache is extremely slow for entities with compound natural id
Description
Attachments
- 28 Mar 2023, 01:25 PM
- 22 Feb 2023, 04:41 PM
Activity
Thorsten Goetzke March 28, 2023 at 1:25 PM
Hello,
I am locally patching around that as well. A simple Arrays.hashCode() seems to break, because it can cause entity loading
org.hibernate.proxy.AbstractLazyInitializer.initialize-> into afterTransaction → into org.hibernate.resource.jdbc.internal.ResourceRegistryStandardImpl.releaseResources and after that sometimes some JDBC Statements are closed that are expected to be open.
My current idea is to try this, which is probably not correct in all cases, but at least it doesnt crash the application or makes it so slow that it might as well be crashed…:
My argument would be that it would be nice if there was an option to completly turn off the entire Natural-Id cache handling. Our application never uses Natural id lookup, the natural ids are just there so that the sql schema generation creates the required unique constraints. As a workaround we could convert natural-ids to explicit unique constraints, but thats a lot of rewriting.
Sylvain Dusart March 6, 2023 at 3:33 PM
I just created the PR.
Sylvain Dusart February 22, 2023 at 11:16 PMEdited
I think the hashCode must be computed on the result of disassemble( value, session ), otherwise each part of the compound natural id would require an implementation of equals/hashCode methods (not the case for the State class used in NaturalIdTest#testNaturalIdLoaderCached).
I have pushed my modifications to https://github.com/sdusart/hibernate-orm/tree/HHH-16218 but I’m really not confident about what I did…
At least, ./gradlew clean build succeeds 
Sylvain Dusart February 22, 2023 at 5:36 PM
My suggestion seems to break at least one test in hibernate-core :
{code}
Task :hibernate-core:test
NaturalIdTest > testNaturalIdLoaderCached FAILED
java.lang.AssertionError at NaturalIdTest.java:224
11764 tests completed, 1 failed, 1515 skipped
Task :hibernate-core:test FAILED
FAILURE: Build failed with an exception.
{code}
I’ll try to have a look soon.
Hibernate maintains a cache for id<->naturalId correspondances (pkToNaturalIdMap and naturalIdToPkMap maps in org.hibernate.engine.internal.NaturalIdResolutionsImpl.EntityResolutions#cache).
This cache is updated after creations and loads (at least, I did not test other use-cases) .
In the maps, the natural id is encapsulated in a ResolutionImpl instance and the hashcode is computed in the constructor :
final int prime = 31; int hashCodeCalculation = 1; hashCodeCalculation = prime * hashCodeCalculation + entityDescriptor.hashCode(); hashCodeCalculation = prime * hashCodeCalculation + entityDescriptor.getNaturalIdMapping().calculateHashCode( naturalIdValue, persistenceContext.getSession() );For entities with a simple natural id, entityDescriptor.getNaturalIdMapping() is an instance of SimpleNaturalIdMapping with this calculateHashCode method :
@Override public int calculateHashCode(Object value) { //noinspection unchecked return value == null ? 0 : ( (JavaType<Object>) getJavaType() ).extractHashCode( value ); }For entities with a compound natural id, entityDescriptor.getNaturalIdMapping() is an instance of CompoundNaturalIdMapping with this calculateHashCode method :
@Override public int calculateHashCode(Object value) { return 0; }As a consequence, for a given entity with a compound natural id, the hashcodes of all ResolutionImpl objects used as keys in naturalIdToPkMap are the same, which creates collisions and consumes a lot of cpu cycles.
The calculateHashCode method in CompoundNaturalIdMapping should be something like
@Override public int calculateHashCode(Object value) { return Arrays.hashCode((Object[]) value); }to align with the areEqual method.
The attached test case creates 20000 objects in database then re-reads these objects.
It is done for two entity classes : EntityWithSimpleNaturalId and EntityWithCompoundNaturalId.
Logs before modification :
16:33:31,782 INFO CompoundNaturalIdCacheTest:64 - Starting creations 16:33:53,745 INFO CompoundNaturalIdCacheTest:74 - Persisted 20000 EntityWithCompoundNaturalId objects, duration=21957ms 16:33:53,906 INFO CompoundNaturalIdCacheTest:83 - Persisted 20000 EntityWithSimpleNaturalId objects, duration=160ms 16:33:55,140 INFO CompoundNaturalIdCacheTest:115 - Loading at most 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithCompoundNaturalId 16:34:12,915 INFO CompoundNaturalIdCacheTest:123 - Loaded 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithCompoundNaturalId, duration=17777ms 16:34:12,921 INFO CompoundNaturalIdCacheTest:115 - Loading at most 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithSimpleNaturalId 16:34:13,021 INFO CompoundNaturalIdCacheTest:123 - Loaded 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithSimpleNaturalId, duration=99msLogs after modification :
16:35:37,950 INFO CompoundNaturalIdCacheTest:64 - Starting creations 16:35:38,247 INFO CompoundNaturalIdCacheTest:74 - Persisted 20000 EntityWithCompoundNaturalId objects, duration=292ms 16:35:38,368 INFO CompoundNaturalIdCacheTest:83 - Persisted 20000 EntityWithSimpleNaturalId objects, duration=119ms 16:35:39,679 INFO CompoundNaturalIdCacheTest:115 - Loading at most 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithCompoundNaturalId 16:35:40,096 INFO CompoundNaturalIdCacheTest:123 - Loaded 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithCompoundNaturalId, duration=418ms 16:35:40,103 INFO CompoundNaturalIdCacheTest:115 - Loading at most 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithSimpleNaturalId 16:35:40,232 INFO CompoundNaturalIdCacheTest:123 - Loaded 20000 instances of class org.hibernate.orm.test.mapping.naturalid.compound.CompoundNaturalIdCacheTest$EntityWithSimpleNaturalId, duration=129ms