Attributeconverter converttoEntityAttribute and convertToDatabaseColumn code is called within a query.list() call -> slow

Description

I am not sure if this not really is a bug.
StandardCacheEntryImpl calls TypeHelper.disassemble which causes the freshly converted (converttoEntityAttribute) attribute again to be converted the other way round (convertToDatabaseColumn). If this conversion is slow this is a problem. This is processed within one query.list() call. It would be nice if there was a way to suppress this forward and backward conversion that slows processing down a lot.

It ends up to be in AttributeConverterMutabilityPlan.deepCopyNotNull(). There I can see the convertToDatabaseColumn() method to be called immediately followed by a call to converttoEntityAttribute(). If I somehow could switch this off my application would be several orders of magnitude faster. What is this for?:

public class AttributeConverterMutabilityPlanImpl<T> extends MutableMutabilityPlan<T> {
private final AttributeConverter attributeConverter;

public AttributeConverterMutabilityPlanImpl(AttributeConverter attributeConverter) {
this.attributeConverter = attributeConverter;
}

@Override
@SuppressWarnings("unchecked")
protected T deepCopyNotNull(T value) {
return (T) attributeConverter.convertToEntityAttribute( attributeConverter.convertToDatabaseColumn( value ) );
}
}

Here is the relevant part of the stacktrace:

AttributeConverterMutabilityPlanImpl<T>.deepCopyNotNull(T) line: 29
AttributeConverterMutabilityPlanImpl<T>(MutableMutabilityPlan<T>).deepCopy(T) line: 35
AttributeConverterMutabilityPlanImpl<T>(MutableMutabilityPlan<T>).disassemble(T) line: 24
AttributeConverterTypeAdapter<T>(AbstractStandardBasicType<T>).disassemble(Object, SessionImplementor, Object) line: 284
TypeHelper.disassemble(Object[], Type[], boolean[], SessionImplementor, Object) line: 129
StandardCacheEntryImpl.<init>(Object[], EntityPersister, Object, SessionImplementor, Object) line: 55
AbstractEntityPersister$StandardCacheEntryHelper.buildCacheEntry(Object, Object[], Object, SessionImplementor) line: 5216
SingleTableEntityPersister(AbstractEntityPersister).buildCacheEntry(Object, Object[], Object, SessionImplementor) line: 4227
TwoPhaseLoad.doInitializeEntity(Object, EntityEntry, boolean, SessionImplementor, PreLoadEvent) line: 182
TwoPhaseLoad.initializeEntity(Object, boolean, SessionImplementor, PreLoadEvent) line: 125
QueryLoader(Loader).initializeEntitiesAndCollections(List, Object, SessionImplementor, boolean, List<AfterLoadAction>) line: 1139
QueryLoader(Loader).processResultSet(ResultSet, QueryParameters, SessionImplementor, boolean, ResultTransformer, int, List<AfterLoadAction>) line: 998
QueryLoader(Loader).doQuery(SessionImplementor, QueryParameters, boolean, ResultTransformer) line: 936
QueryLoader(Loader).doQueryAndInitializeNonLazyCollections(SessionImplementor, QueryParameters, boolean, ResultTransformer) line: 342
QueryLoader(Loader).doList(SessionImplementor, QueryParameters, ResultTransformer) line: 2622
QueryLoader(Loader).listUsingQueryCache(SessionImplementor, QueryParameters, Set<Serializable>, Type[]) line: 2464
QueryLoader(Loader).list(SessionImplementor, QueryParameters, Set<Serializable>, Type[]) line: 2426
QueryLoader.list(SessionImplementor, QueryParameters) line: 501
QueryTranslatorImpl.list(SessionImplementor, QueryParameters) line: 371
HQLQueryPlan.performList(QueryParameters, SessionImplementor) line: 216
SessionImpl.list(String, QueryParameters) line: 1339
QueryImpl.list() line: 87

Here is a test that shows the issue in the behavior.:

package org.hibernate.test.converter;

import java.beans.XMLDecoder;
import java.beans.XMLEncoder;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.persistence.AttributeConverter;
import javax.persistence.Convert;
import javax.persistence.Converter;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.Lob;
import javax.persistence.Table;

import org.hibernate.Session;
import org.hibernate.cfg.Configuration;
import org.hibernate.cfg.Environment;
import org.hibernate.query.Query;
import org.hibernate.testing.junit4.BaseCoreFunctionalTestCase;
import org.junit.Test;

import static org.junit.Assert.assertEquals;

/**

  • Test to check the number of attributeconverter calls on a simple save and list
    *

  • @author Carsten Hammer
    */
    public class AttributeConverterLobTest extends BaseCoreFunctionalTestCase {
    @Override
    protected Class<?>[] getAnnotatedClasses() {
    return new Class[] { EntityImpl.class };
    }


@Override
public void configure(Configuration cfg) {
super.configure( cfg );
cfg.setProperty( Environment.USE_SECOND_LEVEL_CACHE, "true" );
cfg.setProperty( Environment.GENERATE_STATISTICS, "true" );
}

@Test
public void testMappingAttributeWithLobAndAttributeConverter() {
Session session = openSession();
session.beginTransaction();
EntityImpl object = new EntityImpl();
object.status=new HashMap<>();
object.status.put( "asdf", Integer.valueOf( 6 ) );
object.status.put( "key", "table" );
object.id=1;
session.save( object );
session.getTransaction().commit();
session.close();
/**

  • What? Why the hell 2 and not 1?
    */
    assertEquals(2,ConverterImpl.todatabasecounter);
    /**
    * Why a from database conversion at all?
    */
    assertEquals(1,ConverterImpl.fromdatabasecounter);


session = openSession();
session.beginTransaction();
Query<EntityImpl> createQuery = session.createQuery( "select e from EntityImpl e", EntityImpl.class );
List<EntityImpl> resultList = createQuery.getResultList();
assertEquals(1,resultList.size());
session.getTransaction().commit();
session.close();
/**

  • Why again a to database conversion? These conversions are very expensive and should only be done if really needed..
    */
    assertEquals(3,ConverterImpl.todatabasecounter);
    assertEquals(3,ConverterImpl.fromdatabasecounter);
    assertEquals("table",resultList.get(0 ).status.get( "key" ));
    assertEquals(3,ConverterImpl.fromdatabasecounter);
    }

@Converter
public static class ConverterImpl implements AttributeConverter<Map, byte[]> {
public static int todatabasecounter=0;
public static int fromdatabasecounter=0;
@Override
public byte[] convertToDatabaseColumn(Map map) {
todatabasecounter++;
ByteArrayOutputStream out=new ByteArrayOutputStream();
try(XMLEncoder encoder=new XMLEncoder(out)){
encoder.writeObject( map );
}
return out.toByteArray();
}

@Override
public Map convertToEntityAttribute(byte[] dbData) {
fromdatabasecounter++;
try(ByteArrayInputStream in=new ByteArrayInputStream(dbData)){
XMLDecoder decoder=new XMLDecoder(in);
return (Map) decoder.readObject();
}
catch (IOException e) {
return null;
}
}
}

@Entity(name = "EntityImpl")
@Table( name = "EntityImpl" )
public static class EntityImpl {
@Id
private Integer id;

@Lob
@Convert(converter = ConverterImpl.class)
private Map status;
}
}

Activity

Show:

Carsten Hammer September 29, 2016 at 4:18 PM

Steve explained in https://hibernate.atlassian.net/browse/HHH-10818#icft=HHH-10818 that the junit tests I created for this issue at https://github.com/hibernate/hibernate-orm/pull/1547 are wrong. I do not understand why. Can anybody who lives on this planet explain why? winking face

Carsten Hammer September 21, 2016 at 6:23 AM

It seems AvailableSettings.USE_DIRECT_REFERENCE_CACHE_ENTRIES=true causes the entity to be used as reference in the cache. Seems not to help in this case because this case is about a field of the entity and not the entity itself. Would it be theoretically possible to implement something similar for the field/attributeconverter to get rid of the attribute conversions when reusing objects from cache? Otherwise I have a lot of javatypedescriptors to implement.

Carsten Hammer September 19, 2016 at 1:53 PM

Yes, what can I say - you are right, should be obvious, sometimes I do not see obvious things. I guess as as soon as I understand why the attributeconverter has to be called in both directions in the junit test I will think it is obvious too.

Steve Ebersole September 19, 2016 at 1:26 PM

I mean it seems kind of obvious that Hibernate would determine dirtiness based on the individual state, rather than #equals on the entity itself...

Carsten Hammer September 16, 2016 at 9:08 AM

I applied some changes to my junit test code in the pull request that shows that the attributeconverter is called in in both directions without taking the second level cache into account it seems. Hope that my expectation (and therefore the junit test) is not too stupid. This is what I currently see although I think my real problem is something else - maybe what you point me to.
I am still investigating. I wished there would be some way to get information from hibernate why and when it decides to do the conversions/update, a log setting or aop trick, something like this. If not only the entitys equals/hashCode methods are taken into account but also the entities properties equals/hashCode method this is something that I was not aware of.
Thank you so far!

Details

Assignee

Reporter

Components

Priority

Created September 12, 2016 at 7:25 PM
Updated September 29, 2016 at 4:18 PM