Sometimes thread are locked forever when trying to get HQL query plan

Description

Hello!

Sometimes thread are locked forever when trying to get HQL query plan

At some point the number of locked threads which services user requests has growed up and no one of them could not response to users. Then we got many thread dumps every two seconds and after seeing of them we concluded that all http threads is locked on retreiving HQL query plan. All stack traces are identical for every locked thread:

"http-nio-8443-exec-270" #42815 daemon prio=5 os_prio=0 tid=0x00007febd62a0800 nid=0x3cec waiting on condition [0x00007feaec938000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000004c5ce6188> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
    at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
    at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
    at org.hibernate.internal.util.collections.BoundedConcurrentHashMap$Segment.attemptEviction(BoundedConcurrentHashMap.java:1583)
    at org.hibernate.internal.util.collections.BoundedConcurrentHashMap$Segment.get(BoundedConcurrentHashMap.java:1299)
    at org.hibernate.internal.util.collections.BoundedConcurrentHashMap.get(BoundedConcurrentHashMap.java:1852)
    at org.hibernate.engine.query.spi.QueryPlanCache.getHQLQueryPlan(QueryPlanCache.java:150)
    at org.hibernate.internal.AbstractSharedSessionContract.getQueryPlan(AbstractSharedSessionContract.java:546)
    at org.hibernate.internal.AbstractSharedSessionContract.createQuery(AbstractSharedSessionContract.java:655)
    at org.hibernate.internal.SessionImpl.createQuery(SessionImpl.java:3318)
    at org.hibernate.query.criteria.internal.CriteriaQueryImpl$1.buildCompiledQuery(CriteriaQueryImpl.java:318)
    at org.hibernate.query.criteria.internal.compile.CriteriaCompiler.compile(CriteriaCompiler.java:127)
    at org.hibernate.internal.SessionImpl.createQuery(SessionImpl.java:3611)
    at org.hibernate.internal.SessionImpl.createQuery(SessionImpl.java:203)
    at com.example.hibernate.HibernateSessionBean$CurrentSessionDelegator.createQuery(HibernateSessionBean.java:139)
    at com.example.CoreDaoImpl.createQuery(CoreDaoImpl.java:96)
    at com.example.CoreDao.createQuery(CoreDao.java:115)
    at com.example.CoreDao.createQuery(CoreDao.java:120)
    at com.example.CoreDao.getSingleResult(CoreDao.java:61)
    ...

Unfortunately there was not found the thread which has got this reentrant lock and is not giving away it.
Only rebooting of the application helped us.

Maybe there are some bug in getting of the lock and not releasing it in org.hibernate.internal.util.collections.BoundedConcurrentHashMap class.

Thank you!

Environment

HotSpot, JDK 8u101, CentOS, Oracle 12 DB

Activity

Show:
Владимир Тарасов
October 5, 2018, 10:34 AM

Hello!
The problem was in our code. We had StackOverflowException at the thread where lock was taken. Therefore it was not released.
You may to close this task.
Thanks!

Guillaume Smet
October 5, 2018, 10:39 AM

Thanks for the follow-up.

Assignee

Unassigned

Reporter

Владимир Тарасов

Fix versions

None

backPortable

None

Suitable for new contributors

None

Requires Release Note

None

Pull Request

None

backportDecision

None

Components

Affects versions

Priority

Major
Configure