Uploaded image for project: 'Hibernate ORM'
  1. HHH-12448

Possible memory leak in Envers due to Narayana Transaction Reaper

    Details

    • Bug Testcase Reminder (view):

      Bug reports should generally be accompanied by a test case!

    • Last commented by a user?:
      true
    • Sprint:

      Description

      Hi,

      As requested in the forum (post below) I have created this issue.

      In a JTA multi-threaded environment we are working on a project where we are processing large amounts of data. But we ran into a problem where it seems that our software is leaking memory. We analyzed the heapdumps and determined that it was Envers’s AuditProcessManager that was keeping data from being garbage collected.

      Perhaps this issue should be posted in the Jboss/Narayana forums. I have been debugging and testing and determined that the cause lies in the fact that the transaction times out and the Transaction Reaper aborts the transaction at a point where Hibernate is not getting a chance to cleanup.

      The issue seems to be related and/or originated in the following issue/fix: HHH-7910 Closed

      I have tried to write it down as simple as possible. The following is happening (Note that the lock I am talking about is the _syncLock in TwoPhaseCoordinator):
      Thread: work
      Thread: flush. Envers registers beforeCompletion and afterCompletion on Session’s actionQueue (afterCompletion will do the cleanup)
      Thread: doCommit
      Thread: beforeCompletion -> lock -> Envers does its work
      Reaper: abort transaction
      Reaper: afterCompletion -> wait for lock
      Thread: finds out transaction is rolled back and runs delayedAfterCompletion
      Thread: ends beforeCompletion -> unlock
      Reaper: gets lock, calls afterCompletion and finds out it is not the original Thread, thus sets delayedAfterCompletion.
      Thread: afterCompletion -> wait for lock
      Reaper: finishes afterCompletion -> unlock
      Thread: gets lock and finds out afterCompletion has already been called (_afterCalled in TwoPhaseCoordinator) -> nothing to do and unlock

      Now Envers still has a reference to the audit-work in the Map in AuditProcessManager it is supposed to cleanup during afterCompletion, but it will never be called.

      This issues has been found using an older version of Hibernate and Narayana, but I have tested this in a small example application with the latest version of Hibernate (5.2.15.Final) and the latest version of Narayana (5.8.0.Final) and I can reproduce the issue.

      I have created a github-repository where I have pushed my test application: https://github.com/ronaldmik/memoryleak

      Any help would be appreciated. If this should (also) be posted in the Jboss/Narayana forums, let me know.

      Thanks in advance!

      Forum post: https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: