HibernateSearchIndexes gets corrupted with exception : java.io.EOFException: read past EOF: MMapIndexInput (path="/.../.../_39d.cfs")

Description

On production Env

Ours is a J2EE application (clustered across 2 nodes) internally using Hibernate Core 4.3.6, Spring 4.3.1 and Hibernate Search 4.5.1.
We are using hibernate search indexes in our application heavily.
These are stored on FILESYSTEM - ZFS (Extreme Flash on Oracle ExaStack), and shared between both the nodes.

Application runs fine, but frequent locking exception comes in logs. (Cause: Unable to acquire lock)
We are concerned that, these exceptions might cause inconsistency in data.
However application seems to be working fine - in terms of user experience.

But if we now restart the applications (both the nodes), application startup fails with exception: java.io.EOFException: read past EOF: MMapIndexInput (path="/.../.../_39d.cfs")
Note:The file name '_39d.cfs' is not fixed, and it varies in different occurences.

As this is a production env, it is impacting adversely.

We have created a trigger at server startup, which based on a flag, manually creates all the hibernate indexes at startup. (Takes around 12-14 hours)
So as a workaround to this issue, we delete all the corrupted hibernate indexes from the filesystem, and recreate the indexes manually using this startup trigger.

Stack Trace:
<Error> <Deployer> <PRODCAS1-172-16-5-29> <CAS-SERVER-OHS-1> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <8118a889-fb3b-41f7-9951-b3129fe5130a-00000044> <1515523387207> <BEA-149265> <Failure occurred in the execution of deployment request with ID "17254792326560163" for task "23". Error is: "weblogic.application.ModuleException: java.io.EOFException: read past EOF: MMapIndexInput(path="/artifacts_dumps/hibernate-search_index-caslms/casTCFSL/com.nucleus.contact.PhoneNumber/_39d.cfs")"
weblogic.application.ModuleException: java.io.EOFException: read past EOF: MMapIndexInput(path="/artifacts_dumps/hibernate-search_index-caslms/casTCFSL/com.nucleus.contact.PhoneNumber/_39d.cfs")
at weblogic.application.internal.ExtensibleModuleWrapper.start(ExtensibleModuleWrapper.java:140)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:124)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:216)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:211)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:73)
at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:24)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:729)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:258)
at weblogic.application.internal.SingleModuleDeployment.activate(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:165)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:80)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:587)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:150)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:116)
at weblogic.deploy.internal.targetserver.operations.StartOperation.doCommit(StartOperation.java:151)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:339)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:846)
at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1275)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:442)
at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:176)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:548)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)
Caused By: java.io.EOFException: read past EOF: MMapIndexInput(path="/artifacts_dumps/hibernate-search_index-caslms/casTCFSL/com.nucleus.contact.PhoneNumber/_39d.cfs")
at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte(MMapDirectory.java:284)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:107)
at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:69)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:75)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94)
at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:105)
at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:256)
at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:143)
at org.hibernate.search.indexes.impl.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:218)
at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:120)
at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.initialize(SharingBufferReaderProvider.java:103)
at org.hibernate.search.indexes.impl.PropertiesParseHelper.createDirectoryBasedReaderProvider(PropertiesParseHelper.java:158)
at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.createIndexReader(DirectoryBasedIndexManager.java:208)
at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:104)
at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:261)
at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:528)
at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManagers(IndexManagerHolder.java:495)
at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:104)
at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:363)
at org.hibernate.search.spi.SearchFactoryBuilder.buildNewSearchFactory(SearchFactoryBuilder.java:219)
at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:143)
at org.hibernate.search.hcore.impl.HibernateSearchSessionFactoryObserver.sessionFactoryCreated(HibernateSearchSessionFactoryObserver.java:74)
at org.hibernate.internal.SessionFactoryObserverChain.sessionFactoryCreated(SessionFactoryObserverChain.java:52)
at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:588)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1857)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:850)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:843)
at org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.withTccl(ClassLoaderServiceImpl.java:398)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:842)
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:60)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:338)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:373)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:362)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor.findDefaultEntityManagerFactory(PersistenceAnnotationBeanPostProcessor.java:580)
at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor.findEntityManagerFactory(PersistenceAnnotationBeanPostProcessor.java:546)
at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor$PersistenceElement.resolveEntityManager(PersistenceAnnotationBeanPostProcessor.java:712)
at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor$PersistenceElement.getResourceToInject(PersistenceAnnotationBeanPostProcessor.java:685)
at org.springframework.beans.factory.annotation.InjectionMetadata$InjectedElement.inject(InjectionMetadata.java:169)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor.postProcessPropertyValues(PersistenceAnnotationBeanPostProcessor.java:354)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1214)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:543)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:187)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1213)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1053)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1018)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:566)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:349)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1214)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:543)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:187)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1213)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1053)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1018)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:566)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:349)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1214)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:543)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.PostProcessorRegistrationDelegate.registerBeanPostProcessors(PostProcessorRegistrationDelegate.java:240)
at org.springframework.context.support.AbstractApplicationContext.registerBeanPostProcessors(AbstractApplicationContext.java:697)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:526)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
at weblogic.servlet.internal.EventsManager$FireContextListenerAction.run(EventsManager.java:678)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
at weblogic.servlet.internal.EventsManager.executeContextListener(EventsManager.java:243)
at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:200)
at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:185)
at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1838)
at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:2876)
at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1661)
at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:823)
at weblogic.application.internal.ExtensibleModuleWrapper$StartStateChange.next(ExtensibleModuleWrapper.java:360)
at weblogic.application.internal.ExtensibleModuleWrapper$StartStateChange.next(ExtensibleModuleWrapper.java:356)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
at weblogic.application.internal.ExtensibleModuleWrapper.start(ExtensibleModuleWrapper.java:138)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:124)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:216)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:211)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:73)
at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:24)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:729)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:258)
at weblogic.application.internal.SingleModuleDeployment.activate(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:165)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:80)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:587)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:150)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:116)
at weblogic.deploy.internal.targetserver.operations.StartOperation.doCommit(StartOperation.java:151)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:339)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:846)
at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1275)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:442)
at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:176)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:548)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

Note: Before moving to clustered env (when onlt 1 node was there), neither we face any locking issue, nor we face this corruption of indexes issue.

Environment

Production Env - Banking Domain

Oracle Linux - Exalogic/Exadata stack
AS: Weblogic 12.1.3
DB: Oracle 12.1.0.2
JDK: 1.8

Activity

Show:
Yoann Rodière
February 16, 2018, 7:28 AM

How do you handle the index synchronization between your two nodes?

Moving to a clustered architecture is not just a matter of sharing the index, you also have to take care to synchronize writes to the index so that two simultaneous writes would not corrupt the index. There is a variety of ways to achieve this, described in the official documentation: https://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#_backend

Please give us more information:

  • describe in more details which method you chose

  • tell us how exactly you are sharing the index between nodes (shared drive mounted over each node? periodic synchronization?)

  • give us the detailed Hibernate Search configuration (the "hibernate.search.*" configuration properties)

Debashish Bharali
February 16, 2018, 7:53 AM

Thanks for prompt reply.
Let me go through the doc.
I will share you the requested info ASAP.

Debashish Bharali
February 16, 2018, 10:28 AM
Edited

Please find below the requested info:
1. Hibernate Search Configuration/Properties:

  1. hibernate.search.indexing_strategy — event

  2. hibernate.search.default.directory_provider — filesystem

  3. hibernate.search.default.directory_provider — native

  4. hibernate.search.default.indexBase — /artifacts_dumps/hibernate-search_index-caslms/casTCFSL

  5. hibernate.search.generate_statistics — true

  6. hibernate.search.jmx_enabled — true

  7. hibernate.search.default.worker.execution — async

  8. hibernate.search.worker.thread_pool.size — 20

  9. hibernate.search.default.exclusive_index_use — false

2.Tell us how exactly you are sharing the index between nodes:

  1. Within both the nodes, we have specified same location in 'hibernate.search.default.indexBase' property.

  2. This location is in a network shared drive (internal to Oracle ExaStack - ZFS) - mounted separately on both the VMs. (The ZFS is accessed over IPoIB - Infiniband - i.e. network is internal to the ExaStack - on same physical stack)

3. describe in more details which method you chose.

  1. We are using Lucene (ver 3.6.2) for this. All the update calls are handled internally by Lucene.

Sanne Grinovero
February 16, 2018, 11:56 AM

Hi ,

version 4.x of Hibernate Search is no longer maintained so such a topic is better addressed in a forum. Also, it's not a bug but a configuration mistake so I'll close it.

Lucene indexes are getting "locked" when they are written to in exclusive mode, so to protect them against multiple writers corrupting the index. So normally you should have only strictly one instance of Hibernate Search using one index path; the default is to not allow this, exactly to prevent issues like you are reporting.

The `exclusive_index_use` being set to false is an explicit request to disable this safety, but you should strictly only use this when your application can guarantee that no writes are ever going to be triggered on the index from both applications. Clearly you describe a scenario in which both applications are concurrenctly attempting to write on the same index, and that's why you get Unable to acquire lock exceptions.

You have various options:

  • Change your application so that only one cluster makes writes to the index

  • Do not use clustering but deploy both applications to use the same Hibernate Search instance

  • Let one Hibernate Search instance delegate the write operations ot the other one

This last solution is what is explained in the manual in this section:

For the older version:

Please read the related chapters on architecture as well, so that you can make an informed choice depending on your needs.

Sanne Grinovero
February 16, 2018, 11:57 AM

Assignee

Unassigned

Reporter

Debashish Bharali

Labels

Suitable for new contributors

None

Pull Request

None

Feedback Requested

None

Priority

Blocker
Configure