Massindexer Freezes
Description
Attachments
- 27 Apr 2017, 07:44 AM
- 27 Apr 2017, 07:42 AM
relates to
Activity
Yoann RodièreApril 27, 2017 at 9:57 AM
Glad I could help. Since this was not a problem with Hibernate Search itself, I'm going to close this ticket as Rejected.
Should you encounter other problems, feel free to ask for support on stackoverflow, with the "hibernate-search" tag: we monitor this tag and try to answer our best.
On a side note, you might be interested to know that we are working on a way to integrate with dependency injection mechanisms (such as Spring DI) so that field bridges can be injected with @Autowired or @Inject. Experimental support may come in version 5.8, if we can merge it in time. You can follow the progress on this ticket: HSEARCH-1316.
Patrice OmsApril 27, 2017 at 9:37 AMEdited
Hello Yoann, Thanks for this quick reply you are absolutely right, calling applicationContext.getBean(...) from the set method creates the lock !
The issue is the FieldBridge is not a SpringComponent that's why I had to use this applicationContext.getBean(..)
You solution is working, and I have found another one using a AutowireCapableBeanFactory:
@Component
class MessageFieldBridge extends MetadataProvidingFieldBridge {
@Autowired
var translationService: MySpringService = _
var initialized = false
def initBean() = {
if(!initialized) {
SpringContextBridgeImpl.get().autowireBean(this)
initialized = true
}
}
override def set(name: String, value: scala.Any, document: Document, luceneOptions: LuceneOptions): Unit = {
initBean()
// Add things to document using autowired beans
}
override def configureFieldMetadata(name: String, builder: FieldMetadataBuilder): Unit = {
initBean()
// do things using autowired beans...
}
}And the Spring Bridge:
trait SpringContextBridge {
def autowireBean(bean: AnyRef)
}
@Service
class SpringContextBridgeImpl extends SpringContextBridge with ApplicationContextAware {
@Autowired
var autowireBeanFactory: AutowireCapableBeanFactory = _
override def autowireBean(bean: AnyRef) = {
autowireBeanFactory.autowireBean(bean)
}
override def setApplicationContext(applicationContext: ApplicationContext): Unit = {
SpringContextBridgeImpl.applicationContext = applicationContext
}
}
object SpringContextBridgeImpl {
private var applicationContext: ApplicationContext = _
def get(): SpringContextBridge = applicationContext.getBean(classOf[SpringContextBridge])
}Yoann RodièreApril 27, 2017 at 8:26 AM
Hello @Patrice Oms,
Thanks for your report. Could you please give us the reference of the previous ticket you are mentioning?
From the screen captures you gave us, I can see indeed that a "Hibernate Search sync consumer thread" is waiting. However, these threads, as their names indicate, are consumers, and expect input so that they can work on it. So the fact they are waiting is probably not the cause of your problem, but rather the symptom: something that is supposed to create the input is not creating the input.
What's more interesting is the "Hibernate Search: entityloader-1" thread. This thread is in monitor status (read: blocked, waiting for synchronization). What's more, this thread is executing some of your own code: the MessageFieldBridge class, the set() method. The thread is actually blocking while trying to retrieve a Spring Bean.
Could you confirm that you are not trying to retrieve a spring bean each time the MessageFieldBridge.set is called? For example by calling ApplicationContext.getBean()? You must realize that MessageFieldBridge.set is in the hot path of the indexing code, so calling potentially costly operations such as ApplicationContext.getBean() inside this method could potentially slow down the indexing process to such an extent that it would seem to stop. To solve that, you should cache the bean somehow (for instance add a private volatile MyBean myBean field to your bridge, and replace your current code (applicationContext.getBean(...).doSomething()) with something like:
if ( myBean == null ) {
myBean = applicationContext.getBean( ... );
}
myBean.doSomething();Details
Assignee
UnassignedUnassignedReporter
Patrice OmsPatrice OmsComponents
Affects versions
Priority
Blocker
Details
Details
Assignee
Reporter
Components
Affects versions
Priority
Blocker
This behaviour has been reported in a previous version and reported as closed, but we are still facing the MassIndexer freeze from time to time, on both development and production environment.
Our datasource is configured with default Spring Boot pool settings (max 100 connections) and is a oracle-xe-11 in a docker container for development and a regular oracle in production.
datasource: platform: oracle initialize: false continueOnError: true driverClassName: oracle.jdbc.OracleDriver url: jdbc:oracle:thin:@//localhost:1521/XE test-on-borrow: true test-while-idle: true validation-query: select 1 from dual jpa: show-sql: false database-platform: org.hibernate.dialect.Oracle10gDialect hibernate: naming-strategy: org.hibernate.cfg.ImprovedNamingStrategy ddl-auto: noneWe are indexing 7 entity types, the reindexing code is pretty straightforward
@Transactional override def rebuildIndex: Unit = { try { val start = System.currentTimeMillis() LOGGER.info("Creating Lucene Index...") val fullTextEntityManager: FullTextEntityManager = Search.getFullTextEntityManager(entityManager) fullTextEntityManager.createIndexer().startAndWait() LOGGER.info("Index created successfully in " + (System.currentTimeMillis() - start) + "ms") } catch { case e: InterruptedException => LOGGER.error("An error occurred trying to build the search index: " + e) }The reindexing sometimes freezes and sometimes completes successfully. When the reindexing freezes we have this in the logs then it stops:
2017-04-27 09:39:56.770 INFO 11997 --- [ entityloader-1] o.h.s.b.i.SimpleIndexingProgressMonitor : HSEARCH000030: 1750 documents indexed in 5337 ms 2017-04-27 09:39:56.771 INFO 11997 --- [ entityloader-1] o.h.s.b.i.SimpleIndexingProgressMonitor : HSEARCH000031: Indexing speed: 327.899567 documents/second; progress: 98.70% 2017-04-27 09:39:56.907 INFO 11997 --- [ntifierloader-1] o.h.s.b.i.SimpleIndexingProgressMonitor : HSEARCH000027: Going to reindex 19 entitiesWith the help of debugger we can see that all thread are waiting