Important: See "Approach 2: entity change events" in HSEARCH-3281.
As highlighted but the use case here:
Our "async" worker is delegating to a background thread the write-to-index aspect, but it still has to gather all data in the application thread synchronously.
This is not the first time such a thing is asked, and we might be able to batch some operations in the background work? For example we have plans to have the MassIndexer hinted on how to optimally load associations for a given type, the same hints could be applied here.
It could be in in all effects implemented by simply feeding the queue of an always available MassIndexer.
Note that one important aspect of this would be persistence of entity change events: if the server crashes after the transaction is committed, we need to be aware of the entities that still need reindexing.
There is previous work here: https://github.com/hibernate/hibernate-search/pull/979
We cannot use it directly, but we should have a look before we start working on an implementation using Kafka to store the events (so that we can eventually use Debezium).