Out of Date
Details
Details
Assignee
Unassigned
UnassignedReporter
Yoann Rodière
Yoann RodièreComponents
Fix versions
Affects versions
Priority
Created May 15, 2017 at 2:05 PM
Updated September 26, 2023 at 9:37 AM
Resolved September 26, 2023 at 9:37 AM
See https://hibernate.atlassian.net/browse/HSEARCH-2674#icft=HSEARCH-2674 : it's approximately the same problem.
When you have multiple Hibernate Search nodes, all of them targeting the same indexes in a single Elasticsearch cluster, you may end up having some "dynamic shards" (i.e. ES indexes) ignored when querying, simply because each Hibernate Search node is only aware of the shards it created itself.
One solution would be to not ever mention the index names when querying (except when using explicit shard filtering), and only filter by type.
We'd have to check how well this performs, though: ideally Elasticsearch would first narrow down the list of indexes to query based on all the indexes that mention the targeted types in their metadata, but I'm not sure it does.
=> Actually no, we cannot do this, because the list of targeted indexes may or may not have been filtered by the
ShardIdentifierProvider
depending on filters, and we have absolutely no clue as to whether there has been some filtering. So we cannot arbitrarily decide to make the query target every index... I'll mention this issue in HSEARCH-2674, which I'm afraid will have to be fixed first.=> Also, I tried a bit to execute queries on all indexes, but on a specific set of types, and it seems Elasticsearch actually executes the request against all indexes, so the performance gain (which was the point of dynamic sharding in the first place) is lost, and we may even have worse performance than without sharding... Maybe we should consider using Elasticsearch shards + custom routing instead of creating multiple indices when doing dynamic sharding in the Elasticsearch case? (see HSEARCH-2634)