Provide a (better) way to configure Lucene directories on a per-index or per-shard basis

Description

In Search 5, all directory-related options could be configured on a per-index basis, or even per-shard when sharding was enabled.
This was both complicated, and potentially limiting for us, since resources (thread pools, ...) could not easily be shared between multiple indexes.

In Search 6, we currently only allow one set of directory-related options at the backend level.
While this is simpler and could allow more optimization, it is also very restricting for users: they won't be able to, for example, store different indexes or different shards of the same index on different disks.
They could put each index in a different backend, but that would prevent multi-index searches (as it's not possible to search across multiple backends).

Maybe we should find some middle ground?
The current structure of configuration properties is fine, but we could introduce an alternative syntax when multiple storage configurations are needed.

For example:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Define the default configuration as usual # Maybe rename .directory to .store? hibernate.search.backend.myBackend.store.type = local-filesystem hibernate.search.backend.myBackend.store.root = /data/indexes/common # Define a named store for write-intensive indexes (stored on a specific disk) hibernate.search.backend.myBackend.stores.write-intensive.type = local-filesystem hibernate.search.backend.myBackend.stores.write-intensive..root = /data/indexes/highthroughput # Define a named store for in-memory indexes hibernate.search.backend.myBackend.stores.in-memory.type = local-heap # Assign stores to specific indexes; indexes that are not mentioned will use the default store hibrenate.search.backend.indexes.com.mycompany.model.BusinessEvent.store = write-intensive hibrenate.search.backend.indexes.com.mycompany.model.TechnicalEvent.store = in-memory # Maybe even allow assigning a different store to each shard of a given index? hibrenate.search.backend.indexes.com.mycompany.model.SomeShardedEntity.0.store = store0 hibrenate.search.backend.indexes.com.mycompany.model.SomeShardedEntity.1.store = store1 hibrenate.search.backend.indexes.com.mycompany.model.SomeShardedEntity.2.store = store2

"store" is just a first attempt at naming this concept, but we should look for a better term.

Environment

None

Status

Assignee

Unassigned

Reporter

Yoann Rodière

Labels

None

Suitable for new contributors

None

Pull Request

None

Feedback Requested

None

Components

Fix versions

Priority

Major