Async backends using Infinispan should enable Directory option writeFileListAsynchronously

Description

This is an option which allows storage of index related metadata to be pushed to the cluster using asynchronous operations.
Very effective on throughput but I suspect it's only safe to enable for async backends.
It might be ok for synchronous backends too but that still needs extensive testing; for now I would only expose the configuration property and enable it by default for async backends.

See also:

Environment

None

Activity

Show:
Gunnar Morling
January 22, 2015, 12:51 PM

, : Some questions for you guys:

  • Is the property "hibernate.search.default.write_metadata_async" established in actually still needed, if it can be set automatically depending on whether the backend is async or not? In other words, is there a need to set it to false for an async backend / set it to true for a sync backend? Otherwise it'd be nice to get rid of one of these knobs people need to reason about.

  • If the property still is needed, why is it not named "hibernate.search.{default|<index_name}.infinispan.write_metadata_async"? The property only makes sense for ISPN, and IMO that should be reflected in the name. That was suggested already back then, but apparently that proposal was never picked up.

Hardy Ferentschik
January 22, 2015, 2:18 PM

Is the property "hibernate.search.default.write_metadata_async" established in actually still needed, if it can be set automatically depending on whether the backend is async or not? In other words, is there a need to set it to false for an async backend / set it to true for a sync backend? Otherwise it'd be nice to get rid of one of these knobs people need to reason about.

no idea. this is more 's cup of tea

If the property still is needed, why is it not named "hibernate.search. {default|<index_name}.infinispan.write_metadata_async"? The property only makes sense for ISPN, and IMO that should be reflected in the name. That was suggested already back then, but apparently that proposal was never picked up.

+1 for the question and property naming. I don't know the answer.

Sanne Grinovero
January 22, 2015, 4:03 PM

If the property still is needed, why is it not named "hibernate.search. default|<index_name
.infinispan.write_metadata_async"? The property only makes sense for ISPN, and IMO that should be reflected in the name. That was suggested already back then, but apparently that proposal was never picked up.

Consistency with all other DirectoryProvider settings. Take for example the configuration property:

This is specific for the Master DirectoryProvider. Each DirectoryProvider has different properties, and they are not "grouped" with a prefix. I guess we could argue if that's good or not, but that's how it is.
All the configuration properties in the scope

are global to the Infinispan Service, which is shared across all services.

Is the property "hibernate.search.default.write_metadata_async" established in actually still needed, if it can be set automatically depending on whether the backend is async or not? In other words, is there a need to set it to false for an async backend / set it to true for a sync backend? Otherwise it'd be nice to get rid of one of these knobs people need to reason about.

I agree it would be nice to remove it, but I'm not sure of how safe this whole idea of async writes is. It's conceptually hard to figure out what could go wrong, I think it's premature to enable such an optimisation without having a way for the user to turn it off. Eventually we should do it, but at this level my suggestion is to make sure we try guessing the default, but otherwise respect the user will if he sets it explicitly.

Consider that the issue description states:

It might be ok for synchronous backends too but that still needs extensive testing

If you just disable it for a sync backend and don't give me a way to enable it, I will never be able to test for that.

Gunnar Morling
January 22, 2015, 5:36 PM

I guess we could argue if that's good or not, but that's how it is.

Yes, that was my thinking, having a grouping sub-namespace to such specific settings, e.g. "hibernate.search. myindex.infinispan.write_metadata_async". That'd convey that it an index-scoped setting, specific to ISPN. Anyways, good to know how it is, I guess I had done it as described

I will never be able to test for that.

That convinces me.

Gunnar Morling
January 22, 2015, 5:37 PM

Assignee

Gunnar Morling

Reporter

Sanne Grinovero

Labels

Suitable for new contributors

None

Pull Request

None

Feedback Requested

None

Components

Fix versions

Priority

Major
Configure