Provide an option to wait for Elasticsearch status yellow rather than green

Description

Waiting for green is a bit aggressive especially for demos and other code level work. It requires to manually change in the configuration the replica numbers.
was mentioning that waiting for yellow might be good enough.

We probably should make it an option.

Environment

None

Activity

Show:
Sanne Grinovero
April 18, 2016, 4:45 PM

Would it help if we made the timeout message better?

I'm not sure why people would prefer to set an Hibernate Search specific property to workaround this, rather than fix the ES configuration to run on one node. Either way, a setting needs to be changed...

Guillaume Smet
April 18, 2016, 6:01 PM

FWIW, it's failing with the default Elasticsearch install (i.e. download, untar and run bin/elasticsearch). We need to change it to yellow to get it work.

Sanne Grinovero
April 18, 2016, 7:33 PM

FWIW, it's failing with the default Elasticsearch install

I'm aware: I am using that too, and figured that the Maven plugin reconfigures ES to allow the singleton cluster when I had to debug it.

My doubt is just that either way, the user will have an error, and then will either already know how to reconfigure ES or google for it, and in each of these outcomes you'll likely end up reconfiguring ES.

So assuming that we fix I suspect having an ad-hoc option in Search is a bit redundant.

N.B. for a quick demo you can also start ES from our project invoking:
mvn elasticsearch:run
This must be run from the /elasticsearch module. CTRL+C to terminate it.

Gunnar Morling
April 19, 2016, 9:36 AM

Think about the case where you want to have 3 replicas of an index, but only 2 are available (as one of your nodes broke down). Do you want HSEARCH to do its thing (all requests can be satisfied, only the replication rules are not satisfied) or not? I.e. do you want HSEARCH to send writes to the index also if the replica count currently is not what you want, or do you want to be extra safe and wait for the full number of replicas. To distinguish these two, a setting in HSEARCH would make sense.

I feel, working with yellow is good in most cases, esp. since the cluster can degrade from green to yellow any time at runtime, and we won't check or detect this after start-up. So I suggest to just change it so we only wait for yellow (log a warning if it's not green), and if there is user demand, we can still make it configurable down the road. But then we'd also have to think about what to do about degredation at runtime, i.e. should we regularly check and log errors then?

Thoughts?

Sanne Grinovero
April 19, 2016, 5:43 PM

No I disagree on considering "yellow" a sane default. If the cluster is yellow it's very possible that it's still starting and we should give it a chance to form... yes this is an edge case but for this specific edge you won't have to wait for many seconds, so it's just about being "user friendly" for people starting up a bunch of VMs (i.e. clouds).
Yet if the cluster is "yellow" because something is really wrong, then allowing to write to it is as good as the unsafe-by-default on MongoDB which we've laughed about. I do agree that the user might be happy with writing on one replica only - definitely a valid scenario - but he/she has plenty of configuration options to make this explicit in the Elasticsearch configuration (i.e. ES would report "green" in a more relaxed configuration).

+1 to regularly check for the cluster health. I'm sure I already mentioned that this would be needed eventually.. it's way more complex though, as we've already started so what are we going to do with the changes? Sending runtime exceptions up seems not very nice.

My question still stands: why should I prefer reconfiguring an Hibernate Search property to "write on yellow" rather than reconfigure Elasticsearch?
For example, if all you want is to start a demo you probably want to disable discovery altogether.. or you risk connecting to someone else on the conference wifi, or countless other issues.. not least, it will start faster.

This is a good "demo configuration" which I tuned myself: https://github.com/hibernate/hibernate-search/blob/master/elasticsearch/elasticsearchconfiguration/elasticsearch.yml

Assignee

Gunnar Morling

Reporter

Emmanuel Bernard

Labels

None

Suitable for new contributors

Yes, likely

Feedback Requested

None

Components

Fix versions

Affects versions

Priority

Minor
Configure