Provide an option to wait for Elasticsearch status yellow rather than green

Description

Waiting for green is a bit aggressive especially for demos and other code level work. It requires to manually change in the configuration the replica numbers.
was mentioning that waiting for yellow might be good enough.

We probably should make it an option.

Activity

Show:

Sanne GrinoveroApril 19, 2016 at 5:43 PM

No I disagree on considering "yellow" a sane default. If the cluster is yellow it's very possible that it's still starting and we should give it a chance to form... yes this is an edge case but for this specific edge you won't have to wait for many seconds, so it's just about being "user friendly" for people starting up a bunch of VMs (i.e. clouds).
Yet if the cluster is "yellow" because something is really wrong, then allowing to write to it is as good as the unsafe-by-default on MongoDB which we've laughed about. I do agree that the user might be happy with writing on one replica only - definitely a valid scenario - but he/she has plenty of configuration options to make this explicit in the Elasticsearch configuration (i.e. ES would report "green" in a more relaxed configuration).

+1 to regularly check for the cluster health. I'm sure I already mentioned that this would be needed eventually.. it's way more complex though, as we've already started so what are we going to do with the changes? Sending runtime exceptions up seems not very nice.

My question still stands: why should I prefer reconfiguring an Hibernate Search property to "write on yellow" rather than reconfigure Elasticsearch?
For example, if all you want is to start a demo you probably want to disable discovery altogether.. or you risk connecting to someone else on the conference wifi, or countless other issues.. not least, it will start faster.

This is a good "demo configuration" which I tuned myself: https://github.com/hibernate/hibernate-search/blob/master/elasticsearch/elasticsearchconfiguration/elasticsearch.yml

Gunnar MorlingApril 19, 2016 at 9:36 AM

Think about the case where you want to have 3 replicas of an index, but only 2 are available (as one of your nodes broke down). Do you want HSEARCH to do its thing (all requests can be satisfied, only the replication rules are not satisfied) or not? I.e. do you want HSEARCH to send writes to the index also if the replica count currently is not what you want, or do you want to be extra safe and wait for the full number of replicas. To distinguish these two, a setting in HSEARCH would make sense.

I feel, working with yellow is good in most cases, esp. since the cluster can degrade from green to yellow any time at runtime, and we won't check or detect this after start-up. So I suggest to just change it so we only wait for yellow (log a warning if it's not green), and if there is user demand, we can still make it configurable down the road. But then we'd also have to think about what to do about degredation at runtime, i.e. should we regularly check and log errors then?

Thoughts?

Sanne GrinoveroApril 18, 2016 at 7:33 PM

FWIW, it's failing with the default Elasticsearch install

I'm aware: I am using that too, and figured that the Maven plugin reconfigures ES to allow the singleton cluster when I had to debug it.

My doubt is just that either way, the user will have an error, and then will either already know how to reconfigure ES or google for it, and in each of these outcomes you'll likely end up reconfiguring ES.

So assuming that we fix I suspect having an ad-hoc option in Search is a bit redundant.

N.B. for a quick demo you can also start ES from our project invoking:
mvn elasticsearch:run
This must be run from the /elasticsearch module. CTRL+C to terminate it.

Guillaume SmetApril 18, 2016 at 6:01 PM

FWIW, it's failing with the default Elasticsearch install (i.e. download, untar and run bin/elasticsearch). We need to change it to yellow to get it work.

Sanne GrinoveroApril 18, 2016 at 4:45 PM

Would it help if we made the timeout message better?

I'm not sure why people would prefer to set an Hibernate Search specific property to workaround this, rather than fix the ES configuration to run on one node. Either way, a setting needs to be changed...

Fixed

Details

Assignee

Reporter

Components

Fix versions

Priority

Created April 12, 2016 at 3:27 PM
Updated May 21, 2016 at 6:43 PM
Resolved April 21, 2016 at 10:52 AM