In commit e62b27eeadb73c3c867866f4aeaed14fb77baa98 I removed a check in JGroupsMasterMessageListener about making sure that we actually are the master before applying works locally. It was necessary to remove the check because in the case of dynamic sharding, we cannot do this check easily.
It seemed to make sense because the check seemed to be only about handling master re-election. I assumed that works were being sent to the master only. I was wrong: JGroups messages are broadcasted, and each node is supposed to filter them on its own.
So the current behavior, without the check, results in every node applying every work, then figuring out it's not the master, re-sending the message, and so on: we basically create an infinitely growing message stack. Fortunately we don't go that far, because something seems to block message sending at some point, but still, this doesn't work.
Thus we need to restore this check, and find another way to handle dynamic sharding properly.