Mitigate test failures caused by automatic snapshots of AWS-managed Elasticsearch service

Description

Some time ago, AWS changed the automatic snapshot policy of its managed Elasticsearch services: they are no longer performed on a daily basis, but on a hourly basis.

Whenever we test Hibernate Search against such service, and a snapshot occurs during tests, the tests invariably fail because the indexes cannot be deleted during a snapshot.

This has been making the master builds fail quite frequently over the past few weeks, and it's a real nuisance.

There is no way to change the frequency of automatic snapshots.

We should see how we can change our test suite to mitigate the problem.

One option would be that, each time a test fails, we pause for a time (~1 minute, or as long as necessary for snapshots to complete), and re-run the test. This should only be enabled when testing against an AWS-managed Elasticsearch service.

There are options to retry failed tests in the Maven failsafe plugin, but unfortunately they do not allow to specify a delay.

In JUnit 4, we would be able to implement this behavior, but only by using a custom runner or rule in every single test, which is not acceptable.

In Junit 5, there seem to be several extension-based solutions. I wonder if there is a way to enable extensions by default, without any change to the tests, from the maven-failsafe-plugin configuration or system properties? Better, maybe there's a way to enable extensions even in the JUnit 4 compatibility mode?

Environment

None

Assignee

Yoann Rodière

Reporter

Yoann Rodière

Labels

None

Suitable for new contributors

None

Feedback Requested

None

Components

Priority

Major
Configure