Include an IndexDeletionPolicy which deletes older commits asynchronously

Description

When using an Infinispan Directory the deletion of old segments is one of the slowest operations, as it requires several network RPCs to make sure that each file is not being used anymore.
But also commits are a significantly sensitive operation on Infinispan as it scales reads but not writes to the index, and each such commit must be applied sequentially.

Clearly the RPCs being triggered for deletion purposes of stale segments stay in the way of other sequential operations which are actually useful for indexing progress, so we expect it to be effective to delegate the operations related to stale segments cleanup to a background executor.

Such an IndexDeletionPolicy is hence expected to be particularly suited for storage on Infinispan, but isn't necessarily coupled to Infinispan usage only.

Environment

None

Status

Assignee

Gustavo Fernandes

Reporter

Sanne Grinovero

Labels

Suitable for new contributors

None

Pull Request

None

Feedback Requested

None

Components

Priority

Critical
Configure