InsertActionSorter leads to data loss or duplicate keys

Description

ActionQueue.InsertActionSorter leads to data loss and/or duplicate inserts. After sorting the insert actions, it can happen that some actions are missing from the list, and that other actions are duplicated, because the code sorting the batches has a bug which causes batches to get duplicated or lost in the latestBranches list.

I think the problem is in the code swapping the batches in InsertActionSorter#sort:

1 2 3 4 5 6 7 8 for ( int j = i - 1; j >= 0; j-- ) { BatchIdentifier prevBatchIdentifier = latestBatches.get( j ); if(prevBatchIdentifier.getParentEntityNames().contains( entityName )) { latestBatches.remove( i ); latestBatches.add( j, batchIdentifier ); } }

If this gets executed more than once, it removes a preceding batch but inserts the same new batch every time, instead of the batch it removed.

For example, if we have start with batches A, B and C

1 2 3 ABC (batchIdentifier=C) i = 2, j = 1 ABC ==> remove C at 2 ==> AB ==> add C at 1 ==> ACB i = 2, j = 0 ACB ==> remove B at 2 ==> AC ==> add C at 0 ==> CAC

This results in batch B being dropped and batch C being duplicated, and this is what happened to us in a production environment.

A workaround is to turn off sorting of inserts.

Environment

None

Status

Assignee

Unassigned

Reporter

JochenK

Fix versions

None

Labels

None

backPortable

None

Suitable for new contributors

None

Requires Release Note

Affirmative

Pull Request

None

backportDecision

None

Components

Affects versions

5.2.10

Priority

Critical
Configure