Display "instant speed" on top of overall speed in the mass indexer
Description
Activity
Show:
Fixed
Details
Details
Assignee
Reporter

Components
Sprint
None
Fix versions
Priority
Created February 14, 2022 at 8:03 AM
Updated July 4, 2022 at 7:01 AM
Resolved May 10, 2022 at 10:37 AM
Right now, the mass indexer displays the overall indexing speed (total indexed / total time) as the mass indexing speed in the logs. This has the major downside of allowing extreme data points (like the very fast indexing at start) affect the displayed speed for a very long time.
For example if we index 1000 docs in 1s at the start, then 50 document per second for the rest of the process, we’ll display speeds much larger than 50doc/s for a very long time after this initial spike in indexing speed.
I would argue it’s more useful to display a rolling average, or the instant speed. Let’s try to display that, maybe next to the overall speed? E.g.
Indexing speed: 22.827003 documents/second [current] / 28.827003 documents/second [since start]