Sorts on multi-valued fields

Description

In 5, we allow multi-valued sort fields at indexing time, because we don’t know those fields will be multi-valued. But we don’t have any way to sort on such fields.

We need to either:

Inspect metadata at query time to prevent sorts on multi-valued fields and throw an explicit exception.
OR provide ways to sort on multi-valued fields: allow to specify “sort on the max for each document” or similar.

For reference:

Solr forces users to declare multi-valued fields in the schema
Elasticsearch does not
Both allow to sort on multi-valued fields (Solr since 5.3: https://lucidworks.com/2015/09/10/minmax-on-multivalued-field/)

Environment

None

Activity

Show:

Waldemar Kłaczyński

March 9, 2020, 10:11 PM

Copy link to comment

Maybe to speed up sorting, you can save in the field definition how it should behave when storing values. So eventually he would remember only one field, max, min, avg, etc ... Later sorting would consist of reading only one field. It would be a good solution in a flat model. This would speed up document search.

Indexing the field in such a way would prevent you from choosing a different aggregation method, but it is not always necessary for everyone. So the user would decide between search performance and subsequent dynamic search capability.

The user would have to choose whether to remember the calculated value or to calculate it during sorting.

As for calculating values, median counting for nested multiple values is inefficient. This requires creating an array with the number of nested value elements in order to sort it. So calculating values when creating an index would be the solution here. Some functions, especially those that pull values up to aggregation, can be simplified for multiple nested values. So that he doesn't use ArrayList.

Yoann Rodière

March 10, 2020, 7:29 AM

Copy link to comment

Maybe we could. However:

That'll have to be in another ticket, since it's not strictly required for multi-valued support for sorts.
I'm not sure it's worth our time considering there it can be achieved on the user side with a custom bridge that simply does the average before passing the value to Hibernate Search. There are plenty of other features that are missing but do not have such a workaround.
I'm still debugging and testing the initial patch adding multi-value support for sorts. Better not touch that code until the initial patch is merged.