Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Avoid creating huge arrays for collectors when a query does not have an upper bound limit

Description

Instead of allocating such arrays, we could introduce an incremental allocation behind the hood.
E.g.: we'd execute the query once to retrieve the top 100, and if there is more, we'd do it again with 200, then 400, then 800, and so on.

Activity

Show:

Yoann Rodière June 11, 2020 at 10:35 AM

By the way we'd have to check this is still relevant in Search 6, too. Since we split the collection in two phases (one phase to collect topDocs + aggregations, the second to retrieve projections from top docs), this will only be a problem if the topDocs or aggregation collectors size their internal structures according to the limit. If they don't, and use automatically increasing container (List, ...), we're fine.

Yoann Rodière June 11, 2020 at 10:18 AM

See org.hibernate.search.query.engine.impl.QueryHits#scoreDoc in Hibernate Search 5.
I'm not entirely sure this is how it works for queries without a limit, but I think it is. Let's check?

Fixed

Details

Assignee

Reporter

Components

Sprint

Fix versions

Priority

Created June 11, 2020 at 10:14 AM
Updated October 8, 2020 at 12:30 PM
Resolved September 15, 2020 at 3:07 PM