Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Improve support for @BatchSize

Description

Specifically, there are 2 aspects of batch fetch support that can be improved -

  1. Cache generated SQL AST and friends when possible

  2. Implement support for different strategies for handling batch-size in different scenarios - ala legacy BatchFetchStrategy

If the thing being batch fetched has a single-column key, offer support for loading based on a single SQL ARRAY parameter. Otherwise, fallback to the “padded” strategy (though potentially partitioning the fetching into smaller chunks depending on Dialect#getDefaultBatchLoadSizingStrategy.

For the ARRAY strategy, look at supporting that for multi-load also.

Activity

Show:

Steve Ebersole April 6, 2023 at 12:30 PM

There is a potential problem with padding though, depending. Though to a degree it is really a general question about “larger” sizes. Imagine you have:

@BatchSize(size=5000)

Creating a SQL with 5000 (times the number of columns in the key!!) parameters is not feasible - partitioned is actually better there at some point.

Christian Beikov April 6, 2023 at 6:41 AM

IMO the padding and array approaches should be enough.

Steve Ebersole April 6, 2023 at 12:10 AM

I just implemented the following strategies:

  1. if hibernate.query.in_clause_parameter_padding is enabled, a modified version of the legacy “coin change algorithm” approach is used - the modification is to use padding within each partition - SingleIdEntityPartitionedBatchLoader

  2. Otherwise, generally, a “static SQL” approach is used where we use a single SQL with batch-size number of parameters, with null padding for “empty slots” - SingleIdEntityPaddedBatchLoader

  3. Also implemented a dynamic approach where we create SQL on the fly each time for the specific number of ids.

also suggested possibly using an array parameter which I also like.

I think we should limit the options here a bit though. Does anyone see the benefit of the dynamic approach? To me, “static SQL” and array-parameter would be enough; maybe with a fallback to partitioned if the “static SQL” batch-size exceeds a certain limit?

Fixed

Details

Assignee

Reporter

Fix versions

Priority

Created April 5, 2023 at 4:48 PM
Updated April 28, 2023 at 9:36 AM
Resolved April 26, 2023 at 7:01 PM