Allow "caching" predicates/sorts/projections for re-use

Description

The idea, discussed during our last face-to-face meeting, was to be able to re-use search predicates or sorts because they may be expensive to build. That means returning SearchPredicate (or SearchSort, or SearchProjection) objects to the user, and allowing passing them to the query DSL later.

Re-use would be allowed:

  • between two search queries of the same search target

  • maybe, between two compatible search targets

What "compatible" means exactly has yet to be defined. Two identical search targets (same targeted indexes) are compatible for sure, so I think we should start with this definition, and extend it later if need be.
We should also decide if we want to perform runtime checks when re-using

A preliminary implementation is already present in the proof-of-concept. However:

  • the fact that predicates/sorts/projections can be re-used is only partially tested. We'd have to check that the tests are sufficient. See org.hibernate.search.integrationtest.backend.tck.search.sort.SearchSortIT#lambda_caching, org.hibernate.search.integrationtest.backend.tck.search.predicate.SearchPredicateIT#match_caching_root, org.hibernate.search.integrationtest.backend.tck.search.predicate.SearchPredicateIT#match_caching_nonRoot.

  • we do not perform any runtime checks to validate that a given predicate/sort can be used in a given search query (that the search target it originated from is compatible with the one the query originated from)

Environment

None

Activity

Show:
Fabio Massimo Ercoli
June 10, 2019, 4:00 PM
Edited

Maybe the concept of scope compatibility is similar to the concept of index compatibility for a given (predicate | sort | projection).

Maybe we could add some tests, such as:

Yoann Rodière
June 11, 2019, 7:28 AM

:

Maybe the concept of scope compatibility is similar to the concept of index compatibility

The example you gave should actually fail. You created predicates taking into account metadata from one index, and later created a query using these same predicates, but targeting a completely different index. This index's metadata may be completely different, with some fields having a different type.

The kind of compatibility checks I'm talking about are only similar on a very high level. "Compatibility" may not be the right word, actually. "Consistency" is probably a better fit. We want to check that, if a predicate was created targeting the index "Author", it cannot be used in a query targeting the index "Book".

We could allow more things in the future, of course, such as the example you're giving. It would essentially amount to introducing some kind of duck-typing in SearchPredicate: if the predicate targets field name of index author, and index book declares a field name with similar characteristics, then we can use the predicate on index book too. We could do it, but let's not bother with that right now.

The kind of (crude) consistency check I was thinking of could simply be implemented in org.hibernate.search.engine.search.predicate.spi.SearchPredicateBuilderFactory#toImplementation for predicates, for example. Let's imagine you store the set of targeted indexes in SearchPredicate: you would only need to check this set of targeted indexes is the same as the one targeted by the SearchPredicateBuilderFactory.

Maybe we could add some tests, such as: [...]

Tests may be similar, yes. However:

  1. I'm not sure testing all field types is really necessary, but if it's not too hard, I agree it would be nice.

  2. What we want to be sure about here is that incompatibility is detected. You example only tries to check that compatibility is detected, which is fine, but is probably already tested in multiple parts of the code. What's most important to test is that exceptions occur when users do things wrong.
    In short the test would create two slightly different scopes, and check that using a predicate(/sort/projection) created from one scope on a query created from the other scope will fail.

Fabio Massimo Ercoli
June 11, 2019, 9:27 AM
Edited

This index's metadata may be completely different, with some fields having a different type.

Yes, but not in my example

Yes. I was trying to make a step further on top of just comparing the scopes using their index sets.
Not a problem to implement that crud consistency right now

Assignee

Fabio Massimo Ercoli

Reporter

Yoann Rodière

Labels

None

Suitable for new contributors

None

Feedback Requested

None

Components

Fix versions

Priority

Major
Configure