Restore support for full-text filters

Description

The idea is to allow users to declare filters as part of the mapping, and then allow to enable filters on an already-built query to add predicates. Users can pass parameters to the filter when enabling it.

The filters could be as simple as callbacks taking some context as a parameter and adding predicates to it using a DSL.

Note we must also allow filters to specify routing keys (“shard identifiers”) to implement the equivalent of org.hibernate.search.filter.ShardSensitiveOnlyFilter; see also

We should think about the caching feature we have in Search 5: maybe it requires some backend-specific code, maybe it requires filters to be modeled as cacheable objects, ... and then maybe caching could be implemented exclusively at the backend level and potentially apply to every query, not just filters. See org.hibernate.search.util.impl.FilterCacheModeTypeHelper and its uses in particular. See also this comment by Sanne which seems to suggest dropping support for filter caching would not be a big deal.

For testing, see org/hibernate/search/v6poc/integrationtest/mapper/pojo/JavaBeanProgrammaticMappingRoutingIT.java:147

Environment

None

Activity

Show:
Waldemar Kłaczyński
March 28, 2020, 11:56 PM

As for the caching of queries, I rather did not mean extracting entities from the cache. But I just mean a similar caching mechanism for lucene queries. Similar to entities with JPA queries, you can remember the partial results of lucene queries on two levels. Having a list of filter parameters you could remember the result for the same parameters in the L1 cache and in L2.

Waldemar Kłaczyński
March 29, 2020, 12:00 AM
Edited

Anyway, in the previous version this was how it worked, only there was one level cache. I suggest opening this functionality by adding an additional buffer for the transaction in the L1 cache, fast operational memory. And as before in the L2 cache, e.g. infinispan. Thanks to this, high speed of repetitive queries in one transaction could be achieved. And as previously repeated in separate orders, for filters having the same query parameters.

Waldemar Kłaczyński
March 30, 2020, 6:07 AM
Edited

Can build InfinispanQueryCache by extending LRUQueryCache. This would make it easier to remember queries in infinispan as L2. Inside the buffer in memory just like LRUQueryCache as L1 caches.

Wrap the query from the filter in FullTextFilterQuery and calculate the hash from the filter parameters. Thanks to this, unique keys are built for different filters with different keys calculated on the basis of parameters that will be properly treated by org.apache.lucene.search.QueryCache.

 

Waldemar Kłaczyński
March 30, 2020, 9:23 AM

By default, LRUQueryCache is loaded statically. It seems to me that in some cases this may cause erroneous behavior on the JBoss and Wildflay server.

Waldemar Kłaczyński
April 2, 2020, 2:56 PM

I know you can do it differently, but then it complicates the code. The strength of JPA, CDI, etc ... is that it hides a lot of code from subsequent performers. In my case, I hid all permission support under the annotation "@PermissionsBinding", and I have a lot of entities that need to be treated like this. I also need to solve other problems similarly. So that the next programmers do not have to penetrate how it works to use. They will simply use one annotation and have all the permissions functionality in search, etc ...

Assignee

Unassigned

Reporter

Yoann Rodière

Labels

None

Suitable for new contributors

None

Pull Request

None

Feedback Requested

None

Fix versions

Priority

Major
Configure