Add match all terms option when matching in the DSL API

Description

Hi,

A bit of context:

We are early adopters of Hibernate Search and we have very few problems with it (except the @IndexEmbedded problem we helped to fix in 3.4.1, no problem so far).

When the DSL API was introduced, I tried it and I found the problem I describe below. I decided to use the QueryParser API (and the MultiFieldQueryParser API) as a workaround. The fact is that:

  • we use Hibernate Search in every application we have, now;

  • the DSL API is really nice and, as we introduced QueryDSL in our application, we now use a lot of DSL like API and I would like to be able to use Hibernate Search API too;

  • I thought it was a deliberate choice but, recently I found an example so weird, I can't think it's the wanted behaviour.

So this problem isn't new and it exists since the first version of the DSL API.

Now, the description of the problem:

  • we use the following analyzer to index a field in our entity:

  • the content of the field is something like XXXX-AAAA-HAGYU-19910

  • if you search for an exact match "XXXX-AAAA-HAGYU-19910" with the QueryParser, you have a few results: namely the results which have all the different parts (XXXX, AAAA, HAGYU and 19910) in any order. That's the behaviour I expect considering my analyzer.

  • if you search using the DSL API, you have ALL the results containing at least ONE token so A LOT of results in our case.

My expectation is that the DSL API should work as the Lucene parser works and it should return the same results.

The problem is that in ConnectedMultiFieldsTermQueryBuilder, we don't use the QueryParser to build the Lucene query but a getAllTermsFromText() method which uses the analyzer to get all the terms and from that we build a OR query.

So when I search for XXXX-AAAA-HAGYU-19910, the DSL API searches for "XXXX" OR "AAAA" OR "HAGYU" OR "19910".

I really think it's a mistake and that we should use the *QueryParser API to build the Lucene Query and have the correct behaviour.

If needed, I can provide any further information and/or a test case. I just want to be sure you consider it a bug before working further on this. Otherwise I'll stick to using the *QueryParser API.

Thanks for your feedback.

Environment

None

Activity

Show:
Yoann Rodière
July 1, 2020, 11:02 AM

I guess the use case is already addressed by SimpleQueryString for the most part, but there are still details that are not...

In particular I know the simple query string can apply an AND between two clauses (separated by a space), but I'm not sure what happens when a clause is tokenized into two separate terms (e.g. wi-fi). It's quite possible that we end up with an OR between the two terms, and then we're back to the same old problem.

Elasticsearch has quite advanced settings to define how to behave when trying to match multiple terms (what this ticket is about) or multiple fields (), and I think it would be worth having a look.

There are also minor differences between the match predicate and the simple query string predicate that could make the match predicate preferable in some cases: no support for DSL converters, no fuzzy option that can be set by the developer, ... All very minor, but they exist.

Anyway, this is definitely low-priority, at least for Search 6.0.0.Final. I might end up postponing to 6.1 if we don't have enough time.

Guillaume Smet
June 30, 2020, 5:00 PM

For me this one can probably be closed. The simple query string work was a direct follow up of that one.

Guillaume Smet
August 8, 2013, 2:58 PM
Sanne Grinovero
April 16, 2012, 12:21 PM

Hi Guillaume,
thanks! We can split tasks if you happen to find the time to contribute your draft, as for my mind the most valuable contribution is the tests and the requirements expressed in Java. Otherwise looking forward to you finish your other project.

Guillaume Smet
April 16, 2012, 11:44 AM

Hi Sanne,

The Hibernate Search part is easy - and I already have it somewhere - but the Lucene one isn't that easy, depending on your analyzers.

I had to put it on hold for the time being as I have to concentrate my efforts on another project. I'll be back on it as soon as I've finished my other project (should be end of june) if noone beats me to it.


Guillaume

Assignee

Unassigned

Reporter

Guillaume Smet

Labels

None

Suitable for new contributors

None

Pull Request

None

Feedback Requested

None

Components

Fix versions

Priority

Critical