Prototype capability to delegate both indexing and search to an Elasticsearch service

Description

With the introduction of IndexManager in version 4.0 we can now delegate index writes and updates to an external engine.

We still require exposing an IndexReader to the query engine, it would be nice if we could delegate the query execution as well to a remote engine like Solr or ElasticSearch (not thinking about anything not Lucene oriented).

This would also require a way to properly remote the Lucene Query, and likely a way to export our mapping to the alternative indexing engine (for example analyzer definitions and field storage options)

I described some ideas on the forum:
https://forum.hibernate.org/viewtopic.php?f=9&t=1023888

Activity

Sanne GrinoveroAugust 6, 2015 at 8:11 AM

By skimming through the Jest documentation is looks like that to perform many useful operation it requires to pull in the ES client as well. Specifically, the ES query builder is very polished and people have been asking for all the capabilities it provides; We might want to wrap that in a custom API but that would force us to constantly keep up with new features being added to ES (not to mention users needing to learn two APIs).

I haven't decided if we should expose those APIs, I was hoping that we would have a better picture after having a POC to play with.. but to get such a prototype quickly I'd avoid coding all of the client side when you can expose the ES query builders to get a good starting point.

ES / master is using Apache Lucene 5.2.1.Final: you could simply build a snapshot and avoid the dependency problem.

Gunnar MorlingAugust 6, 2015 at 7:16 AM

I am going to work on this with the goal of providing ElasticSearch integration. Some thoughts/questions:

  1. Which client to use?

  • ES default

    • Pro: Offers all functionality provided by latest ES

    • Con: Ties to a specific ES and Lucene version which are pulled into the application classpath; The latter is 4.x atm. Makes it a non-viable option really

  • Jest

    • Pro: Works with several ES versions; Does not pull in Lucene libraries into client

    • Con: Quite dependency-heavy, specifically Guava is pulled in

  • Raw REST/JSON client, e.g. RESTEasy

    • Pro: Probably less dependencies

    • Con: Quite low-level API, not specific to ES domain

For my first steps I will be using Jest.

Fixed

Details

Assignee

Reporter

Components

Fix versions

Priority

Created September 18, 2012 at 12:45 PM
Updated February 16, 2016 at 5:18 PM
Resolved December 8, 2015 at 5:13 PM