We're updating the issue view to help you get more done. 

Inconsistent performance between hibernate search and pure lucene access

Description

I have a simple index that contains:

  • id (pk of the entity)

  • keywords (a list of tokens)

The index contains 100.000 objects and the keywords field has 2 tokens from a list of 40 different values

What I want to do is retrieve all the IDs that matches a given lucene query on the keywords. So for that I'm doing something like:

FullTextSession fullTextSession = Search.createFullTextSession(session);
QueryParser parser = new QueryParser("keywords", luceneAnalyzer);
org.apache.lucene.search.Query hibernateQuery = parser.parse("foo AND bar");
FullTextQuery fullTextQuery = fullTextSession.createFullTextQuery(hibernateQuery, target);
fullTextQuery.setProjection("id");
fullTextQuery.setResultTransformer(resultTransformer);
Iterator it = fullTextQuery.iterate();

Where ResultTransformer is

private static class FirstObjectResultTransformer implements ResultTransformer {

public Object transformTuple(Object[] objects, String[] strings) {
return objects[0];
}

public List transformList(List list) {
return list;
}
}

If I do a load test with a single thread, the execution time of my lucene query is around 200 msec. If I do a load test with 10 threads, the execution time is 2 sec (per user!). If I run the profiler on the service, I see lots of deadocks on SegmentReader.

Switching to a "non-shared" strategy removes the deadlocks but it's still slow (1.5 sec).

Now, If I execute the same query on the same index and the same host with only the lucene API, the query takes around 100msec with 10 concurrent users. I tried to use the lucene API from Hibernate Search but it did not change anything.

What am I missing? Attached the profiling result.

Environment

Linux - Hibernate 3.2.6, Hibernate Annotations 3..3.1 - Lucene 2.3.1

Status

Assignee

Unassigned

Reporter

Stéphane Nicoll

Components

Fix versions

Affects versions

3.0.1.GA

Priority

Critical