When running our application with Hibernate ORM 4.1.2 and Hibernate search 4.1.0 it very quickly uses all allowed open files. Increasing the limit just extends the time that it takes to consume all. This does not happen when running with Hibernate ORM 3.6 and Hibernate search 3.4.
I have created a small testcase. When I run it here (Linux) I get to about 500 open deleted files (to count I use lsof -p xxx|grep -c deleted) within 10 minutes of starting it. It does not matter what filesystem access type I use (simple, nio, mmap). Mmap just tends to hide the issue better and keeps increasing process's virtual mem size. With nio it's very simple to check how many deleted files are still open. Normally within about 20min the process runs out of it's 1024 allowed open files.
I have tried playing with other search settings but none of them have helped so far.