Following HSEARCH-2406, Byteman rules in tests seem ineffective: the added behaviors do not show up.
To be precise, Byteman does change the bytecode, but modified bytecode execution it fails to retrieve the behavior it was supposed to adopt, and defaults to doing nothing.
The cause seems to be a classloader problem.
When rules are "injected", they are registered in a static variable in the Rule class (Rule.ruleKeyMap), and when bytecode modified by Byteman executes, it tries to retrieve the rule from this same static variable.
But in our case, the Rule class is loaded by different class loader in those two cases, which means rule registration and rule retrieval use two independent Rule.ruleKeyMap static variables, which means that the rule is not found upon modified bytecode execution.
How to reproduce this issue:
Run the whole set of tests in hibernate-search-elasticsearch (running only one test will not reproduce the issue)
Notice how org.hibernate.search.test.query.initandlookup.CriteriaObjectInitializerAndHierarchyInheritanceTest fails due to a byteman rule not being applied
How to observe the issue in Byteman code:
Place breakpoints at the beginning of org.jboss.byteman.rule.Rule.getKey() (rule registration) and org.jboss.byteman.rule.Rule.execute(String, Object, Object) (rule retrieval)
Run the whole set of tests in hibernate-search-elasticsearch (running only one test will not reproduce the issue) in debug mode: mvn clean verify -pl elasticsearch -am -Dmaven.failsafe.debug -Dorg.jboss.byteman.verbose=true
Skip the breakpoints until the execution of org.hibernate.search.test.query.initandlookup.CriteriaObjectInitializerAndHierarchyInheritanceTest (with my setup, there was only one thread suspension for each breakpoint before I got to the test I was interested in)
In org.jboss.byteman.rule.Rule.getKey(), take note of the object id of Rule.class and Rule.ruleKeyMap (showed in the Eclipse debugger for instance)
In org.jboss.byteman.rule.Rule.execute(String, Object, Object), do the same
Notice how both the class and its rule map are different objects
One can also place breakpoints on class loads, and notice that indeed the Rule class is loaded twice: once when Byteman initializes, and once when the transformed bytecode tries to retrieve the actual rule. When everything works well (in Byteman's own unit tests, for instance), only the first class load is executed.
I'm currently following two leads:
1. An issue with test-jars triggering strange classloader behavior => NO, see below
2. An issue with Hibernate Search or ORM doing exotic classloader manipulation
3. Something else?
I'm inclined to think the first lead is invalid, because:
I failed to reproduce the issue independently from Hibernate Search, in a Byteman integration test (using test jars).
The issue does show up when running a single test using byteman; see the commits for
Also, since the issue only shows up when running the whole set of tests, there might be a test that messes with classloaders.
On a related theme, I dived a bit in Byteman's JIRA and found BYTEMAN-59: it seems that special care is taken when Byteman initializes in order to load Byteman classes using the right class loader. And looking at the source code for org.jboss.byteman.agent.Main, indeed there are special classloader manipulations.
Note: there are disabled tests because of this issue; when solving it, please re-activate the tests (org.hibernate.search.elasticsearch.test.DefaultElasticsearchClientFactoryTest.discoveryScheme() in particular)