Investigate if it's possible to apply Remote Task Execution to Infinispan Remote dialect

Description

On OGM Infinispan Remote dialect check the feasibility of executing batch operations on the remote server using the Infinispan feature: Remote Task Execution.

We've discovered an actual limitation on Remote Task Execution, see https://issues.jboss.org/browse/ISPN-8020.

The issue talk about Script Task, not about Remote Task, but the problem it would seem to be the same, also on Remote Task Execution: it is not possible using Protostream marshaller for Remote Task invocation.

The workaround suggested: using 2 RemoteCacheManager (one for cache with Protostream marshaller, another one for task only with default marshaller) seems to work.
Compatibility mode of course will be mandatory in this case, to allow different marshaller to operate on the same caches.

Environment

None

Activity

Show:
Fabio Massimo Ercoli
February 8, 2018, 4:43 PM

We've discovered an actual limitation on Remote Task Execution, see https://issues.jboss.org/browse/ISPN-8020.

The issue talk about Script Task, not about Remote Task, but the problem it would seem to be the same, also on Remote Task Execution: it is not possible using Protostream marshaller for Remote Task invocation.

The workaround suggested: using 2 RemoteCacheManager (one for cache with Protostream marshaller, another one for task only with default marshaller) seems to work.
Compatibility mode of course will be mandatory in this case, to allow different marshaller to operate on the same caches.

Fabio Massimo Ercoli
February 8, 2018, 4:57 PM

Another point that it is necessary to deep: if the actual Entities and Associations caches use the Entity model? It would seem not. Rather, generic OGM model POJOs are used, both for the keys and for the values.

This model must be available server side, in order to work on the caches and execute the batch of operations.

If this were not confirmed, and there were also the Entities serialized in the caches, the feature would not be applicable to the use case, because OGM is a framework and the Entities used by the client are obviously not known a priori.

Fabio Massimo Ercoli
February 8, 2018, 5:33 PM
Fabio Massimo Ercoli
February 16, 2018, 2:14 PM
Edited

Report of the Activity

We investigated on the possibility of applying the Infinispan Remote Task Execution to Hibernate OGM Infinispan Remote dialect.
See Remote Task Execution.
The main benefit would be to limit the number of remote calls made by the client to the Infinispan remote servers.
A batch consisting of n operations could be performed as a single call to a possible future Hibernate OGM custom Remote Task.
A nice side effect could be also a mitigation of the current not-transactional behaviour of the Hot Rod Infinispan client.
We present here the reports of the activity, starting by saying that the solution seems to be practicable.

Compatibility mode

Currently it is not possible using Proto Stream marshaller for Remote Task invocation, see ISPN-8020.
We want to leave the content of the cache generated by Hibernate OGM in Protobuf Encoding format. See Protobuf Encoding.
The workaround here could be to define a second Remote Cache Manager using the standard marshaller only for task invocation, or invoke the tasks passing parameters in byte array format.
We need also to allowing the Remote Task Execution to work on the access directly cache entries enabling the Compatibility Mode.
Hibernate OGM already store the the Protobuf generated schema in PROTOBUF_METADATA_CACHE, in this way the schema will be available also for the server marshalling.

Marshalling issue

For Hibernate OGM we can’t use the directly the ProtoStreamMarshaller provided by Infinispan, because we cannot define a MessageMarshaller<T> for every protobuf message. The messages depends on client Entities and they are not known a priori.
The issue is already solved for client-defined marshallers in current implementation,
so we we could potentially readapt the same solution to define Entity agnostic server-defined marshallers.
Server-side marshallers can be installed using a dedicated server task.
It remains to be seen if it is possible to define them with an infinispan extension, that could be more proper solution for life-cycle reason.

Executing the Batch

Parameters to pass to remote task must be Serializable and Entity independent (general purpose objects).
At this time we can imagine to create three types of serializable of operation: PutOperation, PutIfAbsentOperation, DeleteOperation.
Each of these supplying the cache name and serializable tuple format (ex: Map<String,Serializable>).

Nested Types

Luckily protobuf nested types are never used in model generated by Hibernate OGM.
See “nested” Protocol Buffers message.
Even embeddable objects are currently created on separate caches, and correlated to the main Entity using association tuples.
So the parameter objects, we mentioned above, could be more than enough to implement all cases.

Developed Test / Demo Projects

For this issue we developed two test / demo projects:

Fixed

Assignee

Fabio Massimo Ercoli

Reporter

Fabio Massimo Ercoli

Labels

None

Feedback Requested

None

Feedback Requested By

None

backPortable

None

Suitable for new contributors

None

Pull Request

None

backportDecision

None

backportReEvaluate

None

Components

Priority

Major