ignitein-memory-data-grid

Can I access "model" classes inside a ComputeJob?


Is there a way to deserialize a value from an IgniteCache inside a computeJob? When I tried, I got a ClassNotFoundException on the Marshaller.

I could only do it reliably when programming the compute job using BinaryObject (Ex: IgniteCache<BinaryObject, BinaryObject>), or when I installed the jar containing the model on the ignite Nodes (requiring restart of the node).

Right now what I want is for the cluster to be a common pool of resources for computing tasks. In that way, multiple versions of the same app could use the cluster.

Because each app instance would load their own data, I don't really want to have a single version of the "model entity" jar installed on the nodes. Code Deployment using the UriDeploymentSpi would be really amazing.

Is that a possibility or am I using ignite in way that is not intended?

Btw, my compute job looks something like this:

public class FetchAtivoStatsNovoJob implements ComputeJob, IgniteCallable<Map<String, Integer>> {

    @IgniteInstanceResource
    private Ignite ignite;

    private List<Long> filterIds;

    private String ativosNovoCache;

    public FetchAtivoStatsNovoJob(List<Long> filterIds, String ativosNovoCache) {
        this.ativosNovoCache = ativosNovoCache;
        this.filterIds = filterIds;
    }

    @Override
    public Object execute() throws IgniteException {
        try {
            return call();
        } catch (Exception e) {
            throw new IgniteException("Failed to fetch posts.", e);
        }
    }

    @Override
    public Map<String, Integer> call() throws Exception {

        IgniteCache<AtivoKeyNovo, AtivoNovo> cache = ignite.getOrCreateCache(ativosNovoCache);

        ScanQuery<AtivoKeyNovo, AtivoNovo> query = new ScanQuery<>();
        query.setLocal(true);
        query.setFilter((k, ativo) -> filterIds.contains(k.getSubdominioId()));

        Map<String, Integer> result = new HashMap<String, Integer>();

        try (QueryCursor<Cache.Entry<AtivoKeyNovo, AtivoNovo>> cursor = cache.query(query)) {
            for (Cache.Entry<AtivoKeyNovo, AtivoNovo> entry : cursor) {

                AtivoKeyNovo key = entry.getKey();
                AtivoNovo value = entry.getValue();

                // do some calculation, put on the result Map or insert a new record on another IgniteCache
            }
        }
        return result;
    }

    @Override
    public void cancel() {
    }

}

Thanks for any help!


Solution

  • You don't say so, but I assume that you're using peer class loading? As per the documentation:

    The peer class loading functionality does not deploy the key and object classes of the entries stored in caches.

    You could deploy those classes by any other mechanism, as long as they're present before your compute task is executed. Table/class structures tend not to change often, so deploying to the server nodes is pretty common.

    As you suggest, using binary objects directly is also an option.