I'm trying to integrate the ML Flow models loading into my project. On ML Flow I have multiple models uploaded and with multiple versions within. So, what I'm trying to do is to retrieve the latest version for every model I have on ML Flow. One way I can do this is to search for the prefix that I named every model, like:
client.searchModelVersions("name LIKE '" + MODEL_PREFIX + "%'");
This will delivers a Page object that will allow me to iterate across the models I have. And with the models in hand I can get the name of each one and retrieve the latest version. Like:
modelVersions
.collect(
Collectors.toMap(
ModelVersion::getName,
modelVersion -> {
List<ModelVersion> versionsForModel =
client.getLatestVersions(modelVersion.getName());
return versionsForModel.get(versionsForModel.size() - 1);
},
(alreadyComputedModel, newModel) -> alreadyComputedModel))
.values()
.stream();
But as you can see this is an inefficient since I have to ask for the ML Flow API multiple times for every model name.
Is there a method/parameters I can use for the ML Flow always delivers me the latest versions of the models in batch?
I was able to resolve this in memory since ML Flow, apparently, doesn't have an endpoint for this.
private Stream<ModelVersion> getLatestVersionsForModels(Stream<ModelVersion> modelVersions) {
return modelVersions
.collect(
Collectors.toMap(
ModelVersion::getName,
modelVersion -> modelVersion,
(alreadyComputedModel, newModel) ->
Integer.parseInt(alreadyComputedModel.getVersion())
> Integer.parseInt(newModel.getVersion())
? alreadyComputedModel
: newModel))
.values()
.stream();