I want to download the best performing model for a certain ClearlML project. I have the following content in my ClearML experiment platform:
According to: https://clear.ml/docs/latest/docs/clearml_sdk/model_sdk#querying-models I can get a list of models for a specific project:
model_list = Model.query_models(
# Only models from `examples` project
project_name='YOLOv5',
# Only models with input name
model_name=None,
# Only models with `demo` tag but without `TF` tag
tags=['demo', '-TF'],
# If `True`, only published models
only_published=False,
# If `True`, include archived models
include_archived=True,
# Maximum number of models returned
max_results=5
)
print(model_list)
Which prints:
[<clearml.model.Model object at 0x7fefbaf22130>, <clearml.model.Model object at 0x7fefbaf22340>]
So I can run:
model_list[0].get_local_copy()
and get this specific model. But how do I download the best performing one for this project on a certain metric (in this case mAP_0.5:0.95 MAX)?
I ended up doing the following:
try:
import clearml
from clearml import Dataset, Task, Model, OutputModel
assert hasattr(clearml, '__version__') # verify package import not local dir
except (ImportError, AssertionError):
clearml = None
tasks = Task.get_tasks(project_name='YOLOv5', task_name='exp', task_filter={'status': ['completed']})
results = {}
best_task = None
for task in tasks:
results[task.id] = task.get_last_scalar_metrics()['metrics']['mAP_0.5:0.95']['max']
best_model_task_id = max(results, key=results.get)
model_list = Task.get_task(best_model_task_id).get_models()
dest = model_list['output'][0].get_local_copy()
print('Saved model at:', dest)