I have a pytorch face detection model that returns bounding boxes and a few facial landmarks as a dictionary. The bounding boxes and the landmarks are pytorch tensors that where moved to the GPU. When I print the dictionary, it looks for example like this:
{'rects': tensor([[405.5453, 126.3100, 646.7983, 454.6852]], device='cuda:0'), 'points': tensor([[[477.5181, 247.7193],
[591.4567, 247.1899],
[544.2579, 328.7857],
[494.3871, 371.9922],
[583.4272, 370.3429]]], device='cuda:0'), 'scores': tensor([0.9999], device='cuda:0'), 'image_ids': tensor([0], device='cuda:0').
Does the dictionary lie on the CPU and only link to the GPU where the tensors are actually on? Or is the dictionary also loaded on the GPU now? Also, if the dictionary contained one list and one tensor like this:
{'rects': [405.5453, 126.3100, 646.7983, 454.6852] , 'points': tensor([[[477.5181, 247.7193],
[591.4567, 247.1899],
[544.2579, 328.7857],
[494.3871, 371.9922],
[583.4272, 370.3429]]], device='cuda:0')}.
Where is the dictionary then located?
Maybe I'm just confused on what device="cuda:0" actually means, so if someone could answer I would highly appreciate it!
The dictionary is just a python object, it has no device associated with it.
The dictionary keys and values are pointers to other python objects. The dictionary itself has no idea what a device is and does not interact with that aspect of pytorch tensors. The dictionary only knows the pointers for the key/value objects. The device(s) in question are completely independent from the dictionary.
A pytorch tensor is an object that acts as a wrapper around some data. The tensor object has a reference on CPU memory used to interact with the tensor itself, as well as a .data parameter that references the underlying data (which may be on CPU or GPU).
For example, you can put tensors from multiple devices in the same dictionary:
x1 = torch.randn(5)
x2 = torch.randn(5).to('cuda:0')
x3 = torch.randn(5).to('cuda:1')
d = {
'tensor1' : x1,
'tensor2' : x2,
'tensor3' : x3,
}
You can do this because the device aspect is only relevant to the pytorch tensors - it doesn't matter for the dictionary itself.
For your example of:
{'rects': [405.5453, 126.3100, 646.7983, 454.6852] , 'points': tensor([[[477.5181, 247.7193],
[591.4567, 247.1899],
[544.2579, 328.7857],
[494.3871, 371.9922],
[583.4272, 370.3429]]], device='cuda:0')}
The dictionary is stored in CPU memory. The dictionary stores pointers to the value objects in the dictionary.
The key rects references a python list stored in CPU memory.
They key points references the CPU pointer of a pytorch tensor object. This tensor has a .data parameter that references data on the GPU, but the dictionary itself doesn't know about that.