I was going through PyTorch tutorial and came across empty() function. It was mentioned that empty() can be used for uninitialized data. But, when I printed it, I got a value. What is the difference between empty() and rand() which also generates data(I know that rand() generates between 0 and 1). Below is the code I tried:
a = torch.empty(3,4)
print(a)
Output:
tensor([[ 8.4135e-38, 0.0000e+00, 6.2579e-41, 5.4592e-39], [-5.6345e-08, 2.5353e+30, 5.0447e-44, 1.7020e-41], [ 1.4000e-38, 5.7697e-05, 2.5353e+30, 2.1580e-43]])
b = torch.rand(3,4)
print(b)
Output:
tensor([[ 0.1514, 0.8406, 0.2708, 0.3422], [ 0.7196, 0.6120, 0.4476, 0.6705], [ 0.6989, 0.2086, 0.5100, 0.8285]])
Once you call torch.empty(), a block of memory is allocated according to the size (shape) of the tensor. By uninitialized data, it's meant that torch.empty() would simply return the values in the memory block as is. These values could be default values or it could be the values stored in those memory blocks as a result of some other operations, which used that part of the memory block before.
Here's a simple illustration:
# a block of memory with the values in it
In [74]: torch.empty(2, 3)
Out[74]:
tensor([[-1.0049e+08, 4.5688e-41, -9.1450e-38],
[ 3.0638e-41, 4.4842e-44, 0.0000e+00]])
# same run; but note the change in values.
# i.e. different memory addresses than on the previous run were used.
In [75]: torch.empty(2, 3)
Out[75]:
tensor([[-1.0049e+08, 4.5688e-41, -7.9421e-38],
[ 3.0638e-41, 4.4842e-44, 0.0000e+00]])