So I was reading the pytorch document trying to learn and understand somethings(because I'm new to the machine learning ), I found the torch.bernoulli() and I understood (I miss understood it) that it approximates the tensors that the have the values between 1 and 0 to 1 or 0 depends on the value (like classic school less than 0.5 = 0 , more the than or equals 0.5 = 1)
After some experimentations on my own that yeah it works as expected
>>>y = torch.Tensor([0.500])
>>>x
>>> 0.5000
[torch.FloatTensor of size 1]
>>> torch.bernoulli(x)
>>> 1
[torch.FloatTensor of size 1]
But when I looked at the document something a bit weird
>>> a = torch.Tensor(3, 3).uniform_(0, 1) # generate a uniform random matrix with range [0, 1]
>>> a
0.7544 0.8140 0.9842
**0.5282** 0.0595 0.6445
0.1925 0.9553 0.9732
[torch.FloatTensor of size 3x3]
>>> torch.bernoulli(a)
1 1 1
**0** 0 1
0 1 1
[torch.FloatTensor of size 3x3]
in the example the 0.5282 got approximated to 0 , how did that happen ? or it's a fault in the document because I tried it and the 0.5282 got approximated as expected to 1.
Well, Bernoulli is a probability distribution. Specifically, torch.distributions.Bernoulli() samples from the distribution and returns a binary value (i.e. either 0 or 1). Here, it returns 1 with probability p and return 0 with probability 1-p.
Below example will make the understanding clear:
In [141]: m = torch.distributions.Bernoulli(torch.tensor([0.63]))
In [142]: m.sample() # 63% chance 1; 37% chance 0
Out[142]: tensor([ 0.])
In [143]: m.sample() # 63% chance 1; 37% chance 0
Out[143]: tensor([ 1.])
In [144]: m.sample() # 63% chance 1; 37% chance 0
Out[144]: tensor([ 0.])
In [145]: m.sample() # 63% chance 1; 37% chance 0
Out[145]: tensor([ 0.])
In [146]: m.sample() # 63% chance 1; 37% chance 0
Out[146]: tensor([ 1.])
In [147]: m.sample() # 63% chance 1; 37% chance 0
Out[147]: tensor([ 1.])
In [148]: m.sample() # 63% chance 1; 37% chance 0
Out[148]: tensor([ 1.])
In [149]: m.sample() # 63% chance 1; 37% chance 0
Out[149]: tensor([ 1.])
In [150]: m.sample() # 63% chance 1; 37% chance 0
Out[150]: tensor([ 1.])
In [151]: m.sample() # 63% chance 1; 37% chance 0
Out[151]: tensor([ 1.])
So, we sampled it 10 times, out of which we got 1s 7 times which is approximately close to 63%. We need to sample this finitely large number of times to get the exact percentage of 37 and 63 for 0s and 1s respectively; This is because of the Law of Large Numbers.