machine-learningpytorchbackpropagationmax-pooling

maxpooling what should the indices be when there're multiple max values?


What to do when there are multiple values in the kernel that equals to the max? For example, for these values:

array([[0., 0.],
       [0., 0.]])

The max is simply 0. What should max indices look like? should it be True's for all occurrence of max:

array([[ True,  True],
       [ True,  True]])

Or the first occurrence of the max:

array([[ True, False],
       [False, False]])

Pytorch uses the first occurrence of the max, while some source uses the other one (he uses pos = np.where(result == view, 1, 0), which essentially records 1 for all occurrence of max).

Edit: for forward prop it doesn't matter, but during backpropagation, for example, if the upstream gradient is [[1,1],[1,1]], then the gradient of current node will be different: [[1,1],[1,1]] (all occurrence) vs [[1,0],[0,0]] (first occurrence)


Solution

  • After even a single back-propagation it is very unlikely that the kernel values will remain equal. So it shouldn't be a real issue, should it?