pythonimagenumpyinteger-overflowuint

Numpy overflow with np.divide(). RuntimeWarning: overflow encountered in ushort_scalars


I am running the following code on 2 images:

ndvi = np.divide(img8 - img4, img8+img4)

invalid = (ndvi > 1).any()
if invalid:
    print("Stopping Execution")
    print(ndvi)

img8 and img4 are 2 images and have all positive values. ndvi is (img8-img4)/(img8+img4)

Hence, by definition, all elements of ndvi should be between -1 and 1. But I am getting some values>1

The dtypes of all the variables in this context are 'uint16'

When I check the index of the invalid values, and ran the individual code:

temp = (img8[88][118]-img4[88][118])/(img8[88][118]+img4[88][118])

I got the following warning:

<stdin>:1: RuntimeWarning: overflow encountered in ushort_scalars

The values are: img8[88][118] = 1462 img4[88][118] = 1652

The values themself are not large to result in an overflow, but when array sizes become large overflow happens.


Solution

  • When subtracting a large number from a small one and both are unitXX, you get an overflow (really, an underflow), and the result is the modulo-XX of the negative number, which is a large number.
    This is because uint can't represent a negative number, and instead holds a large positive.

    This modulo is actually adding maxint (==65535) to the negative

    In this case, for the index you specified,

    img8[88][118]-img4[88][118] == 1462 - 1652 == -190 == 65535 - 190 = 65345
    

    Divide that by 1462 + 1652 == 3114 and get 20.984264611432241490044958253051


    Solution:

    Convert the dtypes to float before dividing, and usually work with floats for images, rather than uint.

    Floats don't (normally) have the underflow issue for simple subtraction, as they are built to also represent negatives.

    Also would work is using int rather than uint but stick to floats for your own sake.