pythonmatlabopencvcomputer-visionobject-tracking

Find new blobs comparing two different binary images


I have two images taken on same sample at t=0 and t=t. There are few new blobs present in image taken at t. I need to find these new blobs (new blobs are the blobs which are present in new XY location at t=t). I am wondering if someone can help?

I tried OR,AND,XOR, reconstructions but the issue is the blobs which are same between two images are not exactly the same. Sometimes they might have size difference which makes the problem complicated.

Image at t=0 Image at t=0 Image at t=t Image at t=t


Solution

  • Instead of using OR,AND,XOR, we may sum the two images.
    Before summing the images, replace the 255 values with 100 (keeping the range of uint8 [0, 255]).
    In the summed image, there are going to be three values:

    We may assume that pixels with value 100 that touches value 200 belongs to the same original blob.

    For clearing the overlapping pixels (200) with the touching pixels (100 around them), we may use cv2.floodFill.

    After clearing the overlapping pixels and the pixels around them, the pixels that are left (with value 100) are the new blobs.

    Example for clearing the pixels using cv2.floodFill:

    if sum_img[y, x] == 200:
        cv2.floodFill(sum_img, None, (x, y), 0, loDiff=100, upDiff=0)
    

    Setting loDiff=100 is used for filling pixels=100 (and pixels=200) with 0 value (200-loDiff=100, so the 100 is filled with zero).

    For making the solution better, we may find contours (of pixels=200), and ignore the tiny contours.


    Code sample:

    import cv2
    import numpy as np
    
    # Read input images as Grayscale.
    img1 = cv2.imread('image1.png', cv2.IMREAD_GRAYSCALE)
    img2 = cv2.imread('image2.png', cv2.IMREAD_GRAYSCALE)
    
    # Replace 255 with 100 (we want the sum img1+img2 not to overflow)
    img1[img1 >= 128] = 100
    img2[img2 >= 128] = 100
    
    # Sum two images - in the sum, the value of overlapping parts of blobs is going to be 200
    sum_img = img1 + img2
    
    cv2.floodFill(sum_img, None, (0, 0), 0, loDiff=0, upDiff=0)  # Remove the white frame.
    
    cv2.imshow('sum_img before floodFill', sum_img)  # Show image for testing.
    
    # Find pixels with value 200 (the overlapping blobs).
    thesh = cv2.threshold(sum_img, 199, 255, cv2.THRESH_BINARY)[1]
    
    # Find contours (of overlapping blobs parts)
    cnts = cv2.findContours(thesh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
    
    
    # Iterate contours and fill the overlapping part, and the non-zero pixels around it (i.e with value 100) with zero.
    for c in cnts:
        area_tresh = 50
        area = cv2.contourArea(c)
        if area > area_tresh:  # Ignore very tiny contours
            x, y = tuple(c[0, 0])  # Get coordinates of first pixel in the contour
            if sum_img[y, x] == 200:
                cv2.floodFill(sum_img, None, (x, y), 0, loDiff=100, upDiff=0)  # Setting loDiff=100 is set for filling pixels=100 (and pixels=200)
    
    sum_img[sum_img == 200] = 0  # Remove the small remainders
    
    #thesh = cv2.cvtColor(thesh, cv2.COLOR_GRAY2BGR)  # Convert to BGR for testing (for drawing the contours)
    #cv2.drawContours(thesh, cnts, -1, (0, 255, 0), 2)  # Draw contours for testing
    
    # Show images for testing.
    cv2.imshow('thesh', thesh)
    cv2.imshow('sum_img after floodFill', sum_img)
    cv2.waitKey()
    cv2.destroyAllWindows()
    

    Note:
    We may dilate the images first, if the two blobs in proximity are considered to be the same blob (I don't no if a blob can "swim")

    Output sum_img (after floodFill):
    enter image description here


    Update:

    The above solution finds the blobs that exist in image1 and not in image2 and blobs exist in image2 and not in image1.

    In case we want to find only the blobs that are new in image2, and we also assume that blobs that are close in both images are the same one, we may add the following stages:


    Code sample:

    import cv2
    import numpy as np
    
    # Read input images as Grayscale.
    img1 = cv2.imread('image1.png', cv2.IMREAD_GRAYSCALE)
    img2 = cv2.imread('image2.png', cv2.IMREAD_GRAYSCALE)
    
    # Replace 255 with 100 (we want the sum img1+img2 not to overflow)
    img1[img1 >= 128] = 100
    img2[img2 >= 128] = 100
    
    # Dilate both images - assume close blobs are the same blob (two blobs are considered overlapped even if they are close but not tuching).
    dilated_img1 = cv2.dilate(img1, np.ones((11, 11), np.uint8))
    dilated_img2 = cv2.dilate(img2, np.ones((11, 11), np.uint8))
    
    # Sum two images - in the sum, the value of overlapping parts of blobs is going to be 200
    sum_img = dilated_img1 + dilated_img2
    
    cv2.floodFill(sum_img, None, (0, 0), 0, loDiff=0, upDiff=0)  # Remove the white frame.
    
    #cv2.imshow('sum_img before floodFill', sum_img)  # Show image for testing.
    
    # Find pixels with value 200 (the overlapping blobs).
    thesh = cv2.threshold(sum_img, 199, 255, cv2.THRESH_BINARY)[1]
    
    # Find contours (of overlapping blobs parts)
    cnts = cv2.findContours(thesh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
    
    
    # Iterate contours and fill the overlapping part, and the non-zero pixels around it (i.e with value 100) with zero.
    for c in cnts:
        area_tresh = 0  # Optional
        area = cv2.contourArea(c)
        if area > area_tresh:  # Ignore very tiny contours
            x, y = tuple(c[0, 0])  # Get coordinates of first pixel in the contour
            if sum_img[y, x] == 200:
                cv2.floodFill(sum_img, None, (x, y), 0, loDiff=100, upDiff=0)  # Setting loDiff=100 is set for filling pixels=100 (and pixels=200)
    
    sum_img[sum_img == 200] = 0  # Remove the small remainders
    
    sum_img[(img1 == 0) & (dilated_img1 == 100)] = 0  # Remove dilated pixels from dilated_img1
    sum_img[(img2 == 0) & (dilated_img2 == 100)] = 0  # Remove dilated pixels from dilated_img2
    sum_img[(img1 == 100) & (img2 == 0)] = 0  # Remove all the blobs that are only in first image (assume new blobs are "bored" only in image2)
    
    # Visualization:
    merged_img = cv2.merge((sum_img*2, img1*2, img2*2))
    
    # The output image is img1, without the 
    output_image = img1.copy()
    output_image[sum_img == 100] = 0
    
    # Show images for testing.
    cv2.imshow('sum_img', sum_img)
    cv2.imshow('merged_img', merged_img)
    cv2.waitKey()
    cv2.destroyAllWindows()
    

    Output (sum_img*2):
    enter image description here

    Visualization for testing (merged_img):

    enter image description here