I have two images taken on same sample at t=0 and t=t. There are few new blobs present in image taken at t. I need to find these new blobs (new blobs are the blobs which are present in new XY location at t=t). I am wondering if someone can help?
I tried OR,AND,XOR, reconstructions but the issue is the blobs which are same between two images are not exactly the same. Sometimes they might have size difference which makes the problem complicated.
Instead of using OR,AND,XOR, we may sum the two images.
Before summing the images, replace the 255 values with 100 (keeping the range of uint8
[0, 255]).
In the summed image, there are going to be three values:
0
- Background100
- Non-overlapping area200
- Overlapping areaWe may assume that pixels with value 100
that touches value 200
belongs to the same original blob.
For clearing the overlapping pixels (200) with the touching pixels (100 around them), we may use cv2.floodFill
.
After clearing the overlapping pixels and the pixels around them, the pixels that are left (with value 100) are the new blobs.
Example for clearing the pixels using cv2.floodFill
:
if sum_img[y, x] == 200:
cv2.floodFill(sum_img, None, (x, y), 0, loDiff=100, upDiff=0)
Setting loDiff=100
is used for filling pixels=100 (and pixels=200) with 0
value (200-loDiff
=100
, so the 100
is filled with zero).
For making the solution better, we may find contours (of pixels=200), and ignore the tiny contours.
Code sample:
import cv2
import numpy as np
# Read input images as Grayscale.
img1 = cv2.imread('image1.png', cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread('image2.png', cv2.IMREAD_GRAYSCALE)
# Replace 255 with 100 (we want the sum img1+img2 not to overflow)
img1[img1 >= 128] = 100
img2[img2 >= 128] = 100
# Sum two images - in the sum, the value of overlapping parts of blobs is going to be 200
sum_img = img1 + img2
cv2.floodFill(sum_img, None, (0, 0), 0, loDiff=0, upDiff=0) # Remove the white frame.
cv2.imshow('sum_img before floodFill', sum_img) # Show image for testing.
# Find pixels with value 200 (the overlapping blobs).
thesh = cv2.threshold(sum_img, 199, 255, cv2.THRESH_BINARY)[1]
# Find contours (of overlapping blobs parts)
cnts = cv2.findContours(thesh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
# Iterate contours and fill the overlapping part, and the non-zero pixels around it (i.e with value 100) with zero.
for c in cnts:
area_tresh = 50
area = cv2.contourArea(c)
if area > area_tresh: # Ignore very tiny contours
x, y = tuple(c[0, 0]) # Get coordinates of first pixel in the contour
if sum_img[y, x] == 200:
cv2.floodFill(sum_img, None, (x, y), 0, loDiff=100, upDiff=0) # Setting loDiff=100 is set for filling pixels=100 (and pixels=200)
sum_img[sum_img == 200] = 0 # Remove the small remainders
#thesh = cv2.cvtColor(thesh, cv2.COLOR_GRAY2BGR) # Convert to BGR for testing (for drawing the contours)
#cv2.drawContours(thesh, cnts, -1, (0, 255, 0), 2) # Draw contours for testing
# Show images for testing.
cv2.imshow('thesh', thesh)
cv2.imshow('sum_img after floodFill', sum_img)
cv2.waitKey()
cv2.destroyAllWindows()
Note:
We may dilate the images first, if the two blobs in proximity are considered to be the same blob (I don't no if a blob can "swim")
Output sum_img
(after floodFill):
The above solution finds the blobs that exist in image1 and not in image2 and blobs exist in image2 and not in image1.
In case we want to find only the blobs that are new in image2, and we also assume that blobs that are close in both images are the same one, we may add the following stages:
img1
and img2
before summing (two close blobs are going to be overlapping).sum_img
at the end.sum_img
all the blobs that exist only in img1
(and not in img2
).Code sample:
import cv2
import numpy as np
# Read input images as Grayscale.
img1 = cv2.imread('image1.png', cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread('image2.png', cv2.IMREAD_GRAYSCALE)
# Replace 255 with 100 (we want the sum img1+img2 not to overflow)
img1[img1 >= 128] = 100
img2[img2 >= 128] = 100
# Dilate both images - assume close blobs are the same blob (two blobs are considered overlapped even if they are close but not tuching).
dilated_img1 = cv2.dilate(img1, np.ones((11, 11), np.uint8))
dilated_img2 = cv2.dilate(img2, np.ones((11, 11), np.uint8))
# Sum two images - in the sum, the value of overlapping parts of blobs is going to be 200
sum_img = dilated_img1 + dilated_img2
cv2.floodFill(sum_img, None, (0, 0), 0, loDiff=0, upDiff=0) # Remove the white frame.
#cv2.imshow('sum_img before floodFill', sum_img) # Show image for testing.
# Find pixels with value 200 (the overlapping blobs).
thesh = cv2.threshold(sum_img, 199, 255, cv2.THRESH_BINARY)[1]
# Find contours (of overlapping blobs parts)
cnts = cv2.findContours(thesh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
# Iterate contours and fill the overlapping part, and the non-zero pixels around it (i.e with value 100) with zero.
for c in cnts:
area_tresh = 0 # Optional
area = cv2.contourArea(c)
if area > area_tresh: # Ignore very tiny contours
x, y = tuple(c[0, 0]) # Get coordinates of first pixel in the contour
if sum_img[y, x] == 200:
cv2.floodFill(sum_img, None, (x, y), 0, loDiff=100, upDiff=0) # Setting loDiff=100 is set for filling pixels=100 (and pixels=200)
sum_img[sum_img == 200] = 0 # Remove the small remainders
sum_img[(img1 == 0) & (dilated_img1 == 100)] = 0 # Remove dilated pixels from dilated_img1
sum_img[(img2 == 0) & (dilated_img2 == 100)] = 0 # Remove dilated pixels from dilated_img2
sum_img[(img1 == 100) & (img2 == 0)] = 0 # Remove all the blobs that are only in first image (assume new blobs are "bored" only in image2)
# Visualization:
merged_img = cv2.merge((sum_img*2, img1*2, img2*2))
# The output image is img1, without the
output_image = img1.copy()
output_image[sum_img == 100] = 0
# Show images for testing.
cv2.imshow('sum_img', sum_img)
cv2.imshow('merged_img', merged_img)
cv2.waitKey()
cv2.destroyAllWindows()
Visualization for testing (merged_img
):
img1
img1
and img2
img2
, and not too close to blob in img1
(we are looking for the magenta blobs).