rimage-processingsemantic-segmentationcanny-operatornon-maximum-suppression

Applying non-maximum suppression for edge thinning to an image of edge gradient magnitude in R


I am trying to use Canny edge detection to extract dirt circles from an image using the imager package in R. In my sample image (here) there are 3 circles. I have thresholded the red band as it shows the clearest distinction.

Note that cimg objects from the imager package have 4 dimensions, and R throws a hissy fit when I don't specify all of them (ie. image[i,j,1,1]).

My issue is that during the non-maximum suppression stage where I aim to thin the detected edges I am creating edges that are a salt and pepper effect of the edge present in the gradient magnitude image (here), where my desired output is a 1 pixel wide skeleton of all edges in the image (and further cleaning of weak/strong edges will happen later).

My edge gradient magnitude image calculated using 3x3 sobel kernel (variable G in code) is (here) (link to file: https://drive.google.com/file/d/1pD_h6l2vB3OrJaGEM5ABOZ6LjIwALx2_/view?usp=drive_link), and my edge gradient direction image (variable theta in code) is here (inputting link now due to spam filter) (ranges from -90 to 90, not easily interpretable, link to file: https://drive.google.com/file/d/1HlK4jz0gw2siN2CxGVPJWUFbUsAmhEOP/view?usp=drive_link).

Apologies link to files is via a text file and google drive/download etc, didn't know how else to do it.

My code is below:

#packages
library(imager)


#define function
non_max <- function(image, edge_direction){
  for (i in 2:(nrow(image)-1)){
    for (j in 2:(ncol(image)-1)){
      #angle 0 (-22.5 to 22.5 degree range)
      if (edge_direction[i,j,1,1] >= -22.5 & 
          edge_direction[i,j,1,1] < 22.5 &
          image[i,j,1,1] < image[i+1,j,1,1] | 
          image[i,j,1,1] < image[i-1,j,1,1]){
        image[i,j,1,1] <- 0
      }
      #angle 45 (22.5 to 67.5)
      else if (edge_direction[i,j,1,1] >= 22.5 & 
            edge_direction[i,j,1,1] < 67.5 &
            image[i,j,1,1] < image[i-1,j-1,1,1] | 
            image[i,j,1,1] < image[i+1,j+1,1,1]){
          image[i,j,1,1] <- 0
      }
      #angle 90 (67.5 to 90)
      else if (edge_direction[i,j,1,1] >= 67.5 & 
               edge_direction[i,j,1,1] < 90 &
               image[i,j,1,1] < image[i,j-1,1,1] | 
               image[i,j,1,1] < image[i,j+1,1,1]){
        image[i,j,1,1] <- 0
      }
      #angle -45 (-22.5 to -67.5)
      else if (edge_direction[i,j,1,1] >= -67.5 & 
               edge_direction[i,j,1,1] < -22.5 &
               image[i,j,1,1] < image[i+1,j-1,1,1] | 
               image[i,j,1,1] < image[i-1,j+1,1,1]){
        image[i,j,1,1] <- 0
      }
      #angle 90 (67.5 to 90)
      else if (edge_direction[i,j,1,1] >= -90 & 
               edge_direction[i,j,1,1] < -67.5 &
               image[i,j,1,1] < image[i,j-1,1,1] | 
               image[i,j,1,1] < image[i,j+1,1,1]){
        image[i,j,1,1] <- 0
      }
    }
  }
  plot(image)
}

#execute function
non_max(G,theta)

I found that when I use an extra IF statement in each angle condition, I get a less salt and pepper effect but extra edges are created parallel to one another (here). Example code for extra IF statement is below, unsure why this happens (besides me not understanding some syntax for using & and | for complex statements).

#angle 0 (-22.5 to 22.5 degree range)
      if (edge_direction[i,j,1,1] >= -22.5 & 
          edge_direction[i,j,1,1] < 22.5){
        if (image[i,j,1,1] < image[i+1,j,1,1] | 
            image[i,j,1,1] < image[i-1,j,1,1]){
          image[i,j,1,1] <- 0
        }
      }

I am suspicious of the edge direction file, though I think I calculated it correctly (inverse tan of 1st derivative of y gradient (Iy) divided by 1st derivative of x gradient (Ix) and multiplied by 180/pi to convert to degrees).

theta <- atan(Iy/Ix) * (180/pi)

Does anyone have any ideas about what I'm doing wrong? I looked at some python applications and their edges were opposite to how mine ended up (ie. for an angle of 0 they would look to the cells above and below the target cell, see this link: https://towardsdatascience.com/canny-edge-detection-step-by-step-in-python-computer-vision-b49c3a2d8123) but I'm pretty sure I've got mine right.

First time posting a question, let me know what else I can provide to help. Please and thank you!


Solution

  • The condition

    image[i,j,1,1] < image[i+1,j,1,1] | image[i,j,1,1] < image[i-1,j,1,1]
    

    is incorrect. You are discarding pixels that match this condition. Instead, select pixels that match this condition:

    (image[i,j,1,1] > image[i+1,j,1,1] & image[i,j,1,1] >= image[i-1,j,1,1]) | 
    (image[i,j,1,1] >= image[i+1,j,1,1] & image[i,j,1,1] > image[i-1,j,1,1])
    

    That is, you select pixels that are larger than the two neighbors, but accept if one of the neighbors is equal. This will preserve edges under the case where the gradient maximum has the same value in two neighboring pixels.

    Do note that you cannot modify the input image, because that would affect the test for the next pixel. Instead, write the selected pixels into a new output image.


    Your original condition might produce reasonable results if writing to a separate output image, but the results will not always be correct. The non-maximum suppression described in the original Canny paper uses the condition I describe above.