pythonrimagemagickbarcodezbar

How to repair obscured blocks in a QR code image

I have many RGB images that contain a printed sheet of paper with a QR code in an outdoor setting. Because of the bright sun interfering with image capture, about 20% of images are unreadable:

enter image description here

I'm using magick in R to handle the image manipulation, and then the Python package pyzbar (which wraps zbar) to do the detection.

I can use image_threshold to find all the pixels that are within the 95% quantile and force them to pure black, which fixes about half my images:

enter image description here

But some of them remain unreadable, like this one. I can see with my human eyes that I need to fill in some of the anchor points in the upper left, so I mocked this up in MS Paint:

enter image description here

With that manual manipulation, this image is now easily read. Is there any way to do this kind of repair automatically? I don't mind translating code from Python or the ImageMagick CLI, so R-only answers aren't necessary.

My general approach:

library(magick)
library(reticulate)
pyzbar <- import("pyzbar.pyzbar")

magick_to_numpy <- function(img) {
  round(255 * as.numeric(magick::image_data(img, "rgb")))
}

image_read("testfile.jpg") %>% 
  image_threshold("black", "95%") %>% 
  magick_to_numpy() %>% 
  pyzbar$decode()

Usual result:

list()

Desired result:

[[1]]
Decoded(data=b'W TRR C6 T2', type='QRCODE', 
rect=Rect(left=176, top=221, width=373, height=333), 
polygon=[Point(x=176, y=226), Point(x=202, y=554), 
Point(x=549, y=544), Point(x=524, y=221)])
Solution

  • You may be able to improve some of your images, but the one you provided has lost too much black to pure white. So gaps will appear that are too large to close up. The best way I know in Imagemagick command line, would be to process the image converted to grayscale using -lat (local area thresholding) and perhaps some morphology open.

    Input:

    enter image description here

    convert img.jpg -colorspace gray -negate -lat 50x50-1% -negate -morphology open square:7 result.png


    enter image description here

    Normally one would use the % term as positive. But here you have lost too much data and I want to include as much as possible that is not pure white before it makes too much black. So I push it to -1%. The -negate is needed since -lat works only on white objects on black background. So I have to negate before and after. I then try to fill some the black regions using some morphology open.