i was trying to find rectangles the size i choose. using the box width and box height in the code but i find all sorts of rectangle sizes.
i have my test pattern image. to see it when you run the program, first have two monitors, second run the program and move the pop up window to the monitor the program is not showing in the pop up window.
my program does not detect rectangles that are divided with lines and have random letters and numbers touching it.
that is my first problem in the test pattern this is the top picture.
the second problem is the bottom image in the test picture, that is i get too many results in the one image when i tweak the canny and approxPolyDP numerical values to allow for more results. but i'm just trying to find the rectangles the size i want.
here is my code and my test picture:
to use the test picture run the program and video the test picture through the window that is showing video of your desktop.
here is my code, its in python 3:
import numpy as np
import cv2
from mss import mss
from PIL import Image
import imutils
sct = mss()
BOX_WIDTH = 235
BOX_HEIGHT = 10
while 1:
w, h = 240, 640
monitor = {'top': 100, 'left': 900, 'width': w, 'height': h}
img = Image.frombytes('RGB', (w, h), sct.grab(monitor).rgb)
image_data = np.asarray(img)
# Convert the image to grayscale
gray = cv2.cvtColor(image_data, cv2.COLOR_BGR2GRAY)
# Perform Canny edge detection
edges = cv2.Canny(gray, 600900, 150) # i don't know what these values should be? so i used 900, 150 which makes less results
# Find contours in the edges image
hierarchy = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
contours = imutils.grab_contours(hierarchy)
# Iterate over each contour
for contour in contours:
# Approximate the contour to a polygon
polygon = cv2.approxPolyDP(contour, 0.01004 * cv2.arcLength(contour, True), True) # detects if its a square or rectangle
# Check if the polygon has 4 sides
if len(polygon) <= 11:
# Draw the rectangle on the image
x, y, w, h = cv2.boundingRect(contour) # changed polygon to contour
ratio = float(w) / h
length = cv2.arcLength(contour, True)
half_width = BOX_WIDTH // 2
lift_height = BOX_HEIGHT // 6
if (length <= 950) and (length >= 100):
if not ((ratio >= 0.99) and (ratio <= 1.0)):
cv2.rectangle(image_data, (x, y), (x + half_width, y + BOX_HEIGHT - lift_height), (0, 0, 255), 2) # draws green box around rectangle
else:
cv2.rectangle(image_data, (x, y), (x + half_width, y + BOX_HEIGHT - lift_height), (0, 0, 255), 2) # draws green box around square
# Show the result
cv2.imshow('test', image_data)
if cv2.waitKey(25) & 0xFF == ord('q'):
cv2.destroyAllWindows()
break
to fix my first problem, i have tried looking at number 4 in the list in the line its called "4. Contour Approximation", here is the link to the documentation:
https://docs.opencv.org/4.x/dd/d49/tutorial_py_contour_features.html
i tried some test code and it didn't fix it. i have since lost the test code.
for my second problem i have tried adjusting the values in my code. the canny, the approxPolyDP, the len(polygon).
edit. i fixed my problem i put the good code in the first post, edit i found out how i can post the answer so i undo edit and put old bad code in this post again. now i go make the answer reply.
in case you use obs studio version 29.1.3, as of today some days ago i read the opencv update notes and it said opencv can run the obs studio virtual camera!
you can use the (sources, display capture), then (start virtual camera), and this way you can get the desktop video and have access to the stuff videocapture offers too!
below is the code i use when using obs studio to capture my desktop screen:
# for desktop capture, find the rectangle of a specific width and height
#
# running the program pops up a window to watch the video.
# the program video window shows the first monitor,
# but watch the program video window on second extended monitor
import cv2
import imutils
# Path to video file
cap = cv2.VideoCapture(
1,
apiPreference=cv2.CAP_ANY,
params=[cv2.CAP_PROP_FRAME_WIDTH, 1280, cv2.CAP_PROP_FRAME_HEIGHT, 720],
) # I made cap = 1280, 720 resolution to speed the program up on my computer. I have a rtx 3060, obs studio at 60 fps
# Used as counter variable
count = 1
# checks whether frames were extracted
success = 1
# the size of the red box that's around the found rectangle
BOX_WIDTH = 170
BOX_HEIGHT = 26
while success:
# function extract frames
success, image = cap.read()
if count <= 1:
# Saves the frames with frame-count
cv2.imwrite("frame_%d.jpg" % count, image, [int(cv2.IMWRITE_JPEG_QUALITY), 100]) # jpg 100% quality
count += 1
if count == 2:
count = 1
frame = cv2.imread('frame_1.jpg')
# Perform Canny edge detection
edges = cv2.Canny(frame, 100, 200)
# Find contours in the edges image
hierarchy = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
contours = imutils.grab_contours(hierarchy)
# Iterate over each contour
for contour in contours:
# Approximate the contour to a polygon, detects if it's a square or rectangle
# increase "dial" until you see the box around the rectangle your targeting
dial = 0.014 # if you change the resolution then this will probably have to be recalibrated
polygon = cv2.approxPolyDP(contour, dial * cv2.arcLength(contour, True), True)
# Check if the polygon has 4 sides
if len(polygon) == 4:
# Draw the rectangle on the image
contour_x, contour_y, contour_width, contour_height = cv2.boundingRect(contour)
ratio = float(contour_width) / contour_height
half_width = BOX_WIDTH // 2
lift_height = BOX_HEIGHT // 6
# I can set the size of the rectangles I find with this line
# comment out the below if line when you find rectangle with the "dial" variable above
# (set the below if condition contour_width, contour_height,
# until you see the box around your rectangle you found with "dial")
if (contour_width == 59) and (contour_height == 20):
# draws green box around rectangle
# in (contour_x - 4, contour_y - 4), the 4 is to move the drawn green box over the picture
cv2.rectangle(frame, (contour_x - 4, contour_y - 4),
(contour_x + half_width, contour_y + BOX_HEIGHT - lift_height), (0, 255, 0), 2)
# Show the result
cv2.imshow('test', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()