I'm trying to write a program that shows a specific position on a given window. When I run the code OpenCV gives me an error message telling me unexpected type for argument mat. I have no idea what's wrong because when I use the position value in the following code it works fine. Please be nice as I'm fairly new to OpenCV.
Code:
#Imports
import numpy as np
import cv2
import pytesseract
from PIL import ImageGrab
import win32gui
#Assign Tesseract and other Variables
winList = []
toplist = []
pytesseract.pytesseract.tesseract_cmd = r'C:\\Program Files\\Tesseract-OCR\\tesseract.exe'
lH = 33
lW = 20
#Window Handle Object
def enum_win(hwnd, result):
winText = win32gui.GetWindowText(hwnd)
winList.append((hwnd, winText))
win32gui.EnumWindows(enum_win, toplist)
typeHwnd = 0
#Find Window Handle
for (hwnd, winText) in winList:
if winText == "Program WinText Name Here":
typeHwnd = hwnd
if typeHwnd == 0:
print("Oops no Window Found")
while True:
position = win32gui.GetWindowRect(typeHwnd)
mid = (position[0] + (position[0] + position[2]))/2
sPos = [int(mid)-267, position[1] + 295, lW, lH]
screenshot = ImageGrab.grab(bbox=sPos)
screenshot = np.array(screenshot)
cv2.imshow("Screen", screenshot)
key = cv2.waitKey(25)
print(position)
print(mid)
Error Message:
Traceback (most recent call last):
File "C:/Users/MyName/PycharmProjects/Typing Bot/main.py", line 47, in <module>
cv2.imshow("Screen", screenshot)
TypeError: Expected Ptr<cv::UMat> for argument 'mat'
System Specifications:
I am running Windows 10 64 bit and Python 3 with the most recent version of all the imported modules installed.
Error shows problem with array
in imshow
but your problem starts with sPos
in grab
.
grab
needs bbox
which means [x1, y1, x2, y2]
but you use [x, y, width, height]
and this starts problem because grab
returns image with size (0,0)
and np.array()
can't convert it so it returns pillow image
instead of numpy array
and then imshow
gets pillow image
instead of numpy array
and it can't display it.
You have to use [x, y, x+width, y+height]
Minimal working example - with wrong and correct pos
import cv2
import numpy as np
import PIL.ImageGrab
x = 100
y = 100
w = 50
h = 50
pos = [x, y, w, h] # wrong
#pos = [x, y, x+w, y+h] # correct
print('bbox:', pos)
print('---')
img = PIL.ImageGrab.grab(pos)
print('img:', img)
print('type(img):', type(img))
print('img.size:', img.size)
print('---')
arr = np.array(img)
print('arr:', arr)
print('type(arr):', type(arr))
print('arr.size:', arr.size)
print('arr.shape:', arr.shape)
print('---')
cv2.imshow('test', arr)
print('Press ESC to exit')
while cv2.waitKey(100) != 27:
pass