How to do a search for an image in a specific place on the screen, in a continuous search with a limit of 'x' of seconds, until the image is found? Where if the image is not found, return a False And if the image is found, return the coordinates of the location found... It can also be automatically clicked.
This function came to mind as a trigger for 'done' when waiting for a specific visual response from a web page that loads. I am created automation for browsing websites in an operation based only on visual, so I should not use libs as requests or selenium. The lib pyautogui was the best tool I found, but its methods are very rudimentary (focus only on the essentials) and I cannot create more practical functions.
Ow boy... I have already suffered from how lean the pyautogui methods are...
Maybe it will help you.
Below there is a very flexible to make continuous and optimized searches in small areas.
I tried to make the docstring as readable as possible.
If the function doesn't seem clear enough to you, please let me know that I will improve the docstring.
import logging
import pyautogui as pag
from PIL import Image
import time
def pag_suf(img, x, y, margin, clicks=0, duration=0, interval=0,
debug_msg='', time_limit=None, sample_dump=None):
"""
Pyautogui - Search Until Find
Searches the image indefinitely at a specific point on the screen
considering as the search area, the image size plus an expansion margin.
If found, you can click on the center of the image.
:param img: String. Fullpath image | List. List of Fullpath image
:param x: coordinate x
:param y: coordinate y
:param margin: Integer. expansion margin to expand the search area
:param clicks: Integer. number of clicks
:param duration: Float. duration of mouse movement
:param interval: Float. sleep time after click
:param time_limit: Integer. Time limit in seconds
:param debug_msg: String. Debug message to identify log
:param sample_dump: String. File name if image .bmp
:return: List. Coordinates of the center of the found image. |
False. If time_limit reached.
"""
is_string = type(img) == str
list_img = []
if is_string:
list_img.append(img)
else:
list_img = img
# Search for image at the indicated location with tolerance margins
return_value = None
logging.debug(f"{debug_msg}: Finding...")
first_loop = True
start_time = time.time()
while return_value is None:
# Scape in time_limit
if time_limit is not None:
elapsed_time = time.time() - start_time
if elapsed_time > time_limit:
return False
else:
pass
else:
pass
if first_loop is False:
time.sleep(0.5)
else:
first_loop = False
for img in list_img:
im = Image.open(img)
# Defining variables
img_width, img_height = im.size
coor_x = x - img_width / 2 - margin
coor_y = y - img_height / 2 - margin
region_x = img_width + margin * 2
region_y = img_height + margin * 2
# Save collected sample
screen_sample = pag.screenshot(imageFilename=sample_dump,
region=(coor_x, coor_y,
region_x, region_y))
return_value = pag.locate(img, screen_sample)
if return_value is not None:
# logging.debug(img)
break
logging.debug(f"{debug_msg}: Found.")
click_x = coor_x + return_value[0] + img_width / 2
click_y = coor_y + return_value[1] + img_height / 2
# Click on the center of the found location
if clicks != 0:
pag.click(click_x, click_y, clicks,
duration=duration, interval=interval)
click_arr = []
click_arr.append(click_x)
click_arr.append(click_y)
return click_arr