I'm writing a test in Playwright Python and pytest to see if the auto mouse movements can be simulated to look more like of a real user's. I use a local html canvas written from html and javascript, the code is from here. The mouse is supposed to move to point (400,50) in the browser before the HTML canvas file is requested (In the real function, the starting point will instead will randomized. Otherwise, it will always start at (0,0) which would make it look more like a bot). When the canvas is open, it's supposed to draw lines from left to right using WindMouse algorithm with the same x-values for start and end points, respectively. There shouldn't be any lines connected between the lines, except for the one from the starting point to the first line. This should be because after initiating to hold down the mouse's left button with page.mouse.down()
, and then actually drawing with page.mouse.move()
from x=100
to x=1200
with different y-values in range 100 to 1000, the mouse should release outside of the loop with page.mouse.up()
.
As seen in the image below, that's not what happened. Instead the page.mouse.up()
doesn't seem to execute after page.mouse.down()
and page.mouse.move()
. I have researched and found that it might be because when the mouse left button has been held down for a certain amount of time, the browser will recognize the action as a mouse drag instead. If this is the case, how do you disable the browser's ability to automatically switch mouse action recognition; in this case, it would be to disable it from automatically recognizing page.mouse.down()
and page.mouse.move()
after a certain amount of time as a mouse drag? And if this is not the case, how do you fix this problem with Playwright page.mouse.up()
?
Please have a look at the code:
def test_drawing_board():
rel_path = r"/mats/drawing_board.html"
file_path = "".join([r"file://", os.getcwd(), rel_path])
with sync_playwright() as playwright:
# Fetch drawing board
browser = playwright.chromium.launch(headless=False, slow_mo=0.1)
page = browser.new_page()
page.mouse.move(400,50) # Place mouse in a random position in the browser before fetching the page
page.goto(file_path)
#Move mouse
start_point = 100
x = 1200
for y in range(100, 1000, 100):
# Generate mouse points
points = []
wm(start_point, y, x, y, M_0=15, D_0=12, move_mouse=lambda x, y: points.append([x, y]))
# Draw
page.mouse.down()
for point in points:
page.mouse.move(point[0], point[1])
page.mouse.up()
Within the # Draw block
it should include a page.mouse.move(start_point, y)
to move to the beginning of the line before every draw as @tomgalfin suggested, and then page.mouse.down()
. To test out if before drawing the first line, the mouse was in a specific position even before requesting the page. I wanted to confirm the initial mouse position with a line drawn between it and the first line's beginning position. This is achieved with adding a page.mouse.down()
before the # Move mouse
loop as edited. Here's the solved code and the results
def test_drawing_board():
rel_path = r"/mats/drawing_board.html"
file_path = "".join([r"file://", os.getcwd(), rel_path])
with sync_playwright() as playwright:
# Fetch drawing board
browser = playwright.chromium.launch(headless=False, slow_mo=0.1)
page = browser.new_page()
page.mouse.move(400,50) # Place mouse in a random position in the browser before fetching the page
page.goto(file_path)
# Start points
start_point = 100
x = 1200
# Move mouse
page.mouse.down()
for y in range(100, 1000, 100):
# Generate mouse points
points = []
wm(start_point, y, x, y, M_0=15, D_0=12, move_mouse=lambda x, y: points.append([x, y]))
# Draw
page.mouse.move(start_point, y)
page.mouse.down()
for point in points:
page.mouse.move(point[0], point[1])
page.mouse.up()