Matplotlib Rectangle.Contains(event) always returns true

I want to detect the button_press and button_release events on matplotlib.patches.Rectangle areas next to my figure to enable the user to move/rescale individual y-axes when using Twinx().

However, rectangle.Contains(event) always seems to return true, no matter where I click. E.g: when click on the red bar in the figure below, Rectangle1, 2 and 3 are all being printed.

A working example:

import matplotlib.transforms as mtransforms
import matplotlib.pyplot as plt
import matplotlib
fig, ax = plt.subplots()
plt.plot([1,2,3,4,5,6,7,8,9,10], [1,2,3,4,5,6,7,8,9,10])
spine_width=20

color = ["red", "green", "blue"]
for i in range(3):
    shift = (spine_width * i + 0.5*spine_width)
    offset_vec = (1, 0)
    offset_dots = shift * np.array(offset_vec) / 72
    combitransform = ( ax.transAxes
                + mtransforms.ScaledTranslation(
                    *offset_dots, fig.canvas.figure.dpi_scale_trans))

    rectangle = matplotlib.patches.Rectangle((1, 0), 0, 1, lw=spine_width, ec=color[i], alpha=1, transform=combitransform, clip_on=False)

    ax.add_patch(rectangle)

    def on_press(event, rectangle, i):
        if rectangle.contains(event):
            print(f"Clicked on rectangle {i+1} at {event.x} {event.y}!")
    

    fig.canvas.mpl_connect('button_press_event', lambda event, rectangle=rectangle, i=i: on_press(event, rectangle, i))


plt.show()

The problem was probably the fact that the width of the rectangle was zero in my code (lw does not seem to contribute to the click-hitbox). Although it is not entirely clear to me why this would always result in rectangle.contains(event) to evaluate to True.

This ended up working (although transformations are still a bit vague to me, so there might be a better solution):

import matplotlib.transforms as mtransforms
import matplotlib.pyplot as plt
import matplotlib
fig, ax = plt.subplots()
plt.plot([1,2,3,4,5,6,7,8,9,10], [1,2,3,4,5,6,7,8,9,10])
spine_width=20

color = ["red", "green", "blue"]
for i in range(3):
    x_transform = mtransforms.IdentityTransform() + mtransforms.ScaledTranslation(1,0 , ax.transAxes)
    transform = mtransforms.blended_transform_factory(x_transform, ax.transAxes) #x-axis in pixels, y-axis in axes coords (0, 1)
    rectangle = matplotlib.patches.Rectangle((spine_width*i, 0), 20, 1, ec=color[i], fc=color[i], alpha=1, transform=transform, clip_on=False)
    ax.add_artist(rectangle)

    def on_press(event, rectangle, i):
        #Check if click is inside rectangle
        if rectangle.contains_point((event.x, event.y)):
            print("Click inside rectangle", i)

    fig.canvas.mpl_connect('button_press_event', lambda event, rectangle=rectangle, i=i: on_press(event, rectangle, i))


plt.show()