Use RectangleSelector after zooming in a matplotlib.pyplot figure?

I was wondering if there is a way to keep matplotlib's RectangleSelector activate after zooming. To hinder a possible confusion between my problem and existing ones, kindly note that

I am able to use the RectangleSelector at original view persistently:

And the rectangle that I select is visible after zooming:

What would like to achieve is to zoom into and image area first and then select a rectangular region within that zoom.

Here is an example code to reproduce my use case:

import matplotlib.pyplot as plt
from skimage import data as image_data
import matplotlib.widgets as mwidgets

def onselect(eclick, erelease):
    """
    Handles the rectangle selection event.
    :param eclick: the click event
    :param erelease: the release event
    """

    # only if middle button has been held down
    if eclick.button != 2 or erelease.button != 2:
        return

    print(eclick.xdata, eclick.ydata)
    print(erelease.xdata, erelease.ydata)

def update_callback(event):
    """
    Handles the update event.
    :param event: the update event
    """
    if r_selector.active:
        r_selector.update()

fig, ax = plt.subplots(figsize=(20,40))

im = ax.imshow(image_data.retina(), cmap='jet')

props = dict(facecolor='blue', alpha=0.5)
r_selector = mwidgets.RectangleSelector(ax, onselect, interactive=True)

fig.canvas.mpl_connect('draw_event', update_callback)

plt.show()

I turns out that what hindered me from drawing another rectangle while image is zoomed in was my lack of understanding about how zoom button (marked with blue at the image below) works. For example, I was not able to draw a rectangle at the case below.

But once the zoom button is pressed, it disables the RectangleSelector widget completely. However, if you click again to unpress it, RectangleSelector becomes active again.