how to inlay a matplotlib imshow inside another

Here's a reprex

import urllib # to retrieve the file from the internet
from PIL import Image # to interpret the bytes as an image
import numpy as np # to work with arrays of numbers (the image is an array of pixels)
import matplotlib.pyplot as plt # for general plotting

# The URL for the file
url = r"https://upload.wikimedia.org/wikipedia/en/9/9d/The_Tilled_Field.jpg"

img = np.array(Image.open(urllib.request.urlopen(url)))

# Make a figure and axes
fig, ax = plt.subplots(figsize=(4.5, 6.4))

# Add the image to the axes
ax.imshow(img);

# Say where to put the ticks on the axes
ax.set_xticks(range(0,1300,100), minor=False)
ax.set_xticks(range(0,1300,20), minor=True)
ax.set_yticks(range(0,900,100), minor=False)
ax.set_yticks(range(0,900,20), minor=True)

# Add some gridlines
ax.grid(which='minor', color='grey');


# I want this figure to be inlayed inside the bigger figure
ax.imshow(img, extent=[400,528,200,290]);

I thought extent would make the second ax.imshow put the image inside the first, but it doesn't.

If I remove that line, it does exactly what I expect. But when I use that line, instead of plotting on top of the previous image, in the specified extent, it seems to create a brand new plot.

Does anyone know what I'm doing wrong?

I think the solution here is create a new axis for the inlay. Images typically to do not share a common coordinate system and if the images are not transparent they will not be shown on top of each other, when plotted on the same axis. The tricky part is converting the coordinates from the reference image to figure coordinates, especially because images in matplotlib have their origin at the top resulting in an inverted y axis.

However here is a minimal example:

import urllib
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt

url = r"https://upload.wikimedia.org/wikipedia/en/9/9d/The_Tilled_Field.jpg"
img = np.array(Image.open(urllib.request.urlopen(url)))

img_height, img_width = img.shape[:2]

fig_width = 6.4
fig, ax = plt.subplots(figsize=(fig_width, fig_width / img_width * img_height))

ax.imshow(img.sum(axis=2), cmap='gray');
ax.axis('off');

data_to_fig = ax.transData + ax.figure.transFigure.inverted()

# this is given in coordinates of the first image, with origin at the bottom left in units of pixels
left, bottom, width = 0, 0, 300 
height = width / img_width * img_height

(left, bottom), (right, upper)  = data_to_fig.transform([(left, img_height - bottom), (left + width, img_height - bottom - height)])

ax_inlay = fig.add_axes((left, bottom, right - left, upper - bottom))
ax_inlay.axis('off')

ax_inlay.imshow(img);

Which creates:

I hope this helps!