Weird behaviors on interactive imshow plot in Python

I'm trying to construct a grid of black squares, and everytime you click on one it turns white. Now for some reason my code does very weird things:

1. The coordinates I input doesn't correspond to the array coordinates. I tried to change that by letting i = y - (N-1) and j = x with (x,y) the mouse coordinates. But only the first line will be converted properly (top row of the plot). The rest will be inverted vertically.
2. When all squares are white the plot automatically reset to black squares.

Here is my code:

import numpy as np
import matplotlib.pyplot as plt
import matplotlib
import math

N = 3

# Make an empty data set
data = np.zeros((N, N)) 
    
# Make a figure + axes
fig, ax = plt.subplots(1, 1, tight_layout=True)

# Draw the boxes
box = ax.imshow(data, cmap='gray', extent=[0, N, 0, N])

# Draw the grid

for x in range(N + 1):
    ax.axhline(x, lw=2, color='w', zorder=5)
    ax.axvline(x, lw=2, color='w', zorder=5)
    
# Create interactivity
def on_click(event):
    gx = event.xdata
    gy = event.ydata
    
    print('x=',gx)
    print('y=',gy)
    
    i = int(gy) - N + 1
    j = int(gx) 
    
    data[i,j] = 1
    ax.imshow(data, cmap='gray', extent=[0, N, 0, N])
    
    fig.canvas.draw_idle()
    
fig = plt.gcf()   
fig.canvas.mpl_connect('button_press_event', on_click)

# Turn off the axis labels
ax.axis('off')

plt.show()

Thanks for your help

Well I found the issues:

1. It was N - 1 - int(gy)
2. Matplotlib normalizes the values for me when they aren't specified. So a grid of all zeros is the same as a grid of all ones to it (each cell is equal to the mean, in both cases).