Get differences between matrix values within one unit

I have a list of lists nxm and am trying to take the difference between each value i and every other value j within one unit and add to a new list. How would I be able to do this in Python?

Let's define a test matrix and a function to get all neighbours of the cell:

m = [[i for i in range(10)] for i in range(10)]

# Size of "board"
X = len(m)
Y = len(m[0])

neighbors = lambda x, y : [(x2, y2) for x2 in range(x-1, x+2)
                               for y2 in range(y-1, y+2)
                               if (-1 < x <= X and
                                   -1 < y <= Y and
                                   (x != x2 or y != y2) and
                                   (0 <= x2 <= X) and
                                   (0 <= y2 <= Y))]

Then for calculating the differences:

def get_differences(point, neigh, m):
    px, py = point
    return [m[px][py] - m[nx][ny] for nx, ny in neigh]

And test both for a single point:

point = (1,1)
print('Neighbours: ', neighbors(*(point)))
print('Differences: ')
print(get_differences(point, neighbors(*(point)), m))

The part with this elegant neighbours function is taken from this post.