I am learning about Apache MXNet and I have encountered the following simple example:
Specifically, this tutorial contains the following code snippet:
from mxnet import nd
from mxnet import autograd
x = nd.array([[1, 2], [3, 4]])
x.attach_grad()
with autograd.record():
y = 2* x * x
y.backward()
print(x.grad)
The tutorial explains,
The derivative of 2x^2 with respect to x is 4x, thus x.grad = [[4, 8], [12, 16]].
This would make sense to me if x were a real variable, but x is not a real variable, it's a 2x2 matrix. I thought that because the function y=y(x) takes a 2x2 matrix and returns a 2x2 matrix, the rules of single variable calculus do not apply.
Mathematically, why is x.grad equal to 4*x?
It's off-topic but I'd still like to answer.
The derivation of 2x² is 4x. You can calculate that by multiplying the exponent with the number in front of the x and reducing the exponent by 1.
Now you can multiply every value (x) in the matrizes with 4. Which results in the numbers: 4, 8 - 12, 16
All values, no matter if it's one or two dimensional are x in that case. You apply this function to every single value there.