[SOLVED] How does tf.gradients manages complex functions?

How does tf.gradients manages complex functions?

I am working with complex-valued neural networks.

For Complex-valued neural networks Wirtinger calculus is normally used. The definition of the derivate is then (take into acount that functions are non-Holomorphic because of Liouville's theorem):

$\frac{\partial f}{\partial z} = \frac{1}{2}\left( \frac{\partial f}{\partial x} - j \frac{\partial f}{\partial y} \right)$

If you take Akira Hirose book "Complex-Valued Neural Networks: Advances and Applications", Chapter 4 equation 4.9 defines:

Screenshot from 2020-03-03 10-38-21

Where the partial derivative is also calculated using Wirtinger calculus of course.

Is this the case for tensorflow? or is it defined in some other way? I cannot find any good reference on the topic.

Solution

Ok, so I discussed this in an existing thread in github/tensorflow and @charmasaur found the response, the equation used by Tensorflow for the gradient is:

$\nabla_z f = \left( \frac{\partial f}{\partial z} + \frac{\partial f*}{\partial z} \right)*=2\frac{\partial Real(f)}{\partial z*}$

When using the definition of the partial derivatives wrt z and z* it uses Wirtinger Calculus.

For cases of a real-valued scalar function of one or several complex variables, this definitions becomes:

$2\frac{\partial f}{\partial z*} = \left( \frac{\partial f}{\partial x} + j \frac{\partial f}{\partial y} \right)$

Which is indeed the definition used in Complex-Valued Neural Networks (CVNN) applications (In this applications, the function is the loss/error function which is indeed real).