[SOLVED] Sigmoid Derivative using nd4j

Sigmoid Derivative using nd4j

I perform sigmoid which works fine but sigmoidDerivative gives the same result as sigmoid in nd4j. What is the difference between Transforms.sigmoidDerivative(x) and Transforms.sigmoidDerivative(x, true)?

INDArray x = Nd4j.create(new double[] { 0.1812, 0.1235, 0.8466 });
System.out.println(x);
System.out.println(Transforms.sigmoid(x));
System.out.println(Transforms.sigmoidDerivative(x));
System.out.println(Transforms.sigmoidDerivative(x, true));

Gives output:

[[    0.1812,    0.1235,    0.8466]]
[[    0.5452,    0.5308,    0.6999]]
[[    0.5452,    0.5308,    0.6999]]
[[    0.2480,    0.2490,    0.2101]]

Comparing against python's numpy:

>>> def sigmoid(x):
...     return 1.0 / (1 + np.exp(-x))
... 
>>> def sigmoid_derivative(x):
...     a = sigmoid(x)
...     return a * (1.0 - a)

>>> x = np.array([    0.1812,    0.1235,    0.8466])
>>> sigmoid(x)
array([0.54517646, 0.53083582, 0.69985343])
>>> sigmoid_derivative(x)
array([0.24795909, 0.24904915, 0.21005861])

Nd4j pom:

<dependency>
     <groupId>org.nd4j</groupId>
     <artifactId>nd4j-native-platform</artifactId>
     <version>1.0.0-beta3</version>
</dependency>

Solution

You are right, both Transforms.sigmoidDerivative(x) and Transforms.sigmoidDerivative(x, true) should give the same results, it's a bug in dl4j. The correct behavior has the latter method. I have submitted a pull request to resolve it.