artificial-intelligenceneural-network

Values of weights and outputs in Convolutional Neural Networks


I'm wondering what the range of values are for the weights and inputs are in a convolutional neural networks. My understanding is as follows:

If the input is a grayscale image, the input value of the first layer would be 0-255. But in the subsequent layers the input values would range from 0-1, due to the sigmoid function.

But what are the range for the weights? Is it 0-1, or can it be higher? Does this vary a lot?


Solution

  • Since it is using the sigmoid function, the weights usually range from -∞ to +∞ because the sigmoid of -∞ is near 0 and the sigmoid of +∞ is near 1, and you need to be capable of having near 0 and near 1 values as outputs of your neurons.

    And yeah, weights can vary a lot. The more you train the NN, the more weights will get closer to their "needed" value.

    For example if a neuron needs to output 0, the ideal value is -∞ because limit of the sigmoid to -∞ is 0. The more you train it, the more the weight will get closer to -∞.