I'm wondering what the range of values are for the weights and inputs are in a convolutional neural networks. My understanding is as follows:
If the input is a grayscale image, the input value of the first layer would be 0-255. But in the subsequent layers the input values would range from 0-1, due to the sigmoid function.
But what are the range for the weights? Is it 0-1, or can it be higher? Does this vary a lot?
Since it is using the sigmoid function, the weights usually range from -∞ to +∞ because the sigmoid of -∞ is near 0 and the sigmoid of +∞ is near 1, and you need to be capable of having near 0 and near 1 values as outputs of your neurons.
And yeah, weights can vary a lot. The more you train the NN, the more weights will get closer to their "needed" value.
For example if a neuron needs to output 0, the ideal value is -∞ because limit of the sigmoid to -∞ is 0. The more you train it, the more the weight will get closer to -∞.