neural-networkdeep-learningcaffeconv-neural-networkpycaffe

caffe SqueezeNet: where is the fully connected FC layer in prototxt

I am working on caffe SqueezeNet prototxt link.

I am just wondering where is the FC layer? (I only see type: data, conv, relu, pooling, concat, SoftmaxWithLoss and accuracy)

Solution

  • The reason is that FC layers have a ton of parameters, counting for the majority of the network's parameters in some architectures. The authors of SqueezeNet removed the FCs, replacing them with a convolutional layer and a global average pooling.

    The conv layer has a number of filters equal to the number of classes, processing the output of a previous layer to (roughly) a map for each class. The pooling averages the response of each of these maps. They end up with a flattened vector with dimension equal to the number of classes that is, then, fed to the SoftMax layer.

    With these modifications (not forgetting the Fire modules they proposed) they were able to significantly reduce memory footprint.

    I strongly recommend that you read the SqueezeNet paper.