[SOLVED] Issue in understanding the counting of MACC operations of convolution layer

Issue in understanding the counting of MACC operations of convolution layer

I have the below code for calculating the MACC of convolutional layer.

  def count_conv2d(layers, log=False):
    if log:
       print(layers.get_config())
    #number of conv operations = input_h * input_w / stride = output^2
    numshifts = int(layers.output_shape[1] * layers.output_shape[2])

    # MAC/convfilter = kernelsize^2 * InputChannels * OutputChannels
    MACperConv = layers.get_config()["kernel_size"][0] * layers.get_config()["kernel_size"][1] * 
    layers.input_shape[3] * layers.output_shape[3]
    if layers.get_config()["use_bias"]:
        ADD = layers.output_shape[3]
    else:
        ADD = 0
    return MACperConv * numshifts * 2 + ADD

The formula for calculating the MACC operation in the literature review and on online resources is listed below. I would like to ask what is the purpose of ADD in the above function definition?

   K × K × Cin × Hout × Wout × Cout

Thanks, help is highly appreciated.

Solution

When using bias, there is an additional summation (+) operation. The bias vector is as long as there are filters/channels in the convolution layer. After performing the convolution operation, biases are added to every channel of the result.

So if there is bias (use_bias is True), then you have to count the addition operations (+) too and add them to the result. If there is no bias used, you perform 0 addition operation, so you add 0 to the result.