caffepycaffe

Why is the accuracy reported by Caffe accuracy layer differs from the one reported in pycaffe?


I have an accuracy layer for both training phase and testing phase. I also tried to train Caffe form PyCaffe so that I can better plot the curves. However I noticed, the accuracy returned using the

solver.test_nets[0].blobs['accuracy'].data

is different than the one I calculate myself by :

def run_test(solver, test_iter):
    '''
    Tests the network on all test set and calculates the test accuracy
    '''
    correct = 0
    batch_size_test =  solver.test_nets[0].blobs['data'].data.shape[0]

    for test_it in range(test_iter):
        #testing the network on all test set and calculate the test accuracy
        solver.test_nets[0].forward()
        correct += sum(solver.test_nets[0].blobs['ip1'].data.argmax(1) == solver.test_nets[0].blobs['label'].data)
     
    acc = correct / (test_iter * batch_size_test)
    return acc

The accuracy that run_test returns, is the same as the one Caffe reports on the console screen. What's the issue here? I also have this problem with the training phase accuracy and loss, meaning again

    train_loss[it] = solver.net.blobs['loss'].data
    training_acc = str(solver.net.blobs['accuracy_training'].data)

differ from the values reported by Caffe in the console screen.


Solution

  • I made a bad mistake here!. Everything is OK, except that I should only divide the accumulated accuracies by test_iter times :

    def run_test(solver, test_iter):
        '''
        Tests the network on all test set and calculates the test accuracy
        '''
        correct = 0
        batch_size_test =  solver.test_nets[0].blobs['data'].data.shape[0]
    
        for test_it in range(test_iter):
            #testing the network on all test set and calculate the test accuracy
            solver.test_nets[0].forward()
            correct += solver.test_nets[0].blobs['accuracy'].data
    
        acc = correct / test_iter
        return acc 
    

    The snippet :

    solver.test_nets[0].blobs['accuracy'].data
    

    will yield the accuracy for a single batch and obviously in order to get the accuracy for the whole test set, they need to be accumulated for test_iter times, and subsequently divided by test_iter.