[SOLVED] Low Validation Score on Pretrained Alexnet from Pytorch models for ImageNet 2012 dataset

Low Validation Score on Pretrained Alexnet from Pytorch models for ImageNet 2012 dataset

I am using pre-trained AlexNet network to validate some prior work.

The code is as follows:

import os
import torch

import torchvision
import torchvision.datasets as datasets
import torchvision.models as models
import torchvision.transforms as transforms


model = torch.hub.load('pytorch/vision:v0.6.0', 'alexnet', pretrained=True)
model.eval()

batchsize = 50000
workers = 1
dataset_path = 'data/imagenet_2012/'
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
val_data = datasets.ImageFolder(root=os.path.join(dataset_path, 'val'), transform=transforms.Compose( [transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(), normalize,]))
val_loader = torch.utils.data.DataLoader(val_data, batch_size=batchsize, num_workers=workers)


batch = next(iter(val_loader))
images, labels = batch

with torch.no_grad():
    output = model(images)


for i in output:
    out_soft = torch.nn.functional.softmax(i, dim=0)
    print(int(torch.argmax(out_soft)))

When I execute this and compare with ILSVRC2012_validation_ground_truth.txt, I get top-1 accuracy of 5% only.

What am I doing wrong here?

Thank you.

Solution

So, Pytorch/Caffe have their own "ground truth" files, which can be obtained from here: https://gist.github.com/ksimonyan/fd8800eeb36e276cd6f9#note

I manually tested the images in the validation folder of the Imagenet dataset against the val.txt file in the tar file provided at the link above to verify the order.

Update: New validation accuracy based on the groundtruth in the zip file in the link:

Top_1 = 56.522%

Top_5 = 79.066%