I'm using a script that comes with TF-Slim to validate my trained model. It works fine but I'd like to get a list of the misclassified files.
The script makes use of https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/slim/python/slim/evaluation.py but even there I cannot find any options for printing the misclassified files.
How can I achieve that?
At a high level, you need to do 3 things:
1) Get your filename from the data loader. If you are using a tf-slim dataset from tfrecords, it is likely that the filenames are not stored in the tfrecord so you may be out of luck there. However if you are consuming image files directly from the filesystem with tf.WholeFileReader, then you can get the tensor of filenames where you form your batch:
def load_data():
train_image_names = ... # list of filenames
filename_queue = tf.train.string_input_producer(train_image_names)
reader = tf.WholeFileReader()
image_filename, image_file = reader.read(filename_queue)
image = tf.image.decode_jpeg(image_file, channels=3)
.... # load your labels from somewhere
return image_filename, image, label
# in your eval code
image_fn, image, label = load_data()
filenames, images, labels = tf.train.batch(
[image_fn, image, label],
batch_size=32,
num_threads=2,
capacity=100,
allow_smaller_final_batch=True)
2) Mask your filename tensor with your result after doing inference:
logits = my_network(images)
preds = tf.argmax(logits, 1)
mislabeled = tf.not_equal(preds, labels)
mislabeled_filenames = tf.boolean_mask(filenames, mislabeled)
3) Put all this into your eval_op:
eval_op = tf.Print(eval_op, [mislabeled_filenames])
slim.evaluation.evaluate_once(
.... # other options
eval_op=eval_op,
.... # other options)
I don't have a setup to test this, unfortunately. Let me know if it works!