[SOLVED] Caffe and LMDB - generate validation set from training data.mdb files

Caffe and LMDB - generate validation set from training data.mdb files

In my dataset folder, I have a directory with my training data images converted to lmdb, like so:

train_lmdb/
   -- data.mdb
   -- lock.mdb

but in order to generate a custom model, I also need a folder with my validation set, like so:

val_lmdb/
   -- data.mdb
   -- lock.mdb

How do I generate validation data.mdb and lock.mdb from the training set files above?

Is it possible to split a portion of training data into new validation files? I haven't found any documentation on the matter.

Solution

By editing the script for the imageNet example at:

$CAFFE_ROOT/examples/imagenet/create_imagenet.sh

you will be able to generate corresponding training and validation lmdb's.

Change the paths to where your train.txt and val.txt live, along with the path for their matching image files, and you should produce two folders that contain your respective databases.