Train new dataset on transfer learning pre-trained model

Code:

from keras.preprocessing import image as image_util 
from keras.applications.imagenet_utils import preprocess_input
from keras.applications.imagenet_utils import decode_predictions
from keras.applications import ResNet50
import numpy as np 
import argparse
import cv2
import time 

ap = argparse.ArgumentParser()
ap.add_argument("-i","--image",required= True,help ="path of the image")
args = vars(ap.parse_args())

# orig = cv2.imread(args["image"]) #Opencv function to load a image
start_time = time.time()
image = image_util.load_img(args["image"],target_size=(224,224))
image = image_util.img_to_array(image)

#print("!!!!!.....!!!!")
print(image.shape)


image = np.expand_dims(image,axis=0) #(224,224,3) --> (1,224,224,3)
#print("!!!!!.....!!!!")
print(image.shape)
image = preprocess_input(image)

#Loading the model 
model = ResNet50(weights="imagenet")
pred = model.predict(image)
#print("111!!!!!.....!!!!")
#print(pred)
p = decode_predictions(pred)
#print("222!!!!!.....!!!!")
#print(p)

for (i,(imagenetID,label,prob)) in enumerate(p[0]):
    print("{}. {}: {:.2f}%".format(i+1, label, prob*100))

ans = p[0][0]
ans = ans[1]
print("THE PREDICTED IMAGE IS: "+ans)

orig = cv2.imread(args["image"]) #Opencv function to load a image
(imagenetID,label,prob) = p[0][0]
cv2.putText(orig, "{},{:.2f}%".format(label,prob*100),(10,30),cv2.FONT_HERSHEY_COMPLEX,0.5,(0,0,0),1)
cv2.imshow("classification",orig)
cv2.waitKey(0)
print("--- %s seconds ---" % (time.time() - start_time))  

This code works on imagenet weight and has a pre-trained model that can classify various images. I need to train a new object i.e my own dataset . (say for eg. apple). What shall i do to update the weights adding my new dataset?

The general approach is to take only lower layers of a pretrained CNN (such as ResNet) and add new layers on top of the existing CNN.

Once you have your model, you probably should lock the pretrained layers at the beginning of the training so that you don't destroy those already trained weights and then after few cycles when the gradient stabilizes, you can unlock those layers and continue with training.

Easiest way to drop the top layers of the pretrained network is to set include_top argument to False.

base_model = ResNet50(include_top=False, weights="imagenet") 

Then you can start adding your layers as usual, i.e. (n_classes refers to the number of classes that you want to classify)

my_hidden1 = keras.layers.Dense(128, activation="relu")(base_model)
# rest of the custom layers
...
output = keras.layers.Dense(n_classes, activation="softmax")(previous_layer)
model = keras.Model(inputs=base_model.input, outputs=output)

To lock the pretrained layers at the beginning

for layer in base_model.layers:
    layer.trainable = False

Then you can compile and fit your new model for a few epochs (even with larger learning rate), i.e.

optimizer = keras.optimizers.SGD(lr=0.2, momentum=0.9, decay=0.01)
model.compile(optimizer=optimizer, ...)
model.fit(...)

After the initial training is done, you can unlock the base layers and continue with the training (usually, you want to reduce the learning rate at this stage).

for layer in base_model.layers:
    layer.trainable = True

optimizer = keras.optimizers.SGD(lr=0.01, momentum=0.9, decay=0.001)
model.compile(...)
model.fit(...)

Note that you will have to run compile each time you lock or unlock those layers.