I am trying to implement a VGG19 model for image classification. To be honnest, I am not this confident about what I am doing. Whenever I try to run I have the following error (here full output):
D:\logiciels\pycharm\IDH_hiv\hiv_venv\Scripts\python.exe D:/logiciels/pycharm/IDH_hiv/main.py
2022-05-16 09:27:30.951597: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-05-16 09:27:30.951716: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
loading images...
2022-05-16 09:27:44.410794: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-05-16 09:27:44.411737: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cublas64_11.dll'; dlerror: cublas64_11.dll not found
2022-05-16 09:27:44.412877: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cublasLt64_11.dll'; dlerror: cublasLt64_11.dll not found
2022-05-16 09:27:44.414421: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cufft64_10.dll'; dlerror: cufft64_10.dll not found
2022-05-16 09:27:44.415896: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'curand64_10.dll'; dlerror: curand64_10.dll not found
2022-05-16 09:27:44.416760: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cusolver64_11.dll'; dlerror: cusolver64_11.dll not found
2022-05-16 09:27:44.417606: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cusparse64_11.dll'; dlerror: cusparse64_11.dll not found
2022-05-16 09:27:44.418467: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudnn64_8.dll'; dlerror: cudnn64_8.dll not found
2022-05-16 09:27:44.418593: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1850] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2022-05-16 09:27:44.419047: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Training the network...
Epoch 1/100
2022-05-16 09:27:47.130316: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 822083584 exceeds 10% of free system memory.
2022-05-16 09:27:47.371899: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 822083584 exceeds 10% of free system memory.
Traceback (most recent call last):
File "D:\logiciels\pycharm\IDH_hiv\main.py", line 21, in <module>
procedures.vgg19("VGG19_test")
File "D:\logiciels\pycharm\IDH_hiv\procedures.py", line 228, in vgg19
H = model.fit(aug.flow(trainX, trainY, batch_size=BS),
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\tensorflow\python\eager\execute.py", line 54, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:
Detected at node 'Equal' defined at (most recent call last):
File "D:\logiciels\pycharm\IDH_hiv\main.py", line 21, in <module>
procedures.vgg19("VGG19_test")
File "D:\logiciels\pycharm\IDH_hiv\procedures.py", line 228, in vgg19
H = model.fit(aug.flow(trainX, trainY, batch_size=BS),
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\engine\training.py", line 1384, in fit
tmp_logs = self.train_function(iterator)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\engine\training.py", line 1021, in train_function
return step_function(self, iterator)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\engine\training.py", line 1010, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\engine\training.py", line 1000, in run_step
outputs = model.train_step(data)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\engine\training.py", line 864, in train_step
return self.compute_metrics(x, y, y_pred, sample_weight)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\engine\training.py", line 957, in compute_metrics
self.compiled_metrics.update_state(y, y_pred, sample_weight)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\engine\compile_utils.py", line 459, in update_state
metric_obj.update_state(y_t, y_p, sample_weight=mask)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\utils\metrics_utils.py", line 70, in decorated
update_op = update_state_fn(*args, **kwargs)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\metrics.py", line 178, in update_state_fn
return ag_update_state(*args, **kwargs)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\metrics.py", line 729, in update_state
matches = ag_fn(y_true, y_pred, **self._fn_kwargs)
File "D:\logiciels\pycharm\IDH_hiv\hiv_venv\lib\site-packages\keras\metrics.py", line 4086, in sparse_categorical_accuracy
return tf.cast(tf.equal(y_true, y_pred), backend.floatx())
Node: 'Equal'
Incompatible shapes: [64,7,7] vs. [64,1]
[[{{node Equal}}]] [Op:__inference_train_function_2058]
2022-05-16 09:27:58.097496: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: FAILED_PRECONDITION: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]
Process finished with exit code 1
Here is my full code:
# the data is located in this data_dir
data_dir = "E:/HIV-AI Project/20220330 Trial Images/PNG"
# the output model and the graph is saved in this 'output_dir'
output_dir = "Output/"
print("loading images...")
data = []
labels = []
# grab the image paths and shuffle them
imagePaths = sorted(list(paths.list_images(data_dir)))
random.seed(2)
random.shuffle(imagePaths)
IMAGE_WIDTH, IMAGE_HEIGHT = 224, 224
for imagePath in imagePaths:
image = cv2.imread(imagePath)
image = cv2.resize(image, (IMAGE_WIDTH, IMAGE_HEIGHT))
# append the image to the data list
data.append(image)
# extract label from the image path and update the labels list
title = imagePath.split(os.path.sep)[1]
label = ""
if "NI" in title:
label = "NI"
else:
label = "INF"
labels.append(label)
# scale the raw pixel intensities to the range [0, 1]
data = np.array(data, dtype="float") / 255.0
labels = np.array(labels)
# Binarize labels
lb = LabelBinarizer()
labels = lb.fit_transform(labels)
# save the encoder to output directory
with open(os.path.join(output_dir, 'labels'), 'wb') as f:
pickle.dump(lb, f)
# Randomly split the data into test and train sets (15% test and 85% train)
trainX, testX, trainY, testY = train_test_split(data, labels, test_size=0.15, random_state=42)
# construct the image generator for data augmentation
aug = ImageDataGenerator(rotation_range=45, width_shift_range=0.1,
height_shift_range=0.1, shear_range=0.2, zoom_range=0.2,
horizontal_flip=True, fill_mode="nearest")
# initialize our VGG-like Convolutional Neural Network
model = tf.keras.applications.vgg19.VGG19(
include_top=False,
classes=2,
classifier_activation='softmax')
# initialize our initial learning rate, # of epochs to train for,and batch size
INIT_LR = 0.0007
EPOCHS = 100
BS = 64
# initialize the model and optimizers
opt = Adam(learning_rate=INIT_LR, beta_1=0.9, beta_2=0.999, amsgrad=False)
# compile the model with loss function, optimizer and the evaluating metrics
model.compile(loss="binary_crossentropy", optimizer=opt,
metrics=["accuracy"])
# train the network
print('Training the network...', np.shape(trainX), np.shape(trainY), np.shape(testX), np.shape(testY))
H = model.fit(aug.flow(trainX, trainY, batch_size=BS),
validation_data=(testX, testY), steps_per_epoch=len(trainX) // BS,
epochs=EPOCHS)
# Save the model locally for use later
model_path = os.path.join(output_dir, figname + '.h5')
model.save(model_path)
# evaluation
predictions = model.predict(testX, batch_size=32)
print(classification_report(testY.argmax(axis=1), predictions.argmax(axis=1), target_names=lb.classes_))
# plot the training loss and accuracy
N = np.arange(0, EPOCHS)
plt.figure()
plt.plot(N, H.history["loss"], label="train_loss")
plt.plot(N, H.history["val_loss"], label="val_loss")
plt.plot(N, H.history["acc"], label="train_acc")
plt.plot(N, H.history["val_acc"], label="val_acc")
plt.title("Training/Validation Loss and Accuracy")
plt.xlabel("Epoch #")
plt.ylabel("Loss/Accuracy")
plt.legend()
plt.savefig(os.path.join(output_dir, figname + '.png'))
I do understand that there is a problem with the shape of the batches, but I don't know how to solve this.
Here are the shapes of trainX, trainY, testX, testY : (408, 224, 224, 3) (408, 1) (72, 224, 224, 3) (72, 1)
Your mistake is include_top=False in your creation of your model. If you set this to False, the model will leave out the layers actually classifying and will have a last layer with dimensions of 7 by 7, hence your error while trying to compare those values with your labels.
Setting include_top=True should work!
Since you want to actually retrain the model on 2 classes you must do the following:
vgg = tf.keras.applications.vgg19.VGG19(weights='imagenet', include_top=False)
for layer in vgg.layers:
layer.trainable = False
x = Flatten()(vgg.output) #Output obtained on vgg19 is now flattened.
prediction = Dense(2, activation='softmax')(x) # We have 2 classes
#Creating model object
model = Model(inputs=vgg.input, outputs=prediction)
Continue by compiling the model and then training it.