pythonvalidationkerasscikit-learnk-fold

How to use K-Fold cross validation with DenseNet121 model


I am working on classification of images breast cancer using DensetNet121 pretrained model. I split the dataset into training, testing and validation. I want to apply k-fold cross validation. I used cross_validation from sklearn library, but I get the below error when I run the code. I tried to solve it but nothing solved the error. Anyone have idea how to solve this.

in_model = tf.keras.applications.DenseNet121(input_shape=(224,224,3),
                                            include_top=False,
                                             weights='imagenet',classes = 2)
in_model.trainable = False
inputs = tf.keras.Input(shape=(224,224,3))
x = in_model(inputs)
flat = Flatten()(x)
dense_1 = Dense(1024,activation = 'relu')(flat)
dense_2 = Dense(1024,activation = 'relu')(dense_1)
prediction = Dense(2,activation = 'softmax')(dense_2)
in_pred = Model(inputs = inputs,outputs = prediction)
validation_data=(valid_data,valid_labels)
#16
in_pred.summary()
in_pred.compile(optimizer = tf.keras.optimizers.Adagrad(learning_rate=0.0002), loss=tf.keras.losses.CategoricalCrossentropy(from_logits = False), metrics=['accuracy'])
history=in_pred.fit(train_data,train_labels,epochs = 3,batch_size=32,validation_data=validation_data)
model_result=cross_validation(in_pred, train_data, train_labels, 5)

The error:

TypeError: Cannot clone object '<keras.engine.functional.Functional object at 0x000001F82E17E3A0>'
(type <class 'keras.engine.functional.Functional'>): 
it does not seem to be a scikit-learn estimator as it does not implement a 'get_params' method.


Solution

  • Since your model is not a scikit-learn estimator, you won't be able to use sklearn's built-in cross_validate method.

    You can, however use k-fold to split your data into k-folds and get the metrics for each fold. We can use TF's built in model.evaluate, or sklearn's metrics here, too).

    from sklearn.model_selection import KFold
    
    in_model = tf.keras.applications.DenseNet121(
        input_shape=(224, 224, 3), include_top=False, weights="imagenet", classes=2
    )
    in_model.trainable = False
    inputs = tf.keras.Input(shape=(224, 224, 3))
    x = in_model(inputs)
    flat = Flatten()(x)
    dense_1 = Dense(1024, activation="relu")(flat)
    dense_2 = Dense(1024, activation="relu")(dense_1)
    prediction = Dense(2, activation="softmax")(dense_2)
    in_pred = Model(inputs=inputs, outputs=prediction)
    validation_data = (valid_data, valid_labels)
    # 16
    in_pred.summary()
    in_pred.compile(
        optimizer=tf.keras.optimizers.Adagrad(learning_rate=0.0002),
        loss=tf.keras.losses.CategoricalCrossentropy(from_logits=False),
        metrics=["accuracy"],
    )
    
    
    kf = KFold(n_splits=10)
    
    kf.get_n_splits(train_data)
    
    for i, (fold_train_index, fold_test_index) in enumerate(kf.split(train_data)):
        print(f"Fold {i}:")
        print(f"  Train: index={fold_train_index}")
        print(f"  Test:  index={fold_test_index}")
        history = in_pred.fit(
            train_data[fold_train_index],
            train_labels[fold_train_index],
            epochs=3,
            batch_size=32,
            validation_data=validation_data,
        )
    
        in_pred.evaluate(train_data[fold_test_index],train_labels[fold_test_index])