How to access intermediate layers in a pretrained model after retraining it on a new image dataset

I trained a VGG16 model on a labeled image dataset using the categorical crossentropy. I removed the fully connected layer and replaced it with new layers, as follows:

vgg16_model = VGG16(weights="imagenet", include_top=False, input_shape=(224, 224, 3))
model = Sequential()
model.add(vgg16_model)
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(70, activation='softmax'))

Then I trained the full model on my dataset. I want to use it now as a feature extraction model by extracting image features using any of the intermediate layers that belong to vgg16_model.

After training and saving the model, I can only access the layers that I added Dense and Dropout using pop() function to remove them and only keep the trained feature extractor model (vgg16).

i = 0
while i < 4: 
    model.pop()

This keeps only VGG16 model. However, the layers inside are not accessible, I tried:

new_model = Model(model.input,model.layers[-1].output)

But I get this error:

ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 224, 224, 3), dtype=tf.float32, name='input_9'), name='input_9', description="created by layer 'input_9'") at layer "block1_conv1". The following previous layers were accessed without issue: []

How can I modify my model to consider early k layers at a given time, then use the model for prediction?

Defining the keras model in the way you did, unfortunately has many complications when we tried to features from intermediate layer. I've raised a ticket, see here.

As you want to extract image features using any of the intermediate layers that belong to vgg16_model, you can try the following approach.

# [good practice]: check first to know name and shape
# for layer in model.layers:
#     print(layer.name, layer.output_shape)

# vgg16 (None, 7, 7, 512)
# flatten (None, 25088)
# dense (None, 256)
# dropout (None, 256)
# dense_1 (None, 70)

# get the trained model first
trained_vgg16 = keras.Model(
    inputs=model.get_layer(name="vgg16").inputs,
    outputs=model.get_layer(name="vgg16").outputs,
)

x = tf.ones((1, 224, 224, 3))
y = trained_vgg16(x)
y.shape
TensorShape([1, 7, 7, 512])

Next, use this trained_vgg16 model to build the target model. For example,

# extract only 1 intermediate layer
feature_extractor_block3_pool = keras.Model(
    inputs=trained_vgg16.inputs,
    outputs=trained_vgg16.get_layer(name="block3_pool").output,
)

# or, 2 based on purpose.
feature_extractor_block3_pool_block4_conv3 = keras.Model(
    inputs=trained_vgg16.inputs,
    outputs=[
        trained_vgg16.get_layer(name="block3_pool").output,
        trained_vgg16.get_layer(name="block4_conv3").output,
    ],
)

# or, all
feature_extractor = keras.Model(
    inputs=trained_vgg16.inputs,
    outputs=[layer.output for layer in trained_vgg16.layers],
)