pythontensorflowmachine-learningkerastf.keras

How to get an Intermediate output of a keras model?

I want to see the intermediate output of a model for example:

import tensorflow as tf

inputs = tf.keras.Input(shape=(3,))
x = tf.keras.layers.Dense(4, activation=tf.nn.relu)(inputs)
x1 = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x)
outputs = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x1)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
model.compile()

The actual model is a lot complicated and I am just providing a snippet. Is there a way for me to set a breakpoint or similar in the forward pass to see the intermediate model output.

Solution

  • It might depend a bit on your actual setting but you could split your model via its layers - similar like you set up an autoencoder. And forward pass through the backbone, look at it -> pass through the head -> output.

    import tensorflow as tf

inputs = tf.keras.Input(shape=(3,))
x = tf.keras.layers.Dense(4, activation=tf.nn.relu)(inputs)
x1 = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x)
outputs = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x1)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
model.compile()

back = tf.keras.Sequential(model.layers[:2])
head = tf.keras.Sequential(model.layers[2:])

# Instead of doing model(input) you can now do
inter = back(input)
print(inter)
result = head(inter)

    Alternatively you could also define multiple outputs, which are a bit uglier to train but for testing purposes you can pull the trained weights to this cloned model

    inputs = tf.keras.Input(shape=(3,))
x = tf.keras.layers.Dense(4, activation=tf.nn.relu)(inputs)
x1 = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x)
outputs = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x1)
model = tf.keras.Model(inputs=inputs, outputs=[outputs, x1]) #<-- adding your intermediate layer as a second output
model.compile()