tensorflowdeep-learningtensorboardtensorflow-layers

How to use TensorBoard and summary operations with the tf.layers module


I have followed the TensorFlow Layers tutorial to create a CNN for MNIST digit classification using TensorFlow's tf.layers module. Now I'm trying to learn how to use TensorBoard from TensorBoard: Visualizing Learning. Perhaps this tutorial hasn't been updated recently, because it says its example code is a modification of that tutorial's and links to it, but the code is completely different: it manually defines a single-hidden-layer fully-connected network.

The TensorBoard tutorial shows how to use tf.summary to attach summaries to a layer by creating operations on the layer's weights tensor, which is directly accessible because we manually defined the layer, and attaching tf.summary objects to those operations. To do this if I'm using tf.layers and its tutorial code, I believe I'd have to:

  1. Modify the Layers tutorial's example code to use the non-functional interface (Conv2D instead of conv2d and Dense instead of dense) to create the layers
  2. Use the layer objects' trainable_weights() functions to get the weight tensors and attach tf.summary objects to those

Is that the best way to use TensorBoard with tf.layers, or is there a way that's more directly compatible with tf.layers and the functional interface? If so, is there an updated official TensorBoard tutorial? It would be nice if the documentation and tutorials were more unified.


Solution

  • You should be able to use the output of your tf.layers call to get the activations. Taking the first convolutional layer of the linked layers tutorial:

    # Convolutional Layer #1
    conv1 = tf.layers.conv2d(
        inputs=input_layer,
        filters=32,
        kernel_size=[5, 5],
        padding="same",
        activation=tf.nn.relu)
    

    You could do:

    tensor_name = conv1.op.name
    tf.summary.histogram(tensor_name + '/activation', conv1)
    

    Not sure if this is the best way, but I believe it is the most direct way of doing what you want.

    Hope this helps!