pythontensorflowkerasbert-language-model

Error: A KerasTensor is symbolic: it's a placeholder for a shape an a dtype. It doesn't have any actual numerical value


I have trying to recreate this tutorial that's found on tensorflow's docs. However, I've been having an error I cannot solve and seems to be related to the literal source code of the tutorial. Also, there seems to be a conflict of dependencies with the version of tensorflow that we use (2.14.*)

Code (on Google Colab, tried it to on Kaggle)

!pip install --upgrade pip
!pip install -U "tensorflow-text==2.14.*"
!pip install -U "tf-models-official==2.14.*"

import tensorflow as tf
from tensorflow.keras.datasets import imdb
import tensorflow_hub as hub
import tensorflow_text as text
from tensorflow.keras.callbacks import EarlyStopping

tfhub_handle_encoder = "https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-2_H-128_A-2/1"
tfhub_handle_preprocess = "https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3"

def build_classifier_model():
    text_input = tf.keras.layers.Input(shape=(), dtype=tf.string, name='text')

    preprocessing_layer = hub.KerasLayer(
        tfhub_handle_preprocess, name='preprocessing', trainable=False
    )

    encoder_layer = hub.KerasLayer(
        tfhub_handle_encoder, name='BERT_encoder', trainable=True
    )

    # Directly pass symbolic input to preprocessing layer (this is allowed now)
    processed_inputs = preprocessing_layer(text_input)
    encoder_outputs = encoder_layer(processed_inputs)

    # Use pooled_output from BERT encoder
    pooled_output = encoder_outputs['pooled_output']

    net = tf.keras.layers.Dropout(0.1)(pooled_output)
    net = tf.keras.layers.Dense(1, activation=None, name='classifier')(net)

    return tf.keras.Model(text_input, net)

classifier_model = build_classifier_model()

Error on execution:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-24-f031ba765f1f> in <cell line: 0>()
----> 1 classifier_model = build_classifier_model()

7 frames
/usr/local/lib/python3.11/dist-packages/keras/src/backend/common/keras_tensor.py in __array__(self)

ValueError: Exception encountered when calling layer 'preprocessing' (type KerasLayer).

A KerasTensor is symbolic: it's a placeholder for a shape an a dtype. It doesn't have any actual numerical value. You cannot convert it to a NumPy array.

Call arguments received by layer 'preprocessing' (type KerasLayer):
  • inputs=<KerasTensor shape=(None,), dtype=string, sparse=False, name=text>
  • training=None

Dependencies conflicts

ERROR: pip s dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
dopamine-rl 4.1.2 requires tf-keras>=2.18.0, but you have tf-keras 2.15.1 which is incompatible.
thinc 8.3.6 requires numpy<3.0.0,>=2.0.0, but you have numpy 1.26.4 which is incompatible.
grpcio-status 1.71.0 requires protobuf<6.0dev,>=5.26.1, but you have protobuf 4.25.7 which is incompatible.
jax 0.5.2 requires ml_dtypes>=0.4.0, but you have ml-dtypes 0.3.2 which is incompatible.
ydf 0.11.0 requires protobuf<6.0.0,>=5.29.1, but you have protobuf 4.25.7 which is incompatible.
tensorflow-decision-forests 1.11.0 requires tensorflow==2.18.0, but you have tensorflow 2.15.1 which is incompatible.
tensorflow-decision-forests 1.11.0 requires tf-keras~=2.17, but you have tf-keras 2.15.1 which is incompatible.
Successfully installed keras-2.15.0 ml-dtypes-0.3.2 numpy-1.26.4 protobuf-4.25.7 tensorboard-2.15.2 tensorflow-2.15.1 tensorflow-estimator-2.15.0 tensorflow-text-2.15.0 tf-keras-2.15.1 wrapt-1.14.1
WARNING: The following packages were previously imported in this runtime:
  [keras,ml_dtypes,tensorflow,tensorflow_text,tf_keras,wrapt]
You must restart the runtime in order to use newly installed versions.

Take into acccount that said tutorial encourages using a similar version

# A dependency of the preprocessing for BERT inputs
pip install -U "tensorflow-text==2.13.*"
pip install "tf-models-official==2.13.*"

Solution

  • Solved it by installing tensorflow-text 2.14.1 with no dependencies.

    !pip install --no-deps tensorflow==2.14.1 tf_keras==2.14.1 tensorflow-text==2.14.0
    

    Don't ask me why it works. Tried it by installing the version specified in the docs, and it didn't work either