I'm using Tensorflow to develop a simple ML model in Python. The code is below:
import tensorflow as tf
import pandas as pd
# Load CSV Data
def load_data(filename):
data = pd.read_csv(filename)
X = data[['X0','X1','X2','X3']]
Y = data[['Y0','Y1']]
return tf.data.Dataset.from_tensor_slices((X.values, Y.values))
training_data = load_data("binarydatatraining.csv")
print(training_data)
# Build a simple neural network model
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(4, activation='relu'),
tf.keras.layers.Dense(2)
])
# Compile the model
model.compile(optimizer='adam',
loss='mean_squared_error')
# Load validation data
validation_data = load_data("binarydatavalidation.csv")
print(validation_data)
# Train the model
model.summary()
model.fit(training_data.batch(9), epochs=5)
model.summary()
model.fit(training_data.batch(9), epochs=1, validation_data = validation_data, validation_steps = 2)
Everything runs perfectly until I start to include the validation data, which has the same number of parameters as the training data. Then I get the error
ValueError: Exception encountered when calling Sequential.call().
[1mInvalid input shape for input Tensor("sequential_1/Cast:0", shape=(4,), dtype=float32). Expected shape (None, 4), but input has incompatible shape (4,)[0m
Arguments received by Sequential.call():
• inputs=tf.Tensor(shape=(4,), dtype=int64)
• training=False
• mask=None
Printing the validation and training datasets shows that they have the same dimension, and running print(training_data) and print(validation_data) both give
<_TensorSliceDataset element_spec=(TensorSpec(shape=(4,), dtype=tf.int64, name=None), TensorSpec(shape=(2,), dtype=tf.int64, name=None))>
How do I correctly set up the validation data to run inline with the model.fit?
The error suggests that the shape of the input data is not compatible with what the model expects. The model expects input data with a shape of (None, 4), but it's receiving input with the shape (4,). To fix this, add input layer to your model with the shape (4,) and batch the validation data using .batch() before passing it to model.fit() as below:
# Build a simple neural network model
model = tf.keras.models.Sequential([
tf.keras.layers.Input(shape=(4,)), # added Input layer
tf.keras.layers.Dense(4, activation='relu'),
tf.keras.layers.Dense(2)
])
# Load and batch the training/validation data
training_data = load_data("binarydatatraining.csv").batch(9)
print(training_data)
validation_data = load_data("binarydatavalidation.csv").batch(9)
print(validation_data)
Please find the attached replicated gist for your reference. Thank you.