I am trying to prepare a MLP model for a time-series data. Its parameters are:
Here is my complete code for the building of the MLP:
# preparing the model
# flatten input
n_input = X.shape[1] * X.shape[2]
X = X.reshape((X.shape[0], n_input))
import tensorflow as tf
from tensorflow import keras
from keras.models import Sequential
from keras.layers import Dense
# define model
model = Sequential()
model.add(Dense(100, activation='relu', input_dim=n_input))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
# fit model
model.fit(X, y, epochs=100, verbose= 'auto')
Details on my data shape and sample:
print(X.shape, y.shape)
gives (2581654, 10, 2) (2581654,)
[[[71.22222222 0.57692308]
[71.89583333 0.9 ]
[71.40138889 0.96666667]
...
[70.63630437 3.06666667]
[65.2247619 3.86666667]
[65.22804233 3.66666667]]
[[71.89583333 0.9 ]
[71.40138889 0.96666667]
[71.36721612 1.03333333]
...
[65.2247619 3.86666667]
[65.22804233 3.66666667]
[67.0974359 2.2 ]]
[[71.40138889 0.96666667]
[71.36721612 1.03333333]
[70.44217687 1.96666667]
...
[65.22804233 3.66666667]
[67.0974359 2.2 ]
[68.32787879 3.26666667]]
y:
[ 90. 62. 89. ... 225. 211. 221.]
Problem: During training, my loss is being reported as nan. Here is a sample of the output:
Epoch 1/100
80677/80677 [==============================] - 187s 2ms/step - loss: nan
Epoch 2/100
80677/80677 [==============================] - 182s 2ms/step - loss: nan
Epoch 3/100
35140/80677 [============>.................] - ETA: 1:43 - loss: nan
I am expecting that the loss be a numerical value, not NaN. I made sure that my data types are of the correct format. Running x.dtype
and y.dtype
return dtype('float64')
.
I thought of normalizing my outputs, but I don't see how this will help my case as I did not need to normalize my output for uni-variate MLP.
Ensuring the replacement or removing of any NaN values worked for me. I am no longer getting a loss value of "NaN". I have also standardized both the inputs and outputs of my MLP. Below is the code I used:
Removing NaN values:
nan_check = df[['volume', 'speed', 'occupancy']].isna().any(axis=1)
df = df[~nan_check]
Standardizing the input and output features:
from sklearn.preprocessing import StandardScaler
# Flatten your input data if it's 3D
if X.ndim == 3:
num_samples, num_time_steps, num_features = X.shape
X = X.reshape(num_samples, num_time_steps * num_features)
input_scaler = StandardScaler()
X_scaled = input_scaler.fit_transform(X) # X should be your input data
#Create a scaler for the output data:
output_scaler = StandardScaler()
y_scaled = output_scaler.fit_transform(y.reshape(-1, 1)) # y should be your output data