pythonneural-networktime-seriestf.keras

MLP Loss calculated as NaN


I am trying to prepare a MLP model for a time-series data. Its parameters are:

  1. Model is Sequential
  2. Layers: a- Hidden layer: One dense layer, with 100 nodes
    • input_dim = flattened array of my input whose dimensions are (number of timesteps * number of variables). In my case it is (10*2)
    • Activation function: relu b- Output layer: dense layer with only one node
  3. Compile: optimizer is "adam" and loss is "mse"

Here is my complete code for the building of the MLP:

# preparing the model
# flatten input
n_input = X.shape[1] * X.shape[2]
X = X.reshape((X.shape[0], n_input))

import tensorflow as tf
from tensorflow import keras
from keras.models import Sequential
from keras.layers import Dense

# define model
model = Sequential()
model.add(Dense(100, activation='relu', input_dim=n_input))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
# fit model
model.fit(X, y, epochs=100, verbose= 'auto')

Details on my data shape and sample:

[[[71.22222222  0.57692308]
  [71.89583333  0.9       ]
  [71.40138889  0.96666667]
  ...
  [70.63630437  3.06666667]
  [65.2247619   3.86666667]
  [65.22804233  3.66666667]]

 [[71.89583333  0.9       ]
  [71.40138889  0.96666667]
  [71.36721612  1.03333333]
  ...
  [65.2247619   3.86666667]
  [65.22804233  3.66666667]
  [67.0974359   2.2       ]]

 [[71.40138889  0.96666667]
  [71.36721612  1.03333333]
  [70.44217687  1.96666667]
  ...
  [65.22804233  3.66666667]
  [67.0974359   2.2       ]
  [68.32787879  3.26666667]]

y:

[ 90.  62.  89. ... 225. 211. 221.]

Problem: During training, my loss is being reported as nan. Here is a sample of the output:

Epoch 1/100
80677/80677 [==============================] - 187s 2ms/step - loss: nan
Epoch 2/100
80677/80677 [==============================] - 182s 2ms/step - loss: nan
Epoch 3/100
35140/80677 [============>.................] - ETA: 1:43 - loss: nan

I am expecting that the loss be a numerical value, not NaN. I made sure that my data types are of the correct format. Running x.dtype and y.dtype return dtype('float64').

I thought of normalizing my outputs, but I don't see how this will help my case as I did not need to normalize my output for uni-variate MLP.


Solution

  • Ensuring the replacement or removing of any NaN values worked for me. I am no longer getting a loss value of "NaN". I have also standardized both the inputs and outputs of my MLP. Below is the code I used:

    1. Removing NaN values:

      nan_check = df[['volume', 'speed', 'occupancy']].isna().any(axis=1)
      
      df = df[~nan_check]
      
    2. Standardizing the input and output features:

      from sklearn.preprocessing import StandardScaler
      
      # Flatten your input data if it's 3D
      
      if X.ndim == 3:
      
       num_samples, num_time_steps, num_features = X.shape
       X = X.reshape(num_samples, num_time_steps * num_features)
      
      input_scaler = StandardScaler()
      
      X_scaled = input_scaler.fit_transform(X)  # X should be your input data
      
      #Create a scaler for the output data:
      
      output_scaler = StandardScaler()
      y_scaled = output_scaler.fit_transform(y.reshape(-1, 1))  # y should be your output data