I am trying to hypertune the input shape of an LSTM model based on the different values of timesteps. However, I am facing an issue. While initializing the model, the default value of timesteps (which is 2) is chosen, and accordingly, the build_model.scaled_train is created of shape (4096, 2, 64). Thus the value of input_shape during initialization is (2, 64). When the training starts and the value of timesteps is arbitrarily chosen as 16, then build_model.scaled_train has shape (512, 16, 64). This means that input_shape now takes the value (16, 64). However, this is not reflected in the model. The InputLayer retains the shape (2, 64) it got during initialization. Hence, an error - Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 16, 64), found shape=(None, 2, 64).
def build_model(hp):
timesteps = hp.Choice('timesteps', [2, 4, 8, 16], ordered = False)
....
DFS, UFS = get_data_in_shape(DF, UF, timesteps)
build_model.scaled_train, build_model.train_label = train_test_splitting(DFS, UFS)
model = keras.Sequential()
model.add(InputLayer(input_shape = (timesteps, nosamples))
...
...
return model
class MyTuner(BayesianOptimization):
def run_trial(self, trial, *args, **kwargs):
kwargs['batch_size'] = trial.hyperparameters.Choice('batch_size', [32, 64, 128, 256])
return super(MyTuner, self).run_trial(trial, *args, **kwargs)
tuner = MyTuner(
build_model,
objective ='val_loss',
max_trials = 20,
overwrite = True,
directory = '/content/drive/MyDrive/Colab Notebooks',
project_name = 'bo4')
When I start hyperparameter tuning, this happens.
tuner.search(build_model.scaled_train, build_model.train_label, validation_split = 0.2, epochs = 100, callbacks = [early_stopping])
Error -
Search: Running Trial #1
Value |Best Value So Far |Hyperparameter
16 |? |timesteps
4 |? |layers
1024 |? |unitsLSTM
0.15 |? |rate
64 |? |unitsANN
0.001 |? |learning_rate
Epoch 1/100
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-76-10e3851dd45f> in <module>()
----> 1 tuner.search(build_model.scaled_train, build_model.train_label, validation_split = 0.2, epochs = 100, callbacks = [early_stopping]) #, model_checkpoint
6 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
1145 except Exception as e: # pylint:disable=broad-except
1146 if hasattr(e, "ag_error_metadata"):
-> 1147 raise e.ag_error_metadata.to_exception(e)
1148 else:
1149 raise
ValueError: in user code:
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1021, in train_function *
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1010, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1000, in run_step **
outputs = model.train_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 859, in train_step
y_pred = self(x, training=True)
File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py", line 264, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" is '
ValueError: Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 16, 64), found shape=(None, 2, 64)
I think I am making a logical mistake somewhere but cannot find it. Please help.
I made some changes which are written below and it worked fine. But I don't know if it is the optimal solution.
def build_model(hp):
...
...
scaled_train, train_label = train_test_splitting(DFS, UFS)
...
...
return model, scaled_train, train_label
class MyTuner(BayesianOptimization):
def run_trial(self, trial, *args, **kwargs):
hp = trial.hyperparameters
model, scaled_train, train_label = self.hypermodel.build(hp)
kwargs['batch_size'] = trial.hyperparameters.Choice('batch_size', [32, 64, 128, 256])
return self.hypermodel.fit(hp, model, scaled_traintrain, train_label, *args, **kwargs)