I am trying to perform LSTM on a Training dataset with below structure for forecasting
There are about 2 million such sets in the dataframe. So the shape will be something like (2M, 10 or 15, 3). It could be lesser if I categorize the data on other factors
Column A & B always start with 1 and then change (Generally 0-2 % either side, rarely 10%, never 20%). Column C is always between 0 and 100.
My question is : Do I still need to use minmax or Robust Scaler, even though my data is already fairly normalized within a range. Also if I should still use it, should the range be (0,1) or (-1,1)
It is important to use a scaler because your features have different ranges. For example, Column C is about 20 times larger than the other columns. In neural network-based models like LSTMs, this imbalance can lead to disproportionately large gradients for Column C, making the learning process inefficient.
If you have outliers, MinMaxScaler
may not be the best choice, as it is sensitive to extreme values. In that case, StandardScaler
or RobustScaler
would be better options. Regarding the scaling range, (0,1)
is common for MinMaxScaler
, but if your activation functions (e.g., tanh) work better with a symmetric range, (-1,1)
may be preferable.