Create future dataframe with neuralprophet when using autoregression

I tried adding future datafame using

future = m.make_future_dataframe(df, periods= 720, n_historic_predictions=True)

However, only 1 row is added to the dataframe, instead of 720 rows.

Model code :

m = NeuralProphet(
    changepoints_range=0.95,
    normalize='soft', 
    n_changepoints=50,
    trend_reg=1,
    yearly_seasonality=False,
    weekly_seasonality=True,
    daily_seasonality=10,
    n_lags = 7 # If i remove this line the future dataframe works just fine
)
m.add_seasonality(name="monthly", period=30.5, fourier_order=5)
m.set_plotting_backend("plotly")
metrics = m.fit(df, freq="H")

The documentation didnt mention anything that AR would affect make_future_dataframe feature https://neuralprophet.com/code/forecaster.html

Extends dataframe a number of periods (time steps) into the future.Only use if you predict into the unknown future. New timestamps are added to the historic dataframe, with the ‘y’ column being NaN, as it remains to be predicted. Further, the given future events and regressors are added to the periods new timestamps. The returned dataframe will include historic data needed to additionally produce n_historic_predictions, for which there are historic observances of the series ‘y’.

Solution

I came to this problem before. What I needed to realize is that by default, auto regression only predicts one step ahead. This is actually mentioned in documentation, but easy to overlook. This is the reason why only one row is added.

If you want to predict more than one next step, use n_forecasts parameter when calling the NeuralProphet constructor.

Since you need a 720-day prediction for each consecutive prediction, the model also needs last predictions. So to create 720 predictions into future, you need to run "next step" prediction 720 times, always adding the last prediction into the list of lags.

One more important thing that happened to me and cost me few days of headache: I used the validation dataframe for predicting the future, as shown in the documentation, and got success result that I was not able to reproduce since. The problem was that, if you provide a 720-line dataframe, containing real 'y' values, NeuralProphet does 720 individual predictions in silence using the last n lags of real 'y' each time. This creates the illusion that the model can predict 720 days into future at once... which it can not, because in reality you need new predictions to create each newer prediction.