RNN and LSTM Forecasting
RNN forecasters process a sequence one time step at a time while carrying a hidden state. For a simple recurrent model,
The input can include the target history, observed covariates, entity embeddings, and calendar features. The hidden state is the learned summary of the past that the forecast head uses for one or more future horizons.
LSTMs and GRUs modify the recurrent update with gates that control what to keep, forget, and expose. This helps with longer dependencies compared with a plain RNN, where gradients can vanish or explode through many repeated updates. In forecasting, the practical distinction is not just “long memory”; it is whether the model can learn useful state transitions from enough leakage-free historical windows.
There are several output designs. A recursive model predicts one step and feeds that prediction back for later horizons. A direct multi-horizon model emits the full horizon at once. Encoder-decoder models read a history window and decode future steps, often with known future covariates such as holidays or planned prices. DeepAR-style models use recurrent state to parameterize a predictive distribution rather than only a point forecast, connecting recurrent forecasting to probabilistic forecasting.
RNNs and LSTMs can underperform simpler feature engineering for forecasting when series are short, seasonality is easy to encode, or covariate leakage is present. Their strongest use case is usually global learning across many related sequences where a shared recurrent representation can transfer behavior between entities.
Connections
RNN/LSTM forecasters are one branch of deep learning forecasting. They share sequence-modeling concerns with temporal convolutional networks, transformer-based forecasting, and the deep-learning pages on recurrent neural networks and LSTM and GRU.
References
- Salinas, Flunkert, and Gasthaus, DeepAR
- PyTorch LSTM documentation
- Nixtla NeuralForecast documentation
Nav