Temporal Fusion Transformer

The Temporal Fusion Transformer (Lim, Arik, Loeff, Pfister, 2021, International Journal of Forecasting) is Google's attention-based architecture for multi-horizon forecasting that refuses to be a black box. It handles the full mix of real-world inputs in one network: static covariates, known future inputs, and past-observed series. Variable selection networks learn which inputs matter per step, gated residual networks suppress unneeded components, LSTM encoders handle local processing, and an interpretable multi-head attention layer exposes which past time steps the forecast depends on. Quantile loss outputs give full probabilistic forecasts.

TFT was designed for retail demand, electricity, traffic, and volatility forecasting, and the authors report significant gains over existing benchmarks on all four. Google released the reference code in google-research; the model is more widely used through pytorch-forecasting's production-grade implementation. It is heavier to train and tune than simpler MLP forecasters like N-HiTS, and Google does not ship pretrained weights or a hosted API, so every deployment trains from scratch.

Author-reported (Lim et al., IJF 2021): significant accuracy improvements over existing benchmarks on electricity, traffic, retail, and volatility datasets; independent evaluations keep it competitive with, but not ahead of, newer patch-based transformers and foundation models.