Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting

@article{Lim2021TemporalFT,
  title={Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting},
  author={Bryan Lim and Sercan {\"O}. Arik and Nicolas Loeff and Tomas Pfister},
  journal={ArXiv},
  year={2021},
  volume={abs/1912.09363}
}

Figures and Tables from this paper

Attention-based Domain Adaptation for Time Series Forecasting

This work proposes a novel domain adaptation framework, Domain Adaptation Forecaster (DAF), that leverages the statistical strengths from another relevant domain with abundant data samples to improve the performance on the domain of interest with limited data (target).

Cross-domain Time Series Forecasting with Attention Sharing

This work proposes a novel domain adaptation framework, Domain Adaptation Forecaster (DAF), that leverages the statistical strengths from another relevant domain with abundant data samples to improve the performance on the domain of interest with limited data (target).

Comparative Analysis of Deep Learning Models for Multi-Step Prediction of Financial Time Series

Experimental results show that the latest deep learning models such as N-BEATS, ES-LSTM and TCN produced better results for all stock market related datasets by obtaining around 50% less Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) scores for each prediction horizon as compared to other models.

Multi-Task Time Series Forecasting With Shared Attention

This work proposes two self-attention based sharing schemes for multi-task time series forecasting which can train jointly across multiple tasks and shows that the proposed architectures can not only outperform the state-of-the-art single-task forecasting baselines but also outperforms the RNN-based multi- task forecasting method.

Time-series forecasting with deep learning: a survey

This article surveys common encoder and decoder designs used in both one-step-ahead and multi-horizon time-series forecasting—describing how temporal information is incorporated into predictions by each model.

Interpretable Multivariate Time Series Forecasting with Temporal Attention Convolutional Neural Networks

This paper proposes an architecture comprised of a temporal convolutional network with an attention mechanism that makes predictions while presenting the timesteps of the input that were most influential for future outputs, and applies it on two datasets and shows that it gains interpretability without degrading the accuracy compared to the original temporal convolved models.

The Effectiveness of Discretization in Forecasting: An Empirical Study on Neural Time Series Models

It is found that binning almost always improves performance (compared to using normalized real-valued inputs), but that the particular type of binning chosen is of lesser importance.

Series Saliency: Temporal Interpretation for Multivariate Time Series Forecasting

The series saliency framework for temporal interpretation for multivariate time series forecasting, which considers the forecasting interpretation in both feature and temporal dimensions, can be employed to any well-defined deep learning models and works as a data augmentation to get more accurate forecasts.

MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attention

A novel decoder-self attention scheme for forecasting that produces significant improvements in the excess variation of the forecast and a novel positional encoding that allows the neural network to learn context-dependent seasonality functions as well as arbitrary holiday distances.

A Transformer-based Framework for Multivariate Time Series Representation Learning

A novel framework for multivariate time series representation learning based on the transformer encoder architecture, which can offer substantial performance benefits over fully supervised learning on downstream tasks, both with but even without leveraging additional unlabeled data, i.e., by reusing the existing data samples.
...

References

SHOWING 1-10 OF 41 REFERENCES

Multi-Horizon Time Series Forecasting with Temporal Attention Learning

An end-to-end deep-learning framework for multi-horizon time series forecasting, with temporal attention mechanisms to better capture latent patterns in historical data which are useful in predicting the future is proposed.

Exploring Interpretable LSTM Neural Networks over Multi-Variable Data

The structure of LSTM recurrent neural networks to learn variable-wise hidden states is explored, with the aim to capture different dynamics in multi-variable time series and distinguish the contribution of variables to the prediction.

Deep Factors for Forecasting

A hybrid model that incorporates the benefits of both classical and deep neural networks is proposed, which is data-driven and scalable via a latent, global, deep component, and handles uncertainty through a local classical model.

Multiple-output modeling for multi-step-ahead time series forecasting

DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks

A Multi-Horizon Quantile Recurrent Forecaster

We propose a framework for general probabilistic multi-step time series regression. Specifically, we exploit the expressiveness and temporal nature of Recurrent Neural Networks, the nonparametric

Deep State Space Models for Time Series Forecasting

A novel approach to probabilistic time series forecasting that combines state space models with deep learning by parametrizing a per-time-series linear state space model with a jointly-learned recurrent neural network, which compares favorably to the state-of-the-art.

Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting

First, convolutional self-attention is proposed by producing queries and keys with causal convolution so that local context can be better incorporated into attention mechanism, and LogSparse Transformer is proposed, improving forecasting accuracy for time series with fine granularity and strong long-term dependencies under constrained memory budget.

Attend and Diagnose: Clinical Time Series Analysis using Attention Models

For the first time, attention models are utilized for clinical time-series modeling, thereby dispensing recurrence entirely and the SAnD (Simply Attend and Diagnose) architecture is developed, which employs a masked, self-attention mechanism, and uses positional encoding and dense interpolation strategies for incorporating temporal order.

Temporal Regularized Matrix Factorization for High-dimensional Time Series Prediction

This paper develops novel regularization schemes and uses scalable matrix factorization methods that are eminently suited for high-dimensional time series data that has many missing values, and makes interesting connections to graph regularization methods in the context of learning the dependencies in an autoregressive framework.