Multi-Task Time Series Forecasting With Shared Attention

  title={Multi-Task Time Series Forecasting With Shared Attention},
  author={Zekai Chen and E Jiaze and Xiao Zhang and Hao Sheng and Xiuzhen Cheng},
  journal={2020 International Conference on Data Mining Workshops (ICDMW)},
  • Zekai Chen, E. Jiaze, +2 authors Xiuzhen Cheng
  • Published 1 November 2020
  • Computer Science, Mathematics
  • 2020 International Conference on Data Mining Workshops (ICDMW)
Time series forecasting is a key component in many industrial and business decision processes and recurrent neural network (RNN) based models have achieved impressive progress on various time series forecasting tasks. However, most of the existing methods focus on single-task forecasting problems by learning separately based on limited supervised objectives, which often suffer from insufficient training instances. As the Transformer architecture and other attention-based models have… Expand

Figures and Tables from this paper

Rethinking modeling Alzheimer's disease progression from a multi-task learning perspective with deep recurrent neural network.
A multi-task learning framework that can adaptively impute missing values and predict future progression over time from a subject's historical measurements is proposed that achieves 5.6%, 5.7%, 4.0% and 11.8% improvement with respect to mAUC, BCA and MAE (ADAS-Cog13 and Ventricles), respectively. Expand
Graph-Guided Network for Irregularly Sampled Multivariate Time Series
This work introduces RAINDROP, a graphguided network for learning representations of irregularly sampled multivariate time series that outperforms state-of-the-art methods by up to 11.4% and is used to classify time series and interpret temporal dynamics of three healthcare and human activity datasets. Expand


Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
A novel deep learning framework, namely Long- and Short-term Time-series network (LSTNet), to address this open challenge of multivariate time series forecasting, using the Convolution Neural Network and the Recurrent Neural Network to extract short-term local dependency patterns among variables and to discover long-term patterns for time series trends. Expand
DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks
DeepAR is proposed, a methodology for producing accurate probabilistic forecasts, based on training an auto regressive recurrent network model on a large number of related time series, with accuracy improvements of around 15% compared to state-of-the-art methods. Expand
Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
First, convolutional self-attention is proposed by producing queries and keys with causal convolution so that local context can be better incorporated into attention mechanism, and LogSparse Transformer is proposed, improving forecasting accuracy for time series with fine granularity and strong long-term dependencies under constrained memory budget. Expand
Recurrent Neural Network for Text Classification with Multi-Task Learning
This paper uses the multi-task learning framework to jointly learn across multiple related tasks based on recurrent neural network to propose three different mechanisms of sharing information to model text with task-specific and shared layers. Expand
Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting
Diffusion Convolutional Recurrent Neural Network (DCRNN), a deep learning framework for traffic forecasting that incorporates both spatial and temporal dependency in the traffic flow and evaluates the framework on two real-world large scale road network traffic datasets and observes consistent improvement. Expand
Deep Multi-Task Learning with Shared Memory for Text Classification
Experiments on two groups of text classification tasks show that the proposed architectures can improve the performance of a task with the help of other related tasks. Expand
Deep State Space Models for Time Series Forecasting
A novel approach to probabilistic time series forecasting that combines state space models with deep learning by parametrizing a per-time-series linear state space model with a jointly-learned recurrent neural network, which compares favorably to the state-of-the-art. Expand
Attention is All you Need
A new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely is proposed, which generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data. Expand
Regularized multi--task learning
An approach to multi--task learning based on the minimization of regularization functionals similar to existing ones, such as the one for Support Vector Machines, that have been successfully used in the past for single-- task learning is presented. Expand
Multi-task Sequence to Sequence Learning
The results show that training on a small amount of parsing and image caption data can improve the translation quality between English and German by up to 1.5 BLEU points over strong single-task baselines on the WMT benchmarks, and reveal interesting properties of the two unsupervised learning objectives, autoencoder and skip-thought, in the MTL context. Expand