• Corpus ID: 240354462

PCA-based Multi Task Learning: a Random Matrix Approach

@article{Tiomoko2021PCAbasedMT,
  title={PCA-based Multi Task Learning: a Random Matrix Approach},
  author={Malik Tiomoko and Romain Couillet and Fr{\'e}d{\'e}ric Pascal},
  journal={ArXiv},
  year={2021},
  volume={abs/2111.00924}
}
The article proposes and theoretically analyses a computationally efficient multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes [7, 5]. The analysis reveals that (i) by default learning may dramatically fail by suffering from negative transfer, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks… 

Figures and Tables from this paper

Multi-task learning on the edge: cost-efficiency and theoretical optimality
TLDR
A distributed multi-task learning (MTL) algorithm based on supervised principal component analysis (SPCA) is proposed, which is theoretically optimal for Gaussian mixtures and computationally cheap and scalable.

References

SHOWING 1-10 OF 59 REFERENCES
A Survey on MultiTask Learning
TLDR
A survey for MTL is given, which classifies different MTL algorithms into several categories, including feature learning approach, low-rank approach, task clustering approaches, task relation learning approaches, and decomposition approach, and then discusses the characteristics of each approach.
Supervised Principal Component Analysis Via Manifold Optimization
TLDR
This work presents a manifold optimization approach to SPCA that simultaneously solves the prediction and dimensionality reduction problems and explains nearly as much variation as PCA while outperforming existing methods in prediction accuracy.
Semi-Supervised Multitask Learning
TLDR
Experimental results on real data sets demonstrate that semi-supervised MTL yields significant improvements in generalization performance over either semi- supervised single-task learning (STL) or supervised MTL.
Large Margin Multi-Task Metric Learning
TLDR
This paper proposes an alternative formulation for multi-task learning by extending the recently published large margin nearest neighbor (1mnn) algorithm to the MTL paradigm and shows that it consistently outperforms single-task kNN under several metrics and state-of-the-art MTL classifiers.
Multi-task feature selection
TLDR
This work proposes a novel type of joint regularization of the model parameters in order to couple feature selection across tasks and shows empirically that this approach outperforms independent `1-based feature selection on several datasets.
A Regularization Approach to Learning Task Relationships in Multitask Learning
TLDR
A regularization approach to learning the relationships between tasks in multitask learning that can also describe negative task correlation and identify outlier tasks based on the same underlying principle is proposed.
Multi-stage multi-task feature learning
TLDR
A non-convex formulation for multi-task sparse feature learning based on a novel regularizer is proposed and a detailed theoretical analysis is presented showing that MSMTFL achieves a better parameter estimation error bound than the convex formulation.
A Convex Formulation for Learning Task Relationships in Multi-Task Learning
TLDR
This paper proposes a regularization formulation for learning the relationships between tasks in multi-task learning, called MTRL, which can also describe negative task correlation and identify outlier tasks based on the same underlying principle.
Large Dimensional Analysis and Improvement of Multi Task Learning
TLDR
A large dimensional analysis of a simple but extremely powerful when carefully tuned, Least Square Support Vector Machine (LSSVM) version of MTL, in the regime where the dimension $p$ of the data and their number $n$ grow large at the same rate is conducted.
Recent Advances in Supervised Dimension Reduction: A Survey
TLDR
The purpose of this paper is to summarize and organize the current developments in the field into three main classes: PCA-based, Non-negative Matrix Factorization (NMF)-based, and manifold-based supervised dimension reduction methods, as well as provide elaborated discussions on their advantages and disadvantages.
...
...