• Corpus ID: 240354462

PCA-based Multi Task Learning: a Random Matrix Approach

  title={PCA-based Multi Task Learning: a Random Matrix Approach},
  author={Malik Tiomoko and Romain Couillet and Fr{\'e}d{\'e}ric Pascal},
The article proposes and theoretically analyses a computationally efficient multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes [7, 5]. The analysis reveals that (i) by default learning may dramatically fail by suffering from negative transfer, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks… 

Figures and Tables from this paper

Multi-task learning on the edge: cost-efficiency and theoretical optimality
A distributed multi-task learning (MTL) algorithm based on supervised principal component analysis (SPCA) is proposed, which is theoretically optimal for Gaussian mixtures and computationally cheap and scalable.


A Survey on MultiTask Learning
A survey for MTL is given, which classifies different MTL algorithms into several categories, including feature learning approach, low-rank approach, task clustering approaches, task relation learning approaches, and decomposition approach, and then discusses the characteristics of each approach.
Supervised Principal Component Analysis Via Manifold Optimization
This work presents a manifold optimization approach to SPCA that simultaneously solves the prediction and dimensionality reduction problems and explains nearly as much variation as PCA while outperforming existing methods in prediction accuracy.
Semi-Supervised Multitask Learning
Experimental results on real data sets demonstrate that semi-supervised MTL yields significant improvements in generalization performance over either semi- supervised single-task learning (STL) or supervised MTL.
Large Margin Multi-Task Metric Learning
This paper proposes an alternative formulation for multi-task learning by extending the recently published large margin nearest neighbor (1mnn) algorithm to the MTL paradigm and shows that it consistently outperforms single-task kNN under several metrics and state-of-the-art MTL classifiers.
Multi-task feature selection
This work proposes a novel type of joint regularization of the model parameters in order to couple feature selection across tasks and shows empirically that this approach outperforms independent `1-based feature selection on several datasets.
A Regularization Approach to Learning Task Relationships in Multitask Learning
A regularization approach to learning the relationships between tasks in multitask learning that can also describe negative task correlation and identify outlier tasks based on the same underlying principle is proposed.
Multi-stage multi-task feature learning
A non-convex formulation for multi-task sparse feature learning based on a novel regularizer is proposed and a detailed theoretical analysis is presented showing that MSMTFL achieves a better parameter estimation error bound than the convex formulation.
A Convex Formulation for Learning Task Relationships in Multi-Task Learning
This paper proposes a regularization formulation for learning the relationships between tasks in multi-task learning, called MTRL, which can also describe negative task correlation and identify outlier tasks based on the same underlying principle.
Large Dimensional Analysis and Improvement of Multi Task Learning
A large dimensional analysis of a simple but extremely powerful when carefully tuned, Least Square Support Vector Machine (LSSVM) version of MTL, in the regime where the dimension $p$ of the data and their number $n$ grow large at the same rate is conducted.
Recent Advances in Supervised Dimension Reduction: A Survey
The purpose of this paper is to summarize and organize the current developments in the field into three main classes: PCA-based, Non-negative Matrix Factorization (NMF)-based, and manifold-based supervised dimension reduction methods, as well as provide elaborated discussions on their advantages and disadvantages.