• Corpus ID: 240354462

PCA-based Multi Task Learning: a Random Matrix Approach

@article{Tiomoko2021PCAbasedMT,
  title={PCA-based Multi Task Learning: a Random Matrix Approach},
  author={Malik Tiomoko and Romain Couillet and Fr{\'e}d{\'e}ric Pascal},
  journal={ArXiv},
  year={2021},
  volume={abs/2111.00924}
}
The article proposes and theoretically analyses a computationally efficient multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes [7, 5]. The analysis reveals that (i) by default learning may dramatically fail by suffering from negative transfer, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks… 

Figures and Tables from this paper

Multi-task learning on the edge: cost-efficiency and theoretical optimality

TLDR
A distributed multi-task learning (MTL) algorithm based on supervised principal component analysis (SPCA) is proposed, which is theoretically optimal for Gaussian mixtures and computationally cheap and scalable.

References

SHOWING 1-10 OF 59 REFERENCES

Regularized multi--task learning

TLDR
An approach to multi--task learning based on the minimization of regularization functionals similar to existing ones, such as the one for Support Vector Machines, that have been successfully used in the past for single-- task learning is presented.

Safe Screening for Multi-Task Feature Learning with Multiple Data Matrices

TLDR
A novel screening rule based on the dual projection onto convex sets (DPC) to quickly identify the inactive features--that have zero coefficients in the solution vectors across all tasks)--that are applicable to sparse models with multiple data matrices.

A Survey on MultiTask Learning

TLDR
A survey for MTL is given, which classifies different MTL algorithms into several categories, including feature learning approach, low-rank approach, task clustering approaches, task relation learning approaches, and decomposition approach, and then discusses the characteristics of each approach.

Supervised Principal Component Analysis Via Manifold Optimization

TLDR
This work presents a manifold optimization approach to SPCA that simultaneously solves the prediction and dimensionality reduction problems and explains nearly as much variation as PCA while outperforming existing methods in prediction accuracy.

Semi-Supervised Multitask Learning

TLDR
Experimental results on real data sets demonstrate that semi-supervised MTL yields significant improvements in generalization performance over either semi- supervised single-task learning (STL) or supervised MTL.

Large Margin Multi-Task Metric Learning

TLDR
This paper proposes an alternative formulation for multi-task learning by extending the recently published large margin nearest neighbor (1mnn) algorithm to the MTL paradigm and shows that it consistently outperforms single-task kNN under several metrics and state-of-the-art MTL classifiers.

A Regularization Approach to Learning Task Relationships in Multitask Learning

TLDR
A regularization approach to learning the relationships between tasks in multitask learning that can also describe negative task correlation and identify outlier tasks based on the same underlying principle is proposed.

Multi-stage multi-task feature learning

TLDR
A non-convex formulation for multi-task sparse feature learning based on a novel regularizer is proposed and a detailed theoretical analysis is presented showing that MSMTFL achieves a better parameter estimation error bound than the convex formulation.

A Convex Formulation for Learning Task Relationships in Multi-Task Learning

TLDR
This paper proposes a regularization formulation for learning the relationships between tasks in multi-task learning, called MTRL, which can also describe negative task correlation and identify outlier tasks based on the same underlying principle.

Large Dimensional Analysis and Improvement of Multi Task Learning

TLDR
A large dimensional analysis of a simple but extremely powerful when carefully tuned, Least Square Support Vector Machine (LSSVM) version of MTL, in the regime where the dimension $p$ of the data and their number $n$ grow large at the same rate is conducted.
...