• Corpus ID: 220128104

Transfer Learning via $\ell_1$ Regularization

  title={Transfer Learning via \$\ell\_1\$ Regularization},
  author={Masaaki Takada and Hironori Fujisawa},
  journal={arXiv: Machine Learning},
Machine learning algorithms typically require abundant data under a stationary environment. However, environments are nonstationary in many real-world applications. Critical issues lie in how to effectively adapt models under an ever-changing environment. We propose a method for transferring knowledge from a source domain to a target domain via $\ell_1$ regularization. We incorporate $\ell_1$ regularization of differences between source parameters and target parameters, in addition to an… 

Figures from this paper



Efficient Online and Batch Learning Using Forward Backward Splitting

The two phase approach enables sparse solutions when used in conjunction with regularization functions that promote sparsity, such as l1, l2, l22, and l∞ regularization, and is extended and given efficient implementations for very high-dimensional data with sparsity.

Stability and Hypothesis Transfer Learning

It is shown that the relatedness of source and target domains accelerates the convergence of the Leave-One-Out error to the generalization error, thus enabling the use of the leave- one- out error to find the optimal transfer parameters, even in the presence of a small training set.

Fast rates by transferring from auxiliary hypotheses

This work focuses on a broad class of ERM-based linear algorithms that can be instantiated with any non-negative smooth loss function and any strongly convex regularizer, and establishes generalization and excess risk bounds.

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

This work describes and analyze an apparatus for adaptively modifying the proximal function, which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal functions that can be chosen in hindsight.

A Survey on Transfer Learning

The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.

Statistical Learning with Sparsity: The Lasso and Generalizations

Statistical Learning with Sparsity: The Lasso and Generalizations presents methods that exploit sparsity to help recover the underlying signal in a set of data and extract useful and reproducible patterns from big datasets.

Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization

  • Lin Xiao
  • Computer Science
    J. Mach. Learn. Res.
  • 2009
A new online algorithm is developed, the regularized dual averaging (RDA) method, that can explicitly exploit the regularization structure in an online setting and can be very effective for sparse online learning with l1-regularization.

Online sequential extreme learning machine in nonstationary environments

Sparse Online Learning via Truncated Gradient

This work proposes a general method called truncated gradient to induce sparsity in the weights of online-learning algorithms with convex loss and finds for datasets with large numbers of features, substantial sparsity is discoverable.

Learning in Nonstationary Environments: A Survey

In such nonstationary environments, where the probabilistic properties of the data change over time, a non-adaptive model trained under the false stationarity assumption is bound to become obsolete in time, and perform sub-optimally at best, or fail catastrophically at worst.