Supervised feature selection in graphs with path coding penalties and network flows

@article{Mairal2013SupervisedFS,
  title={Supervised feature selection in graphs with path coding penalties and network flows},
  author={Julien Mairal and Bin Yu},
  journal={ArXiv},
  year={2013},
  volume={abs/1204.4539}
}
We consider supervised learning problems where the features are embedded in a graph, such as gene expressions in a gene network. In this context, it is of much interest to automatically select a subgraph with few connected components; by exploiting prior knowledge, one can indeed improve the prediction performance or obtain results that are easier to interpret. Regularization or penalty functions for selecting features in graphs have recently been proposed, but they raise new algorithmic… 

Figures and Tables from this paper

Graph-Structured Sparse Optimization for Connected Subgraph Detection
  • Baojian Zhou, F. Chen
  • Computer Science
    2016 IEEE 16th International Conference on Data Mining (ICDM)
  • 2016
TLDR
This paper explores efficient approximate projection oracles for connected subgraphs, and proposes two new efficient algorithms, namely, Graph-IHT and Graph-GHTP, to optimize a generic nonlinear objective function subject to connectivity constraint on the support of the variables.
Graph-Based Regularization for Regression Problems with Alignment and Highly Correlated Designs
TLDR
This work shows how the proposed graph-based regularization yields mean-squared error guarantees for a broad range of covariance graph structures, including block and lattice graphs, which are optimal for many specific covariance graphs.
Multiclass SVM with graph path coding regularization for face classification
TLDR
A structural knowledge is encoded as directed acyclic graph and a graph path penalty is incorporated to multiclass SVM and the learned classifiers not only improve the performance, but also help in the interpretation of the learned features.
Structured Sparsity with Group-Graph Regularization
TLDR
This paper proposes a g2-regularization that takes group and graph sparsity into joint consideration, and presents an effective approach for its optimization, showing that, enforcing group-graph sparsity lead to better performance than using group sparsity orGraph sparsity only.
Convex relaxations of penalties for sparse correlated variables with bounded total variation
TLDR
The application of the k-support penalty on the complex task of identifying predictive regions from low-sample high-dimensional fMRI brain data is extensively analysed, showing that this method is particularly useful compared to existing methods in terms of accuracy, interpretability, and stability.
Attribute prediction with long-range interactions via path coding
TLDR
The proposed AP2CP not only introduces structured sparsity penalties over paths on a directed acyclic graph, but also captures the intrinsical long-range dependent interactions between attributes.
Nonparametric Redundant Features Removal for Unsupervised Feature Selection : An Empirical Study based on Sparse Feature Graph
TLDR
This work proposes a graph based approach to search and remove redundant features automatically for high dimensional data using a sparse graph generated at feature side and is used to learn the redundant relationship among features.
Redundant features removal for unsupervised spectral feature selection algorithms: an empirical study based on nonparametric sparse feature graph
TLDR
This proposed redundant feature removal algorithm is treated as a data preprocessing approach for existing popular unsupervised spectral feature selection algorithms like multi-cluster feature selection (MCFS) which requires accurate cluster structure information based on samples.
Efficient RNA isoform identification and quantification from RNA-Seq data with network flows
TLDR
This work introduces a new technique called FlipFlop, which can efficiently tackle the sparse estimation problem on the full set of candidate isoforms by using network flow optimization, leading to better isoform identification while keeping a low computational cost.
...
...

References

SHOWING 1-10 OF 100 REFERENCES
Path Coding Penalties for Directed Acyclic Graphs
TLDR
This paper proposes structured sparsity penalties over paths on a DAG (called “path coding” penalties) and designs minimum cost flow formulations to compute the penalties and their proximal operator in polynomial time, allowing us in practice to efficiently select a subgraph with a small number of connected components.
Convex and Network Flow Optimization for Structured Sparsity
TLDR
Two different strategies are presented that show that the proximal operator associated with a sum of l∞-norms can be computed exactly in polynomial time by solving a quadratic min-cost flow problem, allowing the use of accelerated proximal gradient methods.
Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning
TLDR
The extensive simulations on synthetic datasets and datasets from the UCI repository show that efficiently exploring the large feature space through sparsity-inducing norms leads to state-of-the-art predictive performance.
Optimization with Sparsity-Inducing Penalties (Foundations and Trends(R) in Machine Learning)
TLDR
Optimization with Sparsity-Inducing Penalties presents optimization tools and techniques dedicated to such sparsity-inducing penalties from a general perspective and provides an extensive set of experiments to compare various algorithms from a computational point of view.
Efficient RNA isoform identification and quantification from RNA-Seq data with network flows
TLDR
This work introduces a new technique called FlipFlop, which can efficiently tackle the sparse estimation problem on the full set of candidate isoforms by using network flow optimization, leading to better isoform identification while keeping a low computational cost.
Smoothing Proximal Gradient Method for General Structured Sparse Learning
TLDR
This paper proposes a general optimization approach, called smoothing proximal gradient method, which can solve the structured sparse regression problems with a smooth convex loss and a wide spectrum of structured-sparsity-inducing penalties.
Optimization with Sparsity-Inducing Penalties
TLDR
This monograph covers proximal methods, block-coordinate descent, reweighted l2-penalized techniques, working-set and homotopy methods, as well as non-convex formulations and extensions, and provides an extensive set of experiments to compare various algorithms from a computational point of view.
Structured Variable Selection with Sparsity-Inducing Norms
TLDR
This work considers the empirical risk minimization problem for linear supervised learning, with regularization by structured sparsity-inducing norms defined as sums of Euclidean norms on certain subsets of variables, and explores the relationship between groups defining the norm and the resulting nonzero patterns.
Online Learning for Matrix Factorization and Sparse Coding
TLDR
A new online optimization algorithm is proposed, based on stochastic approximations, which scales up gracefully to large data sets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems.
Convex Relaxation for Combinatorial Penalties
TLDR
This paper considers the situation of a model simultaneously penalized by a set- function on the support of the unknown parameter vector which represents prior knowledge on supports, and regularized in Lp-norm, and shows that the natural combinatorial optimization problems obtained may be relaxed into convex optimization problems.
...
...