Learning with Submodular Functions: A Convex Optimization Perspective

@article{Bach2013LearningWS,
  title={Learning with Submodular Functions: A Convex Optimization Perspective},
  author={Francis R. Bach},
  journal={ArXiv},
  year={2013},
  volume={abs/1111.6453}
}
  • F. Bach
  • Published 2013
  • Computer Science, Mathematics
  • ArXiv
Submodular functions are relevant to machine learning for at least two reasons: (1) some problems may be expressed directly as the optimization of submodular functions and (2) the Lovsz extension of submodular functions provides a useful set of regularization functions for supervised and unsupervised learning. In Learning with Submodular Functions: A Convex Optimization Perspective, the theory of submodular functions is presented in a self-contained way from a convex analysis perspective… Expand
Learning and Optimization with Submodular Functions
TLDR
The formal definition of submodularity is reviewed; the optimization of sub modular functions, both maximization and minimization are reviewed; and some applications in relation to learning and reasoning using submodular functions are discussed. Expand
Convex Analysis for Minimizing and Learning Submodular Set Functions
The connections between convexity and submodularity are explored, for purposes of minimizing and learning submodular set functions. First, we develop a novel method for minimizing a particular classExpand
Submodular functions: from discrete to continuous domains
  • F. Bach
  • Computer Science, Mathematics
  • Math. Program.
  • 2019
TLDR
This paper shows that most results relating submodularity and convexity for set-functions can be extended to all submodular functions, and provides practical algorithms which are based on function evaluations, to minimize generic sub modular functions on discrete domains, with associated convergence rates. Expand
Quadratic Decomposable Submodular Function Minimization: Theory and Practice
We introduce a new convex optimization problem, termed quadratic decomposable submodular function minimization (QDSFM), which allows to model a number of learning tasks on graphs and hypergraphs. TheExpand
Quadratic Decomposable Submodular Function Minimization: Theory and Practice
TLDR
A new convex optimization problem, termed quadratic decomposable submodular function minimization (QDSFM), which allows to model a number of learning tasks on graphs and hypergraphs and two new applications of QDSFM are described: hypergraph-adapted PageRank and semi-supervised learning. Expand
Active-set Methods for Submodular Minimization Problems
TLDR
A new active-set algorithm for total variation denoising with the assumption of an oracle that solves the corresponding SFM problem is proposed, which performs local descent over ordered partitions and its ability to warm start considerably improves the performance of the algorithm. Expand
Distributed Submodular Minimization And Motion Coordination Over Discrete State Space
We develop a framework for the distributed minimization of submodular functions. Submodular functions are a discrete analog of convex functions and are extensively used in large-scale combinatorialExpand
High-performance submodular function minimization
TLDR
A highperformance submodular function minimization (HPSFO) algorithm that reaches up to 90% of Vector peak performance, offers parallel speed up and outperforms all the existing tested implementations by not one but several orders of magnitude and for the three workload applications the authors implemented and that scales reasonably well with respect to the problem sizes. Expand
Submodular Optimization and Machine Learning: Theoretical Results, Unifying and Scalable Algorithms, and Applications
TLDR
This dissertation explores a class of unifying and scalable algorithms for a number of submodular optimization problems, and connects them to several machine learning applications, and empirically demonstrates the applicability of these techniques on several synthetic and real world problems. Expand
Maximizing submodular functions using probabilistic graphical models
TLDR
This paper proposes a novel convex relaxation which is based on the relationship between submodular functions, entropies and probabilistic graphical models, and presents extensions to constrained problems and maximizing the difference of submodularity functions, which include all possible set functions. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 246 REFERENCES
Structured sparsity-inducing norms through submodular functions
  • F. Bach
  • Computer Science, Mathematics
  • NIPS
  • 2010
TLDR
This paper shows that for nondecreasing submodular set-functions, the corresponding convex envelope can be obtained from its Lovasz extension, a common tool in sub modular analysis, and defines a family of polyhedral norms, for which it provides generic algorithmic tools and theoretical results. Expand
Efficient Minimization of Decomposable Submodular Functions
TLDR
This paper develops an algorithm, SLG, that can efficiently minimize decomposable submodular functions with tens of thousands of variables, and applies it to synthetic benchmarks and a joint classification-and-segmentation task, and shows that it outperforms the state-of-the-art general purpose sub modular minimization algorithms by several orders of magnitude. Expand
Maximizing submodular functions using probabilistic graphical models
TLDR
This paper proposes a novel convex relaxation which is based on the relationship between submodular functions, entropies and probabilistic graphical models, and presents extensions to constrained problems and maximizing the difference of submodularity functions, which include all possible set functions. Expand
Approximating submodular functions everywhere
TLDR
The problem of approximating a non-negative, monotone, submodular function f on a ground set of size n everywhere is considered, after only poly(n) oracle queries, and it is shown that no algorithm can achieve a factor better than Ω(√n/log n), even for rank functions of a matroid. Expand
Shaping Level Sets with Submodular Functions
  • F. Bach
  • Computer Science, Mathematics
  • NIPS
  • 2011
TLDR
By selecting specific submodular functions, this work gives a new interpretation to known norms, such as the total variation, and defines new norms, in particular ones that are based on order statistics with application to clustering and outlier detection, and on noisy cuts in graphs withApplication to change point detection in the presence of outliers. Expand
Submodularity Cuts and Applications
TLDR
This work presents a novel algorithm for maximizing a submodular set function under a cardinality constraint based on a cutting-plane method and implemented as an iterative small-scale binary-integer linear programming procedure. Expand
A Submodular-supermodular Procedure with Applications to Discriminative Structure Learning
TLDR
This paper presents an algorithm for minimizing the difference between two submodular functions using a variational framework which is based on (an extension of) the concave-convex procedure, and gives a polynomial time heuristic for it. Expand
Learning submodular functions
TLDR
This paper considers PAC-style learning of submodular functions in a distributional setting and uses lossless expanders to construct a new family of matroids which can take wildly varying rank values on superpolynomially many sets; no such construction was previously known. Expand
Maximizing a class of submodular utility functions
TLDR
This paper performs a polyhedral analysis of a relevant mixed-integer set and exploits the structure of the utility function h to strengthen the standard submodular formulation significantly, and shows the effectiveness of the new formulation on expected utility maximization in capital budgeting. Expand
Optimization with Sparsity-Inducing Penalties (Foundations and Trends(R) in Machine Learning)
Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. They were first dedicated to linear variable selection but numerous extensions have nowExpand
...
1
2
3
4
5
...