• Corpus ID: 231951679

Joint Continuous and Discrete Model Selection via Submodularity

@article{Bunton2021JointCA,
  title={Joint Continuous and Discrete Model Selection via Submodularity},
  author={Jonathan Bunton and Paulo Tabuada},
  journal={ArXiv},
  year={2021},
  volume={abs/2102.09029}
}
. In model selection problems for machine learning, the desire for a well-performing model with meaningful structure is typically expressed through a regularized optimization problem. In many scenar- ios, however, the meaningful structure is specified in some discrete space, leading to difficult nonconvex optimization problems. In this paper, we connect the model selection problem with structure-promoting regularizers to submodular function minimization with continuous and discrete arguments. In… 

Figures from this paper

References

SHOWING 1-10 OF 34 REFERENCES

Learning with Submodular Functions: A Convex Optimization Perspective

  • F. Bach
  • Computer Science
    Found. Trends Mach. Learn.
  • 2013
In Learning with Submodular Functions: A Convex Optimization Perspective, the theory of submodular functions is presented in a self-contained way from a convex analysis perspective, presenting tight links between certain polyhedra, combinatorial optimization and convex optimization problems.

Non-monotone Continuous DR-submodular Maximization: Structure and Algorithms

This work investigates the problem of maximizing non-monotone DR-submodular continuous functions under general down-closed convex constraints by investigating geometric properties that underlie such objectives, and devise two optimization algorithms with provable guarantees that are validated on synthetic and real-world problem instances.

Submodular functions: from discrete to continuous domains

  • F. Bach
  • Mathematics, Computer Science
    Math. Program.
  • 2019
This paper shows that most results relating submodularity and convexity for set-functions can be extended to all submodular functions, and provides practical algorithms which are based on function evaluations, to minimize generic sub modular functions on discrete domains, with associated convergence rates.

Guaranteed Non-convex Optimization: Submodular Maximization over Continuous Domains

The weak DR property is introduced that gives a unified characterization of submodularity for all set, integer-lattice and continuous functions and for maximizing monotone DR-submodular continuous functions under general down-closed convex constraints, a Frank-Wolfe variant with approximation guarantee, and sub-linear convergence rate are proposed.

Robust Budget Allocation Via Continuous Submodular Functions

This work revisits a continuous version of the Budget Allocation or Bipartite Influence Maximization problem, and establishes conditions under which such a problem can be solved to arbitrary precision.

Shaping Level Sets with Submodular Functions

  • F. Bach
  • Mathematics, Computer Science
    NIPS
  • 2011
By selecting specific submodular functions, this work gives a new interpretation to known norms, such as the total variation, and defines new norms, in particular ones that are based on order statistics with application to clustering and outlier detection, and on noisy cuts in graphs withApplication to change point detection in the presence of outliers.

Restricted Strong Convexity Implies Weak Submodularity

This work shows that greedy algorithms perform within a constant factor from the best possible subset-selection solution for a broad class of general objective functions.

Fast Semidierential-b ased Submodular Function Optimization: Extended Version

This work presents a practical and powerful new framework for both unconstrained and constrained submodular function optimization based on discrete semidierentials (sub- and super-dierential) and analyzes theoretical properties of the algorithms for minimization and maximization, and shows that many state-of-the-art maximization algorithms are special cases.

Approximate Submodularity and its Applications: Subset Selection, Sparse Approximation and Dictionary Selection

The submodularity ratio is introduced as a measure of how "close" to submodular a set function f is, and it is shown that when f has sub modularity ratio γ, the greedy algorithm for maximizing f provides a (1 - e-γ)-approximation.

An analysis of approximations for maximizing submodular set functions—I

It is shown that a “greedy” heuristic always produces a solution whose value is at least 1 −[(K − 1/K]K times the optimal value, which can be achieved for eachK and has a limiting value of (e − 1)/e, where e is the base of the natural logarithm.