• Corpus ID: 202765959

Combinatorial Bayesian Optimization using the Graph Cartesian Product

@inproceedings{Oh2019CombinatorialBO,
  title={Combinatorial Bayesian Optimization using the Graph Cartesian Product},
  author={Changyong Oh and Jakub M. Tomczak and Efstratios Gavves and Max Welling},
  booktitle={NeurIPS},
  year={2019}
}
This paper focuses on Bayesian Optimization (BO) for objectives on combinatorial search spaces, including ordinal and categorical variables. Despite the abundance of potential applications of Combinatorial BO, including chipset configuration search and neural architecture search, only a handful of methods have been pro- posed. We introduce COMBO, a new Gaussian Process (GP) BO. COMBO quantifies “smoothness” of functions on combinatorial search spaces by utilizing a combinatorial graph. The… 

Figures and Tables from this paper

Mercer Features for Efficient Combinatorial Bayesian Optimization

TLDR
The key idea behind MerCBO is to provide explicit feature maps for diffusion kernels over discrete objects by exploiting the structure of their combinatorial graph representation in Mercer Features for Combinatorial Bayesian Optimization.

Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytope

TLDR
The main idea is to use a random mapping which embeds the combinatorial space into a convex polytope in a continuous space, on which all essential process is performed to determine a solution to the black-box optimization in the combinatorsial space.

Encoding categorical variables in physics-informed graphs for Bayesian Optimization

TLDR
This work presents a method for reshaping and simplifying the graph-structures based on prior physical knowledge based on physics-informed graphs, which improves the optimization performance in comparison to the default Combo approach and other state of the art optimization techniques.

Scalable Combinatorial Bayesian Optimization with Tractable Statistical models

TLDR
PSR approach relies on reformulation of AFO problem as submodular relaxation with some unknown parameters, which can be solved efficiently using minimum graph cut algorithms and construction of an optimization problem to estimate the unknown parameters with close approximation to the true objective.

Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces

TLDR
The key idea is to define a novel structure-coupled kernel that explicitly integrates the structural information from decoded structures with the learned latent space representation for better surrogate modeling.

Bayesian Optimization over Hybrid Spaces

TLDR
This paper develops a principled approach for constructing diffusion kernels over hybrid spaces by utilizing the additive kernel formulation, which allows additive interactions of all orders in a tractable manner and theoretically analyze the modeling strength of additive hybrid kernels and prove that it has the universal approximation property.

Efficient Black-Box Combinatorial Optimization

TLDR
This work proposes a computationally efficient model learning algorithm based on exponential weight updates that uses multilinear polynomials as surrogate model for functions over the Boolean hypercube and extends this representation via a group-theoretic Fourier expansion to address functions over categorical variables.

Bayesian Optimization over Permutation Spaces

TLDR
Two algorithms for BO over Permutation Spaces (BOPS) are proposed and evaluated, showing that both BOPS-T and Bops-H perform better than the state-of-the-art BO algorithm for combinatorial spaces.

Discrete Bayesian Optimization Algorithms and Applications

TLDR
The Separable Bayesian Optimization algorithm (SBO) is proposed to overcome the limitations of classical BO and the emphasis is on Gaussian Processes as surrogate models.

Combinatorial Black-Box Optimization with Expert Advice

TLDR
A computationally efficient model learning algorithm based on multilinear polynomials and exponential weight updates that improves the computational time up to several orders of magnitude compared to state-of-the-art algorithms in the literature.
...

References

SHOWING 1-10 OF 54 REFERENCES

COMBO: Combinatorial Bayesian Optimization using Graph Representations

TLDR
This paper proposes COMBO, a method that represents values of discrete variables as vertices of a graph and then use the diffusion kernel on that graph, enabling kernel computation in linear time with respect to the number of input variables.

Combinatorial Bayesian Optimization using Graph Representations

TLDR
This paper proposes COMBO, a method that represents values of discrete variables as vertices of a graph and then use the diffusion kernel on that graph, enabling kernel computation in linear time with respect to the number of input variables.

Bayesian Optimization of Combinatorial Structures

TLDR
This article proposes an adaptive, scalable model that identifies useful combinatorial structure even when data is scarce, and pioneers the use of semidefinite programming to achieve efficiency and scalability.

Scalable Bayesian Optimization Using Deep Neural Networks

TLDR
This work shows that performing adaptive basis function regression with a neural network as the parametric form performs competitively with state-of-the-art GP-based approaches, but scales linearly with the number of data rather than cubically, which allows for a previously intractable degree of parallelism.

Bayesian Optimization in a Billion Dimensions via Random Embeddings

TLDR
Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low, and show thatREMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver.

A Tutorial on Bayesian Optimization

TLDR
This tutorial describes how Bayesian optimization works, including Gaussian process regression and three common acquisition functions: expected improvement, entropy search, and knowledge gradient, and provides a generalization of expected improvement to noisy evaluations beyond the noise-free setting where it is more commonly applied.

Bayesian Optimization with Gradients

TLDR
This paper develops a novel Bayesian optimization algorithm, the derivative-enabled knowledge-gradient (dKG), for which one-step Bayes-optimality, asymptotic consistency, and greater one- step value of information than is possible in the derivatives-free setting are shown.

Neural Architecture Search with Bayesian Optimisation and Optimal Transport

TLDR
NASHBOT is developed, a Gaussian process based BO framework for neural architecture search which outperforms other alternatives for architecture search in several cross validation based model selection tasks on multi-layer perceptrons and convolutional neural networks.

Bayesian optimization for automated model selection

TLDR
This work presents a sophisticated method for automatically searching for an appropriate kernel from an infinite space of potential choices, based on Bayesian optimization in model space, and constructs a novel kernel between models to explain a given dataset.
...