# Combinatorial Bayesian Optimization using the Graph Cartesian Product

@inproceedings{Oh2019CombinatorialBO, title={Combinatorial Bayesian Optimization using the Graph Cartesian Product}, author={Changyong Oh and Jakub M. Tomczak and Efstratios Gavves and Max Welling}, booktitle={NeurIPS}, year={2019} }

This paper focuses on Bayesian Optimization (BO) for objectives on combinatorial search spaces, including ordinal and categorical variables. Despite the abundance of potential applications of Combinatorial BO, including chipset configuration search and neural architecture search, only a handful of methods have been pro- posed. We introduce COMBO, a new Gaussian Process (GP) BO. COMBO quantifies “smoothness” of functions on combinatorial search spaces by utilizing a combinatorial graph. The…

## 56 Citations

### Mercer Features for Efficient Combinatorial Bayesian Optimization

- Computer ScienceAAAI
- 2021

The key idea behind MerCBO is to provide explicit feature maps for diffusion kernels over discrete objects by exploiting the structure of their combinatorial graph representation in Mercer Features for Combinatorial Bayesian Optimization.

### Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytope

- Computer Science, MathematicsArXiv
- 2020

The main idea is to use a random mapping which embeds the combinatorial space into a convex polytope in a continuous space, on which all essential process is performed to determine a solution to the black-box optimization in the combinatorsial space.

### Encoding categorical variables in physics-informed graphs for Bayesian Optimization

- Computer Science2022 IEEE International Conference on Omni-layer Intelligent Systems (COINS)
- 2022

This work presents a method for reshaping and simplifying the graph-structures based on prior physical knowledge based on physics-informed graphs, which improves the optimization performance in comparison to the default Combo approach and other state of the art optimization techniques.

### Scalable Combinatorial Bayesian Optimization with Tractable Statistical models

- Computer ScienceArXiv
- 2020

PSR approach relies on reformulation of AFO problem as submodular relaxation with some unknown parameters, which can be solved efficiently using minimum graph cut algorithms and construction of an optimization problem to estimate the unknown parameters with close approximation to the true objective.

### Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces

- Computer ScienceNeurIPS
- 2021

The key idea is to define a novel structure-coupled kernel that explicitly integrates the structural information from decoded structures with the learned latent space representation for better surrogate modeling.

### Bayesian Optimization over Hybrid Spaces

- Computer ScienceICML
- 2021

This paper develops a principled approach for constructing diffusion kernels over hybrid spaces by utilizing the additive kernel formulation, which allows additive interactions of all orders in a tractable manner and theoretically analyze the modeling strength of additive hybrid kernels and prove that it has the universal approximation property.

### Efficient Black-Box Combinatorial Optimization

- Computer Science
- 2020

This work proposes a computationally efficient model learning algorithm based on exponential weight updates that uses multilinear polynomials as surrogate model for functions over the Boolean hypercube and extends this representation via a group-theoretic Fourier expansion to address functions over categorical variables.

### Bayesian Optimization over Permutation Spaces

- Computer ScienceAAAI
- 2022

Two algorithms for BO over Permutation Spaces (BOPS) are proposed and evaluated, showing that both BOPS-T and Bops-H perform better than the state-of-the-art BO algorithm for combinatorial spaces.

### Discrete Bayesian Optimization Algorithms and Applications

- Computer Science
- 2020

The Separable Bayesian Optimization algorithm (SBO) is proposed to overcome the limitations of classical BO and the emphasis is on Gaussian Processes as surrogate models.

### Combinatorial Black-Box Optimization with Expert Advice

- Computer ScienceKDD
- 2020

A computationally efficient model learning algorithm based on multilinear polynomials and exponential weight updates that improves the computational time up to several orders of magnitude compared to state-of-the-art algorithms in the literature.

## References

SHOWING 1-10 OF 54 REFERENCES

### COMBO: Combinatorial Bayesian Optimization using Graph Representations

- Computer Science
- 2019

This paper proposes COMBO, a method that represents values of discrete variables as vertices of a graph and then use the diffusion kernel on that graph, enabling kernel computation in linear time with respect to the number of input variables.

### Combinatorial Bayesian Optimization using Graph Representations

- Computer ScienceArXiv
- 2019

This paper proposes COMBO, a method that represents values of discrete variables as vertices of a graph and then use the diffusion kernel on that graph, enabling kernel computation in linear time with respect to the number of input variables.

### Bayesian Optimization of Combinatorial Structures

- Computer ScienceICML
- 2018

This article proposes an adaptive, scalable model that identifies useful combinatorial structure even when data is scarce, and pioneers the use of semidefinite programming to achieve efficiency and scalability.

### Scalable Bayesian Optimization Using Deep Neural Networks

- Computer ScienceICML
- 2015

This work shows that performing adaptive basis function regression with a neural network as the parametric form performs competitively with state-of-the-art GP-based approaches, but scales linearly with the number of data rather than cubically, which allows for a previously intractable degree of parallelism.

### Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes

- Computer ScienceNeurocomputing
- 2020

### Bayesian Optimization in a Billion Dimensions via Random Embeddings

- Computer ScienceJ. Artif. Intell. Res.
- 2016

Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low, and show thatREMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver.

### A Tutorial on Bayesian Optimization

- Computer ScienceArXiv
- 2018

This tutorial describes how Bayesian optimization works, including Gaussian process regression and three common acquisition functions: expected improvement, entropy search, and knowledge gradient, and provides a generalization of expected improvement to noisy evaluations beyond the noise-free setting where it is more commonly applied.

### Bayesian Optimization with Gradients

- Computer ScienceNIPS
- 2017

This paper develops a novel Bayesian optimization algorithm, the derivative-enabled knowledge-gradient (dKG), for which one-step Bayes-optimality, asymptotic consistency, and greater one- step value of information than is possible in the derivatives-free setting are shown.

### Neural Architecture Search with Bayesian Optimisation and Optimal Transport

- Computer ScienceNeurIPS
- 2018

NASHBOT is developed, a Gaussian process based BO framework for neural architecture search which outperforms other alternatives for architecture search in several cross validation based model selection tasks on multi-layer perceptrons and convolutional neural networks.

### Bayesian optimization for automated model selection

- Computer ScienceNIPS
- 2016

This work presents a sophisticated method for automatically searching for an appropriate kernel from an infinite space of potential choices, based on Bayesian optimization in model space, and constructs a novel kernel between models to explain a given dataset.