Learning Set Functions Under the Optimal Subset Oracle via Equivariant Variational Inference

  title={Learning Set Functions Under the Optimal Subset Oracle via Equivariant Variational Inference},
  author={Zijing Ou and Tingyang Xu and Qinliang Su and Yingzhen Li and Peilin Zhao and Yatao Bian},
Learning set functions becomes increasingly more important in many applications like product recommendation and compound selection in AI-aided drug discovery. The majority of existing works study methodologies of set function learning under the function value oracle, which, however, requires expensive supervision signals. This renders it impractical for applications with only weak supervisions under the Optimal Subset (OS) oracle, the study of which is surprisingly overlooked. In this work, we… 


Differentiable Submodular Maximization
The effectiveness of the approach for jointly learning and optimizing on synthetic maxcut data, and on real world applications such as product recommendation and image collection summarization, is demonstrated.
Set Prediction without Imposing Structure as Conditional Density Estimation
This paper proposes a stochastically augmented prediction algorithm that enables multiple predictions, reflecting the possible variations in the target set, and empirically demonstrates the capability to learn multi-modal densities and produce multiple plausible predictions.
GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning
This work forms GLISTER as a mixed discretecontinuous bi-level optimization problem to select a subset of the training data, which maximizes the log-likelihood on a held-out validation set, and proposes an iterative online algorithm GLISTER-ONLINE, which performs data selection iteratively along with the parameter updates, and can be applied to any loss-based learning algorithm.
SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios
It is argued that SIMILAR not only works in standard active learning but also easily extends to the realistic settings considered above and acts as a one-stop solution for active learning that is scalable to large real-world datasets.
Advances in Variational Inference
An overview of recent trends in variational inference is given and a summary of promising future research directions is provided.
Inference Suboptimality in Variational Autoencoders
It is found that divergence from the true posterior is often due to imperfect recognition networks, rather than the limited complexity of the approximating distribution, and the parameters used to increase the expressiveness of the approximation play a role in generalizing inference.
Relaxed Multivariate Bernoulli Distribution and Its Applications to Deep Generative Models
A multivariate generalization of the Relaxed Bernoulli distribution is proposed, which can be reparameterized and can capture the correlation between variables via a Gaussian copula and demonstrate its effectiveness in two tasks: density estimation withBernoulli VAE and semisupervised multi-label classification.
Adam: A Method for Stochastic Optimization
This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Auto-Encoding Variational Bayes
A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.
Utilizing Variational Optimization to Learn Markov Random Fields
  • M. Tappen
  • Computer Science
    2007 IEEE Conference on Computer Vision and Pattern Recognition
  • 2007
This paper shows how variational optimization can be used to learn the parameters of an MRF, and suggests that the field of experts model, which is difficult to interpret, can be understood as imposing piecewise continuity on the image.