Corpus ID: 44061130

Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance

@article{Jean2018SemisupervisedDK,
  title={Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance},
  author={Neal Jean and Sang Michael Xie and Stefano Ermon},
  journal={ArXiv},
  year={2018},
  volume={abs/1805.10407}
}
Large amounts of labeled data are typically required to train deep learning models. For many real-world problems, however, acquiring additional data can be expensive or even impossible. We present semi-supervised deep kernel learning (SSDKL), a semi-supervised regression model based on minimizing predictive variance in the posterior regularization framework. SSDKL combines the hierarchical representation learning of neural networks with the probabilistic modeling capabilities of Gaussian… Expand
Metric-Based Semi-Supervised Regression
TLDR
The experimental results indicate that the proposed method to cope with semi-supervised regression problems achieves promising results and could capture the trend of a non-linear function and normally predict well even though this dataset comprises extreme outliers. Expand
Deep Low-Density Separation for Semi-supervised Classification
TLDR
A novel hybrid method that applies low-density separation to the embedded features of neural network-based embeddings and effectively classifies thousands of unlabeled users from a relatively small number of hand-classified examples is introduced. Expand
Weakly-supervised Multi-output Regression via Correlated Gaussian Processes
TLDR
This paper considers multi-output regression under a weakly-supervised setting where a subset of data points from multiple groups are unlabeled and derives a modified Variational bound for fast and stable convergence in model inference and a scalable variational bound that is amenable to stochastic optimization. Expand
Lautum Regularization for Semi-Supervised Transfer Learning
TLDR
The theory suggests that one may improve the transferability of a deep neural network by imposing a Lautum information based regularization that relates the network weights to the target data. Expand
High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
TLDR
A method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces is introduced, using label guidance from the blackbox function to structure the VAE latent space, facilitating the Gaussian process fit and yielding improved BO performance. Expand
A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
TLDR
This work proposes a simple but effective baseline that is robust to noisy labels, even with severe noise, and involves a variance regularization term that implicitly penalizes the Jacobian norm of the neural network on the whole training set (including the noisy-labeled data), which encourages generalization and prevents overfitting to the corrupted labels. Expand
Twin Neural Network Regression is a Semi-Supervised Regression Algorithm
TLDR
Semi-supervised training of twin neural network regression improves TNNR performance, which is already state of the art, significantly. Expand
Efficient Statistical Tests: A Neural Tangent Kernel Approach
TLDR
This work proposes a shift-invariant convolutional neural tangent kernel (SCNTK) based outlier detector and two-sample tests with maximum mean discrepancy (MMD) that isO(n) in the number of samples due to using the random feature approximation. Expand
Learning Data-adaptive Nonparametric Kernels
TLDR
A Data-Adaptive Nonparametric Kernel (DANK) learning framework by imposing an adaptive matrix on the kernel/Gram matrix in an entry-wise strategy that outperforms other representative kernel learning based algorithms on various classification and regression benchmark datasets. Expand
Implicit Kernel Learning
TLDR
This paper explores learning the spectral distribution of kernel via implicit generative models parametrized by deep neural networks via Implicit Kernel Learning (IKL), which is simple to train and inference is performed via sampling random Fourier features. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 45 REFERENCES
Semi-supervised Learning by Entropy Minimization
TLDR
This framework, which motivates minimum entropy regularization, enables to incorporate unlabeled data in the standard supervised learning, and includes other approaches to the semi-supervised problem as particular or limiting cases. Expand
Semi-Supervised Regression with Co-Training
TLDR
Experiments show that COREG can effectively exploit unlabeled data to improve regression estimates and is proposed as a co-training style semi-supervised regression algorithm. Expand
Stochastic Variational Deep Kernel Learning
TLDR
An efficient form of stochastic variational inference is derived which leverages local kernel interpolation, inducing points, and structure exploiting algebra within this framework to enable classification, multi-task learning, additive covariance structures, and Stochastic gradient training. Expand
Realistic Evaluation of Semi-Supervised Learning Algorithms
TLDR
This work creates a unified reimplemention and evaluation platform of various widelyused SSL techniques and finds that the performance of simple baselines which do not use unlabeled data is often underreported, that SSL methods differ in sensitivity to the amount of labeled and unlabeling data, and that performance can degrade substantially when the unlabelED dataset contains out-of-class examples. Expand
Temporal Ensembling for Semi-Supervised Learning
TLDR
Self-ensembling is introduced, where it is shown that this ensemble prediction can be expected to be a better predictor for the unknown labels than the output of the network at the most recent training epoch, and can thus be used as a target for training. Expand
Deep Hybrid Models: Bridging Discriminative and Generative Approaches
Most methods in machine learning are described as either discriminative or generative. The former often attain higher predictive accuracy, while the latter are more strongly regularized and can dealExpand
Minimum variance semi-supervised boosting for multi-label classification
TLDR
The experiments show that the proposed algorithm outperforms its supervised counterpart as well as the existing information theoretic based semi-supervised methods, and its performance is steadily improving as more unlabeled data is available. Expand
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
TLDR
The recently proposed Temporal Ensembling has achieved state-of-the-art results in several semi-supervised learning benchmarks, but it becomes unwieldy when learning large datasets, so Mean Teacher, a method that averages model weights instead of label predictions, is proposed. Expand
Auxiliary Deep Generative Models
TLDR
This work extends deep generative models with auxiliary variables which improves the variational approximation and proposes a model with two stochastic layers and skip connections which shows state-of-the-art performance within semi-supervised learning on MNIST, SVHN and NORB datasets. Expand
Introduction to Semi-Supervised Learning
TLDR
This introductory book presents some popular semi-supervised learning models, including self-training, mixture models, co-training and multiview learning, graph-based methods, and semi- supervised support vector machines, and discusses their basic mathematical formulation. Expand
...
1
2
3
4
5
...