Deep Neural Network Hyperparameter Optimization with Orthogonal Array Tuning

@article{Zhang2019DeepNN,
  title={Deep Neural Network Hyperparameter Optimization with Orthogonal Array Tuning},
  author={X. Zhang and Xiaocong Chen and Lina Yao and Chang Ge and Manqing Dong},
  journal={ArXiv},
  year={2019},
  volume={abs/1907.13359}
}
Deep learning algorithms have achieved excellent performance lately in a wide range of fields (e.g., computer version). However, a severe challenge faced by deep learning is the high dependency on hyper-parameters. The algorithm results may fluctuate dramatically under the different configuration of hyper-parameters. Addressing the above issue, this paper presents an efficient Orthogonal Array Tuning Method (OATM) for deep learning hyper-parameter tuning. We describe the OATM approach in five… Expand
Machine Learning Model Optimization with Hyper Parameter Tuning Approach
Hyper-parameters tuning is a key step to find the optimal machine learning parameters. Determining the best hyper-parameters takes a good deal of time, especially when the objective functions areExpand
High-speed hyperparameter optimization for deep ResNet models in image recognition
TLDR
A manual search approach by enhancing the traditional data augmentation proposed by the previous approaches is developed and it significantly improves the classification error rate and reduces the computing time compared with the previous automatic approaches. Expand
Convolution Neural Network Hyperparameter Optimization Using Simplified Swarm Optimization
TLDR
This study proposes the idea of applying Simplified Swarm Optimization (SSO) on the hyperparameter optimization of LeNet models while using MNIST, FashionMNIST, and Cifar10 as validation, and results show that the proposed algorithm has higher accuracy than the original LeNet model. Expand
Hyperparameter Optimization for Deep Residual Learning in Image Classification
  • Abbasi Jafar, Myungho Lee
  • Computer Science
  • 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)
  • 2020
TLDR
A manual search approach for ResNet architecture is built by altering the traditional data augmentation proposed by the previous approaches and significantly reduces the computing time compared with the previous automatic approaches. Expand
HELP: An LSTM-based approach to hyperparameter exploration in neural network learning
TLDR
The Hyperparameter Exploration LSTM-Predictor (HELP) is proposed which is an improved random exploring method using a probability-based exploration with an L STM-based prediction that has a higher probability to find a better hyperparameter with less time. Expand
MOFA: Modular Factorial Design for Hyperparameter Optimization
TLDR
A new HPO method, MOdular FActorial Design (MOFA), to use techniques from Experimental Designs to improve sample efficiency of model-free methods and theoretically and empirically show that MOFA has great advantages over existing model-based and model- free methods. Expand
Tuning machine learning dropout for subsurface uncertainty model accuracy
TLDR
A method to compare the entire uncertainty model based on the prediction realizations and the addition of a single objective function, known as uncertainty model goodness, to the loss function to integrate model uncertainty for improved tuning of the dropout frequency in deep learning models is proposed. Expand
Optimal hyperparameter selection of deep learning models for COVID-19 chest X-ray classification
TLDR
A discriminative fine-tuning approach, which dynamically assigns different learning rates to each layer of the network, and a visual interpretation of the model’s output to prove that the model can aid radiologists in rapidly screening for the symptoms of COVID-19. Expand
A Novel Sketch Recognition Model based on Convolutional Neural Networks
  • A. T. Kabakus
  • Computer Science
  • 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA)
  • 2020
TLDR
A novel sketch recognition model based on Convolutional Neural Networks is proposed which outperformed the related work on the same dataset and revealed the model’s efficiency in terms of predicting the classes of the given sketches. Expand
Simplified Swarm Optimisation for the Hyperparameters of a Convolutional Neural Network
Simplified Swarm Optimisation for the Hyperparameters of a Convolutional Neural Network Wei-Chang Yeh, Yi-Ping Lin, Yun-Chia Liang, Chyh-Ming Lai, and Xiao-Zhi Gao 1 Department of IndustrialExpand
...
1
2
3
...

References

SHOWING 1-10 OF 15 REFERENCES
Algorithms for Hyper-Parameter Optimization
TLDR
This work contributes novel techniques for making response surface models P(y|x) in which many elements of hyper-parameter assignment (x) are known to be irrelevant given particular values of other elements. Expand
Multi-Task Bayesian Optimization
TLDR
This paper proposes an adaptation of a recently developed acquisition function, entropy search, to the cost-sensitive, multi-task setting and demonstrates the utility of this new acquisition function by leveraging a small dataset to explore hyper-parameter settings for a large dataset. Expand
Recent advances in convolutional neural networks
TLDR
This paper details the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation, and introduces various applications of convolutional neural networks in computer vision, speech and natural language processing. Expand
Practical Bayesian Optimization of Machine Learning Algorithms
TLDR
This work describes new algorithms that take into account the variable cost of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation and shows that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms. Expand
Efficient and Robust Automated Machine Learning
TLDR
This work introduces a robust new AutoML system based on scikit-learn, which improves on existing AutoML methods by automatically taking into account past performance on similar datasets, and by constructing ensembles from the models evaluated during the optimization. Expand
Intent Recognition in Smart Living Through Deep Recurrent Neural Networks
TLDR
A 7-layer deep learning model is proposed to classify raw EEG signals with the aim of recognizing subjects’ intents, to avoid the time consumed in pre-processing and feature extraction. Expand
Recurrent Neural Network Language Model Adaptation for Conversational Speech Recognition
TLDR
Two adaptation models for recurrent neural network language models (RNNLMs) to capture topic effects and longdistance triggers for conversational automatic speech recognition (ASR) are proposed and modest WER and perplexity reductions are shown. Expand
Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals
TLDR
A novel deep neural network based learning framework that affords perceptive insights into the relationship between the MI-EEG data and brain activities is proposed, which outperforms a series of baselines and the competitive state-of-the-art methods. Expand
Bayesian Gait Optimization for Bipedal Locomotion
TLDR
This work uses Bayesian optimization to efficiently find gait parameters that optimize the desired performance metric and validates the approach to Bayesian gait optimization on a low-cost and fragile real bipedal walker and shows that good walking gaits can be efficiently found by Bayesian optimized. Expand
A Review of Random Search Methods
TLDR
This chapter provides a brief review of random search methods for simulation optimization and expands the scope to address simulation optimization problems with continuous decision variables and/or multiple (stochastic) performance measures. Expand
...
1
2
...