How Deep Should be the Depth of Convolutional Neural Networks: a Backyard Dog Case Study

@article{Gorban2019HowDS,
  title={How Deep Should be the Depth of Convolutional Neural Networks: a Backyard Dog Case Study},
  author={Alexander N. Gorban and Evgeny M. Mirkes and Ivan Y. Tyukin},
  journal={Cognitive Computation},
  year={2019},
  volume={12},
  pages={388-397}
}
The work concerns the problem of reducing a pre-trained deep neuronal network to a smaller network, with just few layers, whilst retaining the network’s functionality on a given task. In this particular case study, we are focusing on the networks developed for the purposes of face recognition. The proposed approach is motivated by the observation that the aim to deliver the highest accuracy possible in the broadest range of operational conditions, which many deep neural networks models strive… Expand
High-Dimensional Separability for One- and Few-Shot Learning
TLDR
New multi-correctors of AI systems are presented and illustrated with examples of predicting errors and learning new classes of objects by a deep convolutional neural network. Expand
Artificial Neural Network Pruning to Extract Knowledge
  • E. Mirkes
  • Computer Science, Mathematics
  • 2020 International Joint Conference on Neural Networks (IJCNN)
  • 2020
TLDR
This paper lists the basic NN simplification problems and controlled pruning procedures to solve these problems and develops procedures to find the optimal structure of NN for each task, measure the influence of each input signal and NN parameter, and provide a detailed verbal description of the algorithms and skills of Nn. Expand
Language Processing Model Construction and Simulation Based on Hybrid CNN and LSTM
  • Shujing Zhang
  • Computer Science, Medicine
  • Comput. Intell. Neurosci.
  • 2021
TLDR
The model structure of a typical convolutional neural network model is analyzed to increase the network depth and width in order to improve its performance, the network structure that further improves the model performance by using the attention mechanism is analyzed, and the current special model structure is summarized. Expand
A Hybrid-Domain Deep Learning-Based BCI For Discriminating Hand Motion Planning From EEG Sources.
TLDR
Experimental results show a significant performance improvement of the proposed hybrid-domain DL approach as compared to temporal-only and time-frequency-only-based benchmark approaches, achieving an average accuracy of [Formula: see text]%. Expand
Swimming Training Evaluation Method Based on Convolutional Neural Network
  • Lei Zhang, Wei Liu
  • Computer Science
  • Complex.
  • 2021
TLDR
An evaluation algorithm suitable for swimming training based on the improved AlexNet network is designed, which uses data augmentation technology to expand the data set, to a certain extent alleviating the problem of overfitting. Expand
Correction of AI systems by linear discriminants: Probabilistic foundations
TLDR
It is proved that this separability property holds for a wide class of distributions including log-concave distributions and distributions with a special ‘SMeared Absolute Continuity’ (SmAC) property defined through relations between the volume and probability of sets of vanishing volume, much wider than the Gaussian distributions. Expand
Independent Component Analysis for Unraveling the Complexity of Cancer Omics Datasets
TLDR
A review of a number of recent works where ICA was shown to be a useful tool for unraveling the complexity of cancer biology from the analysis of different types of omics data, mainly collected for tumoral samples and discusses the emerging ICA applications to the integrative analysis of multi-level omics datasets. Expand
Texture Image Classification Method of Porcelain Fragments Based on Convolutional Neural Network
  • Hongchang Wu
  • Medicine, Computer Science
  • Comput. Intell. Neurosci.
  • 2021
TLDR
This paper uses the layered variational image decomposition method to decompose the image into structural components and texture components and uses a compressed sensing algorithm based on hybrid basis to reconstruct the structure and texture component with large data. Expand
Development of Basketball Tactics Basic Cooperation Teaching System Based on CNN and BP Neural Network
  • Lin Hua, Guangyu Liu
  • Medicine
  • Computational intelligence and neuroscience
  • 2021
TLDR
The results show that the basketball tactical basic cooperation teaching system based on the artificial neural network has a good teaching effect in improving students' basketball tacticalbasic cooperation ability. Expand
Intelligent Simulation of Children's Psychological Path Selection Based on Chaotic Neural Network Algorithm
  • Yue Wang
  • Medicine
  • Computational intelligence and neuroscience
  • 2021
TLDR
The results show that compared with the current mainstream intelligent simulation methods with iterative loop algorithms as the core, the intelligent simulation model based on the chaotic neural network algorithm has a good classification effect and is improved by at least 37%. Expand
...
1
2
...

References

SHOWING 1-10 OF 43 REFERENCES
Very Deep Convolutional Networks for Large-Scale Image Recognition
TLDR
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. Expand
Deep Learning for Understanding Faces: Machines May Be Just as Good, or Better, than Humans
TLDR
An overview of deep-learning methods used for face recognition is provided and different modules involved in designing an automatic face recognition system are discussed and the role of deep learning for each of them is discussed. Expand
ImageNet classification with deep convolutional neural networks
TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective. Expand
Deep Face Recognition
TLDR
It is shown how a very large scale dataset can be assembled by a combination of automation and human in the loop, and the trade off between data purity and time is discussed. Expand
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
TLDR
A new scaling method is proposed that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient and is demonstrated the effectiveness of this method on scaling up MobileNets and ResNet. Expand
Deep Learning Face Representation from Predicting 10,000 Classes
TLDR
It is argued that DeepID can be effectively learned through challenging multi-class face identification tasks, whilst they can be generalized to other tasks (such as verification) and new identities unseen in the training set. Expand
Reducing and Stretching Deep Convolutional Activation Features for Accurate Image Classification
TLDR
RS-DeCAF is proposed, a new model based on “reducing” and “stretching” the dimensionality of DeCAF that can generally achieve the highest classification accuracy when its dimensionality is two to four times of that of De CAF. Expand
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
TLDR
This work revisits both the alignment step and the representation step by employing explicit 3D face modeling in order to apply a piecewise affine transformation, and derive a face representation from a nine-layer deep neural network. Expand
Object Detection With Deep Learning: A Review
TLDR
This paper provides a review of deep learning-based object detection frameworks and focuses on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. Expand
One-trial correction of legacy AI systems and stochastic separation theorems
TLDR
The tuning method that is proposed enables dealing with errors without the need to re-train the AI legacy system, and results in a network of modulating rules “dressing up” and improving performance of existing AI systems. Expand
...
1
2
3
4
5
...