Sparsifying Neural Network Connections for Face Recognition

  title={Sparsifying Neural Network Connections for Face Recognition},
  author={Yi Sun and Xiaogang Wang and Xiaoou Tang},
  journal={2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
This paper proposes to learn high-performance deep ConvNets with sparse neural connections, referred to as sparse ConvNets, for face recognition. The sparse ConvNets are learned in an iterative way, each time one additional layer is sparsified and the entire model is re-trained given the initial weights learned in previous iterations. One important finding is that directly training the sparse ConvNet from scratch failed to find good solutions for face recognition, while using a previously… 

Figures and Tables from this paper

Face Recognition Based on Densely Connected Convolutional Networks

A face recognition model named Dense Face is proposed to explore the performance of densely connected network in face recognition and has achieved higher face verification accuracy and has better robustness than other model such as VGG Face and ResNet model.

A Bayesian Scene-Prior-Based Deep Network Model for Face Verification

A Bayesian scene-prior-based deep learning model is proposed in this paper with the aim to extract important features from background scenes to improve face recognition/verification performance.

Face Recognition via Centralized Coordinate Learning

This work formulate jointly the learning of face features and classification vectors, and propose a simple yet effective centralized coordinate learning (CCL) method, which enforces the features to be dispersedly spanned in the coordinate space while ensuring the classification vectors to lie on a hypersphere.

Joint Supervision for Discriminative Feature Learning in Convolutional Neural Networks

Jointly supervised by softmax loss, H-contrastive loss and center loss, a robust CNN is trained to enhance the discriminative power of the deeply learned features from different classes.

Git Loss for Deep Face Recognition

This work introduces a joint supervision signal, Git loss, which leverages on softmax and center loss functions to enhance the discriminative capability of deep features in CNNs and achieves state-of-the-art accuracy on two major face recognition benchmark datasets.

DeepVisage: Making Face Recognition Simple Yet With Powerful Generalization Skills

This work proposes an easily trainable and single CNN based FR method that exploits the residual learning framework and uses normalized features to compute the loss and obtains very competitive and state-of-the-art results on the LFW, IJB-A, YouTube faces and CACD datasets.

Exponential Discriminative Metric Embedding in Deep Learning

Noise-resistant network: a deep-learning method for face recognition under noise

A multi-input structure in the final fully connected layer of the proposed NR-Network to extract a multi-scale and more discriminative feature from the input image to achieve better performance than two deep benchmark networks for face recognition under noise.

SubFace: Learning with Softmax Approximation for Face Recognition

This paper presents a softmax approximation method called SubFace, which employs the subspace feature to promote the performance of face recognition, and strongly proves the effectiveness of subspace strategy with the margin-based loss.

Compressing Deep Neural Network for Facial Landmarks Detection

This work proposes a method to compress large deep neural networks, which includes importance-based neuron pruning, and introduces weights correlations to prune unimportant neurons, which can reduce index storage and inference computation costs.



Deeply learned face representations are sparse, selective, and robust

This paper designs a high-performance deep convolutional network (DeepID2+) for face recognition that is learned with the identification-verification supervisory signal, and finds it is much more robust to occlusions, although occlusion patterns are not included in the training set.

Deep Learning Face Representation from Predicting 10,000 Classes

It is argued that DeepID can be effectively learned through challenging multi-class face identification tasks, whilst they can be generalized to other tasks (such as verification) and new identities unseen in the training set.

FaceNet: A unified embedding for face recognition and clustering

A system that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure offace similarity, and achieves state-of-the-art face recognition performance using only 128-bytes perface.

Web-scale training for face identification

A link between the representation norm and the ability to discriminate in a target domain is found, which sheds lights on how deep convolutional networks represent faces.

Deep Learning Face Representation by Joint Identification-Verification

This paper shows that the face identification-verification task can be well solved with deep learning and using both face identification and verification signals as supervision, and the error rate has been significantly reduced.

DeepFace: Closing the Gap to Human-Level Performance in Face Verification

This work revisits both the alignment step and the representation step by employing explicit 3D face modeling in order to apply a piecewise affine transformation, and derive a face representation from a nine-layer deep neural network.

Very Deep Convolutional Networks for Large-Scale Image Recognition

This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.

Data-free Parameter Pruning for Deep Neural Networks

It is shown how similar neurons are redundant, and a systematic way to remove them is proposed, which can be applied on top of most networks with a fully connected layer to give a smaller network.

A Practical Transfer Learning Algorithm for Face Verification

This work proposes a principled transfer learning approach for merging plentiful source-domain data with limited samples from some target domain of interest to create a classifier that ideally performs nearly as well as if rich target- domain data were present.

Memory Bounded Deep Convolutional Networks

This work investigates the use of sparsity-inducing regularizers during training of Convolution Neural Networks and shows that training with such regularization can still be performed using stochastic gradient descent implying that it can be used easily in existing codebases.