In the context of artificial neural networks, the rectifier is an activation function defined as where x is the input to a neuron. This is also known… (More)

Semantic Scholar uses AI to extract papers important to this topic.

Highly Cited

2017

Highly Cited

2017

Deep Learning has revolutionized vision via convolutional neural networks (CNNs) and natural language processing via recurrent… (More)

Is this relevant?

2016

2016

- Lie Xu, Oliver Chiu-sing Choy, Yi-Wen Li
- 2016 IEEE International Workshop on Acoustic…
- 2016

Deep neural networks (DNNs) have been widely applied in speech recognition and enhancement. In this paper we present some… (More)

Is this relevant?

Highly Cited

2016

Highly Cited

2016

- Song Han, Xingyu Liu, +4 authors William J. Dally
- 2016 ACM/IEEE 43rd Annual International Symposium…
- 2016

State-of-the-art deep neural networks (DNNs) have hundreds of millions of connections and are both computationally and memory… (More)

Is this relevant?

Highly Cited

2016

Highly Cited

2016

- Matus Telgarsky
- COLT
- 2016

For any positive integer k, there exist neural networks with Θ(k) layers, Θ(1) nodes per layer, and Θ(1) distinct parameters… (More)

Is this relevant?

Highly Cited

2015

Highly Cited

2015

- Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
- 2015 IEEE International Conference on Computer…
- 2015

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier… (More)

Is this relevant?

Highly Cited

2015

Highly Cited

2015

- Bing Xu, Naiyan Wang, Tianqi Chen, Mu Li
- ArXiv
- 2015

In this paper we investigate the performance of different types of rectified activation functions in convolutional neural network… (More)

Is this relevant?

Highly Cited

2015

Highly Cited

2015

- Djork-Arné Clevert, Thomas Unterthiner, Sepp Hochreiter
- ArXiv
- 2015

We introduce the “exponential linear unit” (ELU) which speeds up learning in deep neural networks and leads to higher… (More)

Is this relevant?

Highly Cited

2014

Highly Cited

2014

- Xiaohui Zhang, Jan Trmal, Daniel Povey, Sanjeev Khudanpur
- 2014 IEEE International Conference on Acoustics…
- 2014

Recently, maxout networks have brought significant improvements to various speech recognition and computer vision tasks. In this… (More)

Is this relevant?

Highly Cited

2014

Highly Cited

2014

- Roi Livni, Shai Shalev-Shwartz, Ohad Shamir
- NIPS
- 2014

It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural… (More)

Is this relevant?

Highly Cited

2013

Highly Cited

2013

- George E. Dahl, Tara N. Sainath, Geoffrey E. Hinton
- 2013 IEEE International Conference on Acoustics…
- 2013

Recently, pre-trained deep neural networks (DNNs) have outperformed traditional acoustic models based on Gaussian mixture models… (More)

Is this relevant?