Rectifier (neural networks)

Known as: Rectifier (disambiguation), Softplus, ReLU 
In the context of artificial neural networks, the rectifier is an activation function defined as where x is the input to a neuron. This is also known… (More)
Wikipedia

Topic mentions per year

Topic mentions per year

2004-2018
020406020042018

Papers overview

Semantic Scholar uses AI to extract papers important to this topic.
Highly Cited
2017
Highly Cited
2017
Deep Learning has revolutionized vision via convolutional neural networks (CNNs) and natural language processing via recurrent… (More)
  • figure 1
  • figure 2
  • table 1
  • table 2
  • table 3
Is this relevant?
2016
2016
Deep neural networks (DNNs) have been widely applied in speech recognition and enhancement. In this paper we present some… (More)
  • figure 1
  • table I
  • table II
  • table III
  • figure 3
Is this relevant?
Highly Cited
2016
Highly Cited
2016
State-of-the-art deep neural networks (DNNs) have hundreds of millions of connections and are both computationally and memory… (More)
  • table II
  • figure 8
  • table 1
Is this relevant?
Highly Cited
2016
Highly Cited
2016
For any positive integer k, there exist neural networks with Θ(k) layers, Θ(1) nodes per layer, and Θ(1) distinct parameters… (More)
Is this relevant?
Highly Cited
2015
Highly Cited
2015
Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier… (More)
  • figure 1
  • table 1
  • table 2
  • figure 2
  • figure 3
Is this relevant?
Highly Cited
2015
Highly Cited
2015
In this paper we investigate the performance of different types of rectified activation functions in convolutional neural network… (More)
  • figure 1
  • table 1
  • table 2
  • figure 2
  • figure 3
Is this relevant?
Highly Cited
2015
Highly Cited
2015
We introduce the “exponential linear unit” (ELU) which speeds up learning in deep neural networks and leads to higher… (More)
  • figure 1
  • figure 2
  • figure 3
  • figure 4
  • figure 5
Is this relevant?
Highly Cited
2014
Highly Cited
2014
Recently, maxout networks have brought significant improvements to various speech recognition and computer vision tasks. In this… (More)
  • figure 1
  • figure 2
  • figure 3
  • figure 4
  • figure 5
Is this relevant?
Highly Cited
2014
Highly Cited
2014
It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural… (More)
  • figure 1
  • figure 2
Is this relevant?
Highly Cited
2013
Highly Cited
2013
Recently, pre-trained deep neural networks (DNNs) have outperformed traditional acoustic models based on Gaussian mixture models… (More)
  • table 1
  • table 2
Is this relevant?