Softmax function

Known as: Normalized exponential, Softmax, Softmax activation function 
In mathematics, in particular probability theory and related fields, the softmax function, or normalized exponential, is a generalization of the… (More)
Wikipedia

Related topics

Related topics

18 relations
Activation functionArtificial neural networkConvolutional neural networkCross entropy
(More)

Broader (1)

Computational neuroscience

Papers overview

Semantic Scholar uses AI to extract papers important to this topic.
Highly Cited
2018
Highly Cited
2018
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models… (More)
  • table 1
  • table 2
  • table 3
  • table 4
  • table 5
Is this relevant?
Highly Cited
2017
Highly Cited
2017
L2-constrained Softmax Loss for Discriminative Face Verification
In recent years, the performance of face verification systems has significantly improved using deep convolutional neural networks… (More)
  • figure 1
  • figure 2
  • figure 3
  • figure 4
  • figure 5
Is this relevant?
Highly Cited
2017
Highly Cited
2017
Efficient softmax approximation for GPUs
We propose an approximate strategy to efficiently train neural network based language models over very large vocabularies. Our… (More)
  • figure 1
  • figure 2
  • table 1
  • figure 3
  • table 3
Is this relevant?
2017
2017
Incrementally Learning the Hierarchical Softmax Function for Neural Language Models
Neural network language models (NNLMs) have attracted a lot of attention recently. In this paper, we present a training method… (More)
  • figure 1
  • figure 2
  • table 1
  • figure 3
  • figure 4
Is this relevant?
Highly Cited
2016
Highly Cited
2016
Categorical Reparameterization with Gumbel-Softmax
Categorical variables are a natural choice for representing discrete structure in the world. However, stochastic neural networks… (More)
  • figure 1
  • figure 2
  • figure 3
  • figure 4
  • table 1
Is this relevant?
Highly Cited
2016
Highly Cited
2016
Large-Margin Softmax Loss for Convolutional Neural Networks
Cross-entropy loss together with softmax is arguably one of the most common used supervision components in convolutional neural… (More)
  • figure 1
  • figure 2
  • figure 3
  • figure 4
  • table 1
Is this relevant?
Highly Cited
2013
Highly Cited
2013
Distributed Representations of Words and Phrases and their Compositionality
The recently introduced continuous Skip-gram model is an ef fici nt method for learning high-quality distributed vector… (More)
  • figure 1
  • figure 2
  • table 1
  • table 2
  • table 3
Is this relevant?
Highly Cited
2010
Highly Cited
2010
Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions
We describe a method of incorporating taskspecific cost functions into standard conditional log-likelihood (CLL) training of… (More)
  • figure 1
  • table 1
Is this relevant?
Highly Cited
2009
Highly Cited
2009
Replicated Softmax: an Undirected Topic Model
We introduce a two-layer undirected graphical model, calle d a “Replicated Softmax”, that can be used to model and automatically… (More)
  • figure 1
  • table 1
  • figure 2
  • figure 3
  • figure 4
Is this relevant?
Review
2007
Review
2007
Efficient Bounds for the Softmax Function and Applications to Approximate Inference in Hybrid models
The softmax link is used in many probabilistic model dealing with both discrete and continuous data. However, efficient Bayesian… (More)
  • table 1
  • table 2
Is this relevant?