Corpus ID: 221113156

Mish: A Self Regularized Non-Monotonic Activation Function

@inproceedings{Misra2020MishAS,
  title={Mish: A Self Regularized Non-Monotonic Activation Function},
  author={Diganta Misra},
  booktitle={BMVC},
  year={2020}
}
  • Diganta Misra
  • Published in BMVC 2020
  • Computer Science, Mathematics
  • We propose Mish, a novel self-regularized non-monotonic activation function which can be mathematically defined as: f (x) = x tanh(so f t plus(x)). As activation functions play a crucial role in the performance and training dynamics in neural networks, we validated experimentally on several well-known benchmarks against the best combinations of architectures and activation functions. We also observe that data augmentation techniques have a favorable effect on benchmarks like ImageNet-1k and MS… CONTINUE READING
    33 Citations
    TanhSoft - a family of activation functions combining Tanh and Softplus
    • PDF
    Soft-Root-Sign Activation Function
    • 3
    • PDF
    Discovering Parametric Activation Functions
    • 1
    • PDF
    Dynamic ReLU
    • 2
    • PDF
    Universal Activation Function For Machine Learning
    • PDF
    YOLOv4: Optimal Speed and Accuracy of Object Detection
    • 242
    • PDF
    On the Universality of the Double Descent Peak in Ridgeless Regression

    References

    SHOWING 1-10 OF 53 REFERENCES
    Searching for Activation Functions
    • 728
    Empirical Evaluation of Rectified Activations in Convolutional Network
    • 1,192
    • PDF
    Adam: A Method for Stochastic Optimization
    • 56,546
    • PDF
    Aggregated Residual Transformations for Deep Neural Networks
    • 3,140
    • PDF
    Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
    • 2,607
    • Highly Influential
    • PDF
    YOLOv4: Optimal Speed and Accuracy of Object Detection
    • 242
    • Highly Influential
    • PDF
    Visualizing the Loss Landscape of Neural Nets
    • 448
    • PDF
    Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
    • 8,832
    • PDF