Learning ReLUs via Gradient Descent

  title={Learning ReLUs via Gradient Descent},
  author={Mahdi Soltanolkotabi},
In this paper we study the problem of learning Rectified Linear Units (ReLUs) which are functions of the form x ↦ max(0, ⟨w,x⟩) with w ∈ R denoting the weight vector. We study this problem in the high-dimensional regime where the number of observations are fewer than the dimension of the weight vector. We assume that the weight vector belongs to some closed set (convex or nonconvex) which captures known side-information about its structure. We focus on the realizable model where the inputs are… CONTINUE READING
Highly Cited
This paper has 46 citations. REVIEW CITATIONS

From This Paper

Figures and tables from this paper.


Publications citing this paper.

Similar Papers

Loading similar papers…