#### Filter Results:

- Full text PDF available (67)

#### Publication Year

2000

2017

- This year (5)
- Last 5 years (56)
- Last 10 years (69)

#### Publication Type

#### Co-author

#### Journals and Conferences

#### Data Set Used

#### Key Phrases

Learn More

- John C. Duchi, Elad Hazan, Yoram Singer
- COLT
- 2010

We present a new family of subgradient methods that dynamically incorporate knowledge of the geometry of the data observed in earlier iterations to perform more informative gradientbased learning. Metaphorically, the adaptation allows us to find needles in haystacks in the form of very predictive but rarely seen features. Our paradigm stems from recent… (More)

We describe efficient algorithms for projecting a vector onto the <i>l</i><sub>1</sub>-ball. We present two methods for projection. The first performs exact projection in <i>O(n)</i> expected time, where <i>n</i> is the dimension of the space. The second works on vectors <i>k</i> of whose elements are perturbed outside the <i>l</i><sub>1</sub>-ball,… (More)

- John C. Duchi, Yoram Singer
- Journal of Machine Learning Research
- 2009

We describe, analyze, and experiment with a framework for empirical loss minimization with regularization. Our algorithmic framework alternates between two phases. On each iteration we first perform an unconstrained gradient descent step. We then cast and solve an instantaneous optimization problem that trades off minimization of a regularization term while… (More)

- John C. Duchi, Alekh Agarwal, Martin J. Wainwright
- IEEE Trans. Automat. Contr.
- 2012

The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multiagent co-ordination, estimation in sensor networks, and… (More)

- Alekh Agarwal, John C. Duchi
- NIPS
- 2011

We analyze the convergence of gradient-based optimization algorithms that base their updates on delayed stochastic gradient information. The main application of our results is to the development of gradient-based distributed optimization algorithms where a master node performs parameter updates while worker nodes compute stochastic gradients based on local… (More)

- John C. Duchi, Shai Shalev-Shwartz, Yoram Singer, Ambuj Tewari
- COLT
- 2010

We present a new method for regularized convex optimization and analyze it under both online and stochastic optimization settings. In addition to unifying previously known firstorder algorithms, such as the projected gradient method, mirror descent, and forwardbackward splitting, our method yields new analysis and algorithms. We also derive specific… (More)

- John C. Duchi, Stephen Gould, Daphne Koller
- UAI
- 2008

Gaussian Markov random fields (GMRFs) are useful in a broad range of applications. In this paper we tackle the problem of learning a sparse GMRF in a high-dimensional space. Our approach uses the l1-norm as a regularization on the inverse covariance matrix. We utilize a novel projected gradient method, which is faster than previous methods in practice and… (More)

- Yuchen Zhang, John C. Duchi, Martin J. Wainwright
- Journal of Machine Learning Research
- 2012

We study two communication-efficient algorithms for distributed statistical optimization on large-scale data. The first algorithm is an averaging method that distributes the N data samples evenly to m machines, performs separate minimization on each subset, and then averages the estimates. We provide a sharp analysis of this average mixture algorithm,… (More)

- John C. Duchi, Lester W. Mackey, Michael I. Jordan
- ICML
- 2010

We present a theoretical analysis of supervised ranking, providing necessary and sufficient conditions for the asymptotic consistency of algorithms based on minimizing a surrogate loss function. We show that many commonly used surrogate losses are inconsistent; surprisingly, we show inconsistency even in low-noise settings. We present a new… (More)

Machine learning (ML) and statistical techniques are key to transforming big data into actionable knowledge. In spite of the modern primacy of data, the complexity of existing ML algorithms is often overwhelming—many users do not understand the trade-offs and challenges of parameterizing and choosing between different learning techniques. Furthermore,… (More)