# Greed is good: algorithmic results for sparse approximation

@article{Tropp2004GreedIG,
title={Greed is good: algorithmic results for sparse approximation},
author={Joel A. Tropp},
journal={IEEE Transactions on Information Theory},
year={2004},
volume={50},
pages={2231-2242}
}
• J. Tropp
• Published 1 October 2004
• Computer Science
• IEEE Transactions on Information Theory
This article presents new results on using a greedy algorithm, orthogonal matching pursuit (OMP), to solve the sparse approximation problem over redundant dictionaries. It provides a sufficient condition under which both OMP and Donoho's basis pursuit (BP) paradigm can recover the optimal representation of an exactly sparse signal. It leverages this theory to show that both OMP and BP succeed for every sparse input signal from a wide class of dictionaries. These quasi-incoherent dictionaries…
3,457 Citations

## Figures from this paper

A simple test to check the optimality of sparse signal approximations
• Computer Science
Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005.
• 2005
This paper provides a simple test to check whether the output of a sparse approximation algorithm is nearly optimal, in the sense that no significantly different linear expansion from the dictionary can provide both a smaller approximation error and a better sparsity.
A fixed-point iterative schema for error minimization in k-sparse decomposition
• Computer Science
2011 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT)
• 2011
A new algorithm to solve the sparse approximation problem over redundant dictionaries where the input signal is restricted to be a linear combination of k atoms or fewer from a fixed dictionary and it outperforms OMP method both regarding sparse approximation error and computation time.
Beyond sparsity: Recovering structured representations by ${\ell}^1$ minimization and greedy algorithms
• Computer Science
• 2008
Finding a sparse approximation of a signal from an arbitrary dictionary is a very useful tool to solve many problems in signal processing. Several algorithms, such as Basis Pursuit (BP) and Matching
A Fast Algorithm for Learning Overcomplete Dictionary for Sparse Representation Based on Proximal Operators
• Computer Science
Neural Computation
• 2015
A fast, efficient algorithm for learning an overcomplete dictionary for sparse representation of signals that has lower computational complexity and a higher convergence rate than state-of-the-art algorithms.
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
• Computer Science
IEEE Transactions on Signal Processing
• 2006
A novel algorithm for adapting dictionaries in order to achieve sparse signal representations, the K-SVD algorithm, an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data.
Tight recovery guarantees for orthogonal matching pursuit under Gaussian noise
• Computer Science
• 2019
A slightly sharper sufficient condition is derived for exact support recovery by OMP with high probability depending on the signal-to-noise ratio, defined as the magnitude of the smallest non-zero coefficient of the vector divided by the noise level.
A Weighted Average of Sparse Representations is Better than the Sparsest One Alone
• Computer Science
Structured Decompositions and Efficient Algorithms
• 2008
It is shown that while the Maximum a-posterior Probability estimator aims to find and use the sparsest representation, the Minimum Mean-Squared-Error (MMSE) estimator leads to a fusion of representations to form its result, which is a far more accurate estimation, especially at medium and low SNR.

## References

SHOWING 1-10 OF 44 REFERENCES
• Computer Science
• 1997
A notion of the coherence of a signal with respect to a dictionary is derived from the characterization of the approximation errors of a pursuit from their statistical properties, which can be obtained from the invariant measure of the pursuit.
JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION
• Computer Science
• 2004
It is demonstrated that the solution of the convex program frequently coincides with the solutionof the original approximation problem, and comparable new results for a greedy algorithm, Orthogonal Matching Pursuit, are stated.
Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization
• Computer Science
Proceedings of the National Academy of Sciences of the United States of America
• 2003
This article obtains parallel results in a more general setting, where the dictionary D can arise from two or several bases, frames, or even less structured systems, and sketches three applications: separating linear features from planar ones in 3D data, noncooperative multiuser encoding, and identification of over-complete independent component models.
Maximal Sparsity Representation via l 1 Minimization
• Computer Science
• 2002
This paper extends previous results and proves a similar relationship for the most general dictionary D and shows that previous results are emerging as special cases of the new extended theory.
Improved sparse approximation over quasiincoherent dictionaries
• Computer Science
Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429)
• 2003
A new greedy algorithm for solving the sparse approximation problem over quasiincoherent dictionaries that provides strong guarantees on the quality of the approximations it produces, unlike most other methods for sparse approximation.
Matching pursuits with time-frequency dictionaries
• Computer Science
IEEE Trans. Signal Process.
• 1993
The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions. These
Sparse representations in unions of bases
• Computer Science
IEEE Trans. Inf. Theory
• 2003
It is proved that the result of Donoho and Huo, concerning the replacement of the /spl lscr//sup 0/ optimization problem with a linear programming problem when searching for sparse representations has an analog for dictionaries that may be highly redundant.
Atomic Decomposition by Basis Pursuit
• Computer Science
SIAM J. Sci. Comput.
• 1998
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
Nonlinear approximation
• Computer Science, Mathematics
Acta Numerica
• 1998
This is a survey of nonlinear approximation, especially that part of the subject which is important in numerical computation, and emphasis will be placed on approximation by piecewise polynomials and wavelets as well as their numerical implementation.