Greed is good: algorithmic results for sparse approximation

@article{Tropp2004GreedIG,
  title={Greed is good: algorithmic results for sparse approximation},
  author={Joel A. Tropp},
  journal={IEEE Transactions on Information Theory},
  year={2004},
  volume={50},
  pages={2231-2242}
}
  • J. Tropp
  • Published 1 October 2004
  • Computer Science
  • IEEE Transactions on Information Theory
This article presents new results on using a greedy algorithm, orthogonal matching pursuit (OMP), to solve the sparse approximation problem over redundant dictionaries. It provides a sufficient condition under which both OMP and Donoho's basis pursuit (BP) paradigm can recover the optimal representation of an exactly sparse signal. It leverages this theory to show that both OMP and BP succeed for every sparse input signal from a wide class of dictionaries. These quasi-incoherent dictionaries… 
View on IEEE
users.cms.caltech.edu
3,457 Citations
Highly Influential Citations
352
Background Citations
1,360
Methods Citations
933
Results Citations
52

Figures from this paper

3,457 Citations

Citation Type

Citation Type

A simple test to check the optimality of a sparse signal approximation
A simple test to check the optimality of sparse signal approximations
TLDR
This paper provides a simple test to check whether the output of a sparse approximation algorithm is nearly optimal, in the sense that no significantly different linear expansion from the dictionary can provide both a smaller approximation error and a better sparsity.
A fixed-point iterative schema for error minimization in k-sparse decomposition
  • A. Adamo, G. Grossi
  • Computer Science
    2011 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT)
  • 2011
TLDR
A new algorithm to solve the sparse approximation problem over redundant dictionaries where the input signal is restricted to be a linear combination of k atoms or fewer from a fixed dictionary and it outperforms OMP method both regarding sparse approximation error and computation time.
Some greedy algorithms for sparse polynomial chaos expansions
Beyond sparsity: Recovering structured representations by ${\ell}^1$ minimization and greedy algorithms
Finding a sparse approximation of a signal from an arbitrary dictionary is a very useful tool to solve many problems in signal processing. Several algorithms, such as Basis Pursuit (BP) and Matching…
A Fast Algorithm for Learning Overcomplete Dictionary for Sparse Representation Based on Proximal Operators
TLDR
A fast, efficient algorithm for learning an overcomplete dictionary for sparse representation of signals that has lower computational complexity and a higher convergence rate than state-of-the-art algorithms.
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
TLDR
A novel algorithm for adapting dictionaries in order to achieve sparse signal representations, the K-SVD algorithm, an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data.
Tight recovery guarantees for orthogonal matching pursuit under Gaussian noise
TLDR
A slightly sharper sufficient condition is derived for exact support recovery by OMP with high probability depending on the signal-to-noise ratio, defined as the magnitude of the smallest non-zero coefficient of the vector divided by the noise level.
A Weighted Average of Sparse Representations is Better than the Sparsest One Alone
TLDR
It is shown that while the Maximum a-posterior Probability estimator aims to find and use the sparsest representation, the Minimum Mean-Squared-Error (MMSE) estimator leads to a fusion of representations to form its result, which is a far more accurate estimation, especially at medium and low SNR.
Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 44 REFERENCES
Adaptive greedy approximations
TLDR
A notion of the coherence of a signal with respect to a dictionary is derived from the characterization of the approximation errors of a pursuit from their statistical properties, which can be obtained from the invariant measure of the pursuit.
JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION
TLDR
It is demonstrated that the solution of the convex program frequently coincides with the solutionof the original approximation problem, and comparable new results for a greedy algorithm, Orthogonal Matching Pursuit, are stated.
Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization
  • D. Donoho, Michael Elad
  • Computer Science
    Proceedings of the National Academy of Sciences of the United States of America
  • 2003
TLDR
This article obtains parallel results in a more general setting, where the dictionary D can arise from two or several bases, frames, or even less structured systems, and sketches three applications: separating linear features from planar ones in 3D data, noncooperative multiuser encoding, and identification of over-complete independent component models.
On the stability of the basis pursuit in the presence of noise
Maximal Sparsity Representation via l 1 Minimization
TLDR
This paper extends previous results and proves a similar relationship for the most general dictionary D and shows that previous results are emerging as special cases of the new extended theory.
Improved sparse approximation over quasiincoherent dictionaries
TLDR
A new greedy algorithm for solving the sparse approximation problem over quasiincoherent dictionaries that provides strong guarantees on the quality of the approximations it produces, unlike most other methods for sparse approximation.
Matching pursuits with time-frequency dictionaries
The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions. These…
Sparse representations in unions of bases
TLDR
It is proved that the result of Donoho and Huo, concerning the replacement of the /spl lscr//sup 0/ optimization problem with a linear programming problem when searching for sparse representations has an analog for dictionaries that may be highly redundant.
Atomic Decomposition by Basis Pursuit
TLDR
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
Nonlinear approximation
TLDR
This is a survey of nonlinear approximation, especially that part of the subject which is important in numerical computation, and emphasis will be placed on approximation by piecewise polynomials and wavelets as well as their numerical implementation.
...
1
2
3
4
5
...