Time for dithering: fast and quantized random embeddings via the restricted isometry property

@article{Jacques2016TimeFD,
  title={Time for dithering: fast and quantized random embeddings via the restricted isometry property},
  author={Laurent Jacques and Valerio Cambareri},
  journal={ArXiv},
  year={2016},
  volume={abs/1607.00816}
}
Recently, many works have focused on the characterization of non-linear dimensionality reduction methods obtained by quantizing linear embeddings, e.g., to reach fast processing time, efficient data compression procedures, novel geometry-preserving embeddings or to estimate the information/bits stored in this reduced data representation. In this work, we prove that many linear maps known to respect the restricted isometry property (RIP) can induce a quantized random embedding with controllable… 

Tables from this paper

Fast Binary Embeddings and Quantized Compressed Sensing with Structured Matrices
TLDR
This paper proposes fast methods to replace points from a subset of Χ, associated with the euclidean metric, with points in the cube {±1}m, and it is the first such binary embedding result that applies to fast Johnson‐Lindenstrauss maps while preserving ℓ2 norms.
Fast metric embedding into the Hamming cube
TLDR
It is shown that the double circulant matrix mimics the behavior of a Gaussian matrix in two important ways: first, it yields an almost isometric embedding of any subset of ℓ n 2 intoℓ m 1 and, second, it maps an arbitrary set in R n into a set of well-spread vectors.
Near-optimal sample complexity bounds for circulant binary embedding
TLDR
This work builds upon the recent literature to obtain significantly better dependencies on the problem parameters and believes the proposed techniques can be useful to obtain improved guarantees for other nonlinear embedding problems.
Binarized Johnson-Lindenstrauss embeddings
We consider the problem of encoding a set of vectors into a minimal number of bits while preserving information on their Euclidean geometry. We show that this task can be accomplished by applying a
Quantized Compressive Sensing with RIP Matrices: The Benefit of Dithering
TLDR
This work shows that, for a scalar and uniform quantization, provided that a uniform random vector, or "random dithering", is added to the compressive measurements of a low-complexity signal, a large class of random matrix constructions known to respect the restricted isometry property (RIP) are made "compatible" with this quantizer.
Taking the Edge off Quantization: Projected Back Projection in Dithered Compressive Sensing
TLDR
It is shown here that a simple uniform scalar quantizer is compatible with a large class of random sensing matrices known to respect, with high probability, the restricted isometry property (RIP), and it is validated numerically the predicted error decay as the number of measurements increases.
The rare eclipse problem on tiles: Quantised embeddings of disjoint convex sets
TLDR
This work verifies when this quantised random embeddings technique preserves the separability of two disjoint closed convex sets in a quantised view of the “rare eclipse problem” introduced by Bandeira et al. in 2014.
Rare Eclipses in Quantised Random Embeddings of Disjoint Convex Sets: a Matter of Consistency?
We study the problem of verifying when two disjoint closed convex sets remain separable after the application of a quantised random embedding, as a means to ensure exact classification from the
Breaking the waves: asymmetric random periodic features for low-bitrate kernel machines
TLDR
The take-home message is that when the features of only one of the two signals are quantized, the original kernel is recovered without distortion; its practical interest appears in several cases where the kernel evaluations are asymmetric by nature, such as a client-server scheme.
Sample Complexity Bounds for 1-bit Compressive Sensing and Binary Stable Embeddings with Generative Priors
TLDR
It is demonstrated that the Binary $\epsilon$-Stable Embedding property, which characterizes the robustness of the reconstruction to measurement errors and noise, also holds for 1-bit compressive sensing with Lipschitz continuous generative models with sufficiently many Gaussian measurements.
...
...

References

SHOWING 1-10 OF 83 REFERENCES
On Binary Embedding using Circulant Matrices
TLDR
This work proposes Circulant Binary Embedding (CBE) which generates binary codes by projecting the data with a circulant matrix, and designs a novel time-frequency alternating optimization to learn data-dependent circulants projections, which alternatively minimizes the objective in original and Fourier domains.
Near-optimal sample complexity bounds for circulant binary embedding
TLDR
This work builds upon the recent literature to obtain significantly better dependencies on the problem parameters and believes the proposed techniques can be useful to obtain improved guarantees for other nonlinear embedding problems.
Circulant Binary Embedding
TLDR
This work proposes Circulant Binary Embedding (CBE), which generates binary codes by projecting the data with a circulant matrix, and proposes a novel time-frequency alternating optimization to learn data-dependentcirculant projections, which alternatively minimizes the objective in original and Fourier domains.
Small Width, Low Distortions: Quantized Random Embeddings of Low-Complexity Sets
  • L. Jacques
  • Computer Science
    IEEE Transactions on Information Theory
  • 2017
TLDR
This work addresses the general question of pairwise distances of low-complexity vectors through the specific use of a quantized and dithered random linear mapping, which combines a sub-Gaussian random projection and an embedding of a bounded set of vectors.
A Simple Proof of the Restricted Isometry Property for Random Matrices
Abstract We give a simple technique for verifying the Restricted Isometry Property (as introduced by Candès and Tao) for random matrices that underlies Compressed Sensing. Our approach has two main
Improved Approximation Algorithms for Large Matrices via Random Projections
  • Tamás Sarlós
  • Computer Science
    2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06)
  • 2006
TLDR
The key idea is that low dimensional embeddings can be used to eliminate data dependence and provide more versatile, linear time pass efficient matrix computation.
Combining geometry and combinatorics: A unified approach to sparse signal recovery
TLDR
A unification of geometric and combinatorial approaches to sparse signal recovery is presented, which results in new measurement matrix constructions and algorithms for signal recovery which are superior in either the number of measurements or computational efficiency of decoders.
Isometric sketching of any set via the Restricted Isometry Property
TLDR
This paper shows that for the purposes of dimensionality reduction certain class of structured random matrices behave similarly to random Gaussian matrices, providing efficientdimensionality reduction of general sets.
Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors
TLDR
This paper investigates an alternative CS approach that shifts the emphasis from the sampling rate to the number of bits per measurement, and introduces the binary iterative hard thresholding algorithm for signal reconstruction from 1-bit measurements that offers state-of-the-art performance.
...
...