Optimal Algorithms for L1-subspace Signal Processing

@article{Markopoulos2014OptimalAF,
  title={Optimal Algorithms for L1-subspace Signal Processing},
  author={Panos P. Markopoulos and George N. Karystinos and Dimitris A. Pados},
  journal={IEEE Transactions on Signal Processing},
  year={2014},
  volume={62},
  pages={5046-5058}
}
We describe ways to define and calculate L1-norm signal subspaces that are less sensitive to outlying data than L2-calculated subspaces. We start with the computation of the L1 maximum-projection principal component of a data matrix containing N signal samples of dimension D. We show that while the general problem is formally NP-hard in asymptotically large N, D, the case of engineering interest of fixed dimension D and asymptotically large sample size N is not. In particular, for the case… 

Figures and Tables from this paper

Some Options for L1-subspace Signal Processing

It is proved that the case of engineering interest of fixed dimension D and asymptotically large sample support N is not NP-hard and an optimal algorithm of complexity of complexity $O(N^D)$ is presented.

Fast computation of the L1-principal component of real-valued data

This paper presents for the first time in the literature a fast greedy single-bit-flipping conditionally optimal iterative algorithm for the computation of the L1 principal component with complexity O(N3) and demonstrates the effectiveness of the developed algorithm with applications to the general field of data dimensionality reduction and direction-of-arrival estimation.

L1-Norm Principal-Component Analysis via Bit Flipping

L1-BF is presented: a novel, near-optimal algorithm that calculates the K L1-PCs of X with cost O (NDmin{N, D} + N2(K4 + DK2) + DNK3), comparable to that of standard (L2-norm) Principal-Component Analysis.

Optimal Algorithms for Binary, Sparse, and L 1 -Norm Principal Component Analysis

This work shows that in all these problems, the optimal solution can be obtained in polynomial time if the rank of the data matrix is constant, and presents optimal algorithms that are fully parallelizable and memory efficient, hence readily implementable.

L1-norm principal-component analysis in L2-norm-reduced-rank data subspaces

Reduced-rank L1-PCA aims at leveraging both the low computational cost of standard PCA and the outlier-resistance of L2-norm-based rank-d approximation, calculable exactly with reduced complexity O(N(d-1)K+1).

Computational advances in sparse L1-norm principal-component analysis of multi-dimensional data

  • Shubham ChamadiaD. Pados
  • Computer Science
    2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)
  • 2017
An efficient suboptimal algorithm of complexity O(N<sup>2</sup>(N + D) is presented and its strong resistance to faulty measurements/outliers in the data matrix is demonstrated.

Estimating L 1-Norm Best-Fit Lines for Data

This paper presents a procedure to estimate the L1-norm best-fit onedimensional subspace (a line through the origin) to data in < based on an optimization criterion involving linear programming but which can be performed using simple ratios and sortings.

A Simple and Fast Algorithm for L1-Norm Kernel PCA

A novel reformulation of L1-norm kernel PCA is provided through which an equivalent, geometrically interpretable problem is obtained and a “fixed-point” type algorithm that iteratively computes a binary weight for each observation is presented.

Low rank approximation with entrywise l1-norm error

The first provable approximation algorithms for ℓ1-low rank approximation are given, showing that it is possible to achieve approximation factor α = (logd) #183; poly(k) in nnz(A) + (n+d) poly( k) time, and improving the approximation ratio to O(1) with a poly(nd)-time algorithm.

Adaptive L1-Norm Principal-Component Analysis With Online Outlier Rejection

This paper proposes new methods for both incremental and adaptive L1-PCA, and combines the merits of the first one with the additional ability to track changes in the nominal signal subspace.
...

References

SHOWING 1-10 OF 56 REFERENCES

Robust subspace computation using L1 norm

This paper presents two algorithms to optimize the L1 norm metric: the weighted median algorithm and the quadratic programming algorithm, and shows that it is robust to outliers and can handle missing data.

Efficient computation of robust low-rank matrix approximations in the presence of missing data using the L1 norm

This paper presents a method for calculating the low-rank factorization of a matrix which minimizes the L1 norm in the presence of missing data and shows that the proposed algorithm can be efficiently implemented using existing optimization software.

A pure L1L1-norm principal component analysis

Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization

Experimental results on real world datasets show that the nongreedy method always obtains much better solution than that of the greedy method, and then a robust principal component analysis with non-greedy l1-norm maximization is proposed.

An efficient algorithm for L1-norm principal component analysis

  • L. YuMiao ZhangC. Ding
  • Computer Science
    2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2012
Numerical and visual results show that L1-PCA is consistently better than standard PCA, and the robustness against image occlusions is verified.

R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization

Experiments on several real-life datasets show R1-PCA can effectively handle outliers and it is shown that L1-norm K-means leads to poor results while R2-K-MEans outperforms standard K-Means.

On first-order algorithms for l1/nuclear norm minimization

This paper gives a detailed description of two attractive first-order optimization techniques for solving problems of l1/nuclear norm minimization as ‘optimization beasts’ and discusses the application domains.

Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming

  • Qifa KeT. Kanade
  • Computer Science
    2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)
  • 2005
This paper forms matrix factorization as a L/sub 1/ norm minimization problem that is solved efficiently by alternative convex programming that is robust without requiring initial weighting, handles missing data straightforwardly, and provides a framework in which constraints and prior knowledge can be conveniently incorporated.

Linear discriminant analysis using rotational invariant L1 norm

...