A parallel implementation of Singular Value Decomposition based on Map-Reduce and PARPACK

  title={A parallel implementation of Singular Value Decomposition based on Map-Reduce and PARPACK},
  author={Y. Ding and Guofeng Zhu and Chenyang Cui and Jian Zhou and L. Tao},
  journal={Proceedings of 2011 International Conference on Computer Science and Network Technology},
  • Y. Ding, Guofeng Zhu, +2 authors L. Tao
  • Published 2011
  • Computer Science
  • Proceedings of 2011 International Conference on Computer Science and Network Technology
In the e-commerce on the Web,recommender systems become a powerful technology for extracting valuable information from its customer databases. These systems also help customers find products they want to buy from a business sites. Singular Value Decomposition(SVD) is a useful technology to speedup the recommendations with very fast online performance, requiring just a few simple arithmetic operations. Unfortunately, computing the SVD of a large scale matrix is very expensive. In this paper, we… Expand
Divide‐and‐conquer approach for solving singular value decomposition based on MapReduce
A MapReduce‐based implementation for solving divide‐and‐conquer SVD algorithm that can accommodate full rank SVD computation of large‐scale matrix very efficiently and experimental result shows the efficiency of the algorithm. Expand
Unsupervised dimensionality reduction for very large datasets: Are we going to the right direction?
The results indicate that, at least for large datasets of dimensionality with up to ∼ 1, 000 attributes, the proposed Fractal-based algorithm is the best option, as opposed to the standard variance-preservation strategy that presented considerably worse results. Expand
Effective and Unsupervised Fractal-Based Feature Selection for Very Large Datasets: Removing Linear and Non-linear Attribute Correlations
Curl-Remover eliminates linear and non-linear attribute correlations as well as irrelevant ones, it is unsupervised and suits for analytical tasks in general – not only classification, it presents linear scale-up, it does not require the user to guess the number of attributes to be removed, and, it preserves the attributes' semantics. Expand
Enhanced movie content similarity based on textual, auditory and visual information
This is the first approach that utilizes a wide range of features from all involved modalities, in order to enhance the performance of the content similarity estimation, compared to the metadata-based approaches. Expand
Empirical study using network of semantically related associations in bridging the knowledge gap
An integrated system, such as ARIANA, could assist the human expert in exploratory literature search by bringing forward hidden associations, promoting data reuse and knowledge discovery as well as stimulating interdisciplinary projects by connecting information across the disciplines. Expand


A new singular value decomposition algorithm suited to parallelization and preliminary results
This paper proposes a new singular value decomposition algorithm which can be fully parallelized. I-SVD algorithm was recently proposed using the theory of integrable systems. It computes singularExpand
Application of Dimensionality Reduction in Recommender System - A Case Study
This paper presents two different experiments where one technology called Singular Value Decomposition (SVD) is explored to reduce the dimensionality of recommender system databases and suggests that SVD has the potential to meet many of the challenges ofRecommender systems, under certain conditions. Expand
MapReduce: Simplified Data Processing on Large Clusters
This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable. Expand
Analysis of recommendation algorithms for e-commerce
This paper investigates several te hniques for analyzing large-s ale pur hase and preferen e data for the purpose of producing useful re ommendations to ustomers and devise and apply their ombinations on the authors' data sets to ompare for re Ommendation quality and performan e. Expand
ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods
The Arnoldi factorization, the implicitly restarted Arnoldi method: structure of the Eigenvalue problem Krylov subspaces and projection methods, and more. Expand
A portable implementation of ARPACK for distributed memory parallel architectures
A parallel implementation of ARPACK is presented which is portable across a wide range of distributed memory platforms and requires minimal changes to the serial code. Expand
ARPACK users& guide
  • Siam
  • 1998
ARPACK users& guide. Siam
  • ARPACK users& guide. Siam
  • 1998