Skip to search formSkip to main contentSkip to account menu
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
TLDR
This work investigates the cause for this generalization drop in the large-batch regime and presents numerical evidence that supports the view that large- batch methods tend to converge to sharp minimizers of the training and testing functions - and as is well known, sharp minima lead to poorer generalization.
View PDF on arXiv
Deep Learning Recommendation Model for Personalization and Recommendation Systems
TLDR
A state-of-the-art deep learning recommendation model (DLRM) is developed and its implementation in both PyTorch and Caffe2 frameworks is provided and a specialized parallelization scheme utilizing model parallelism on the embedding tables to mitigate memory constraints while exploiting data parallelism to scale-out compute from the fully-connected layers is designed.
View PDF on arXiv
Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU
TLDR
This paper discusses optimization techniques for both CPU and GPU, analyzes what architecture features contributed to performance differences between the two architectures, and recommends a set of architectural features which provide significant improvement in architectural efficiency for throughput kernels.
View on ACM
Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective
TLDR
The hardware and software infrastructure that supports machine learning at global scale is described, leveraging both GPU and CPU platforms for training and abundant CPU capacity for real-time inference.
View on IEEE
Glow: Graph Lowering Compiler Techniques for Neural Networks
TLDR
Glow features a lowering phase which enables the compiler to support a high number of input operators as well as a large number of hardware targets by eliminating the need to implement all operators on all targets.
View PDF on arXiv
The Architectural Implications of Facebook's DNN-Based Personalized Recommendation
TLDR
A set of real-world, production-scale DNNs for personalized recommendation coupled with relevant performance metrics for evaluation are presented and in-depth analysis is conducted that underpins future system design and optimization for at-scale recommendation.
View on IEEE
Efficient sparse matrix-vector multiplication on x86-based many-core processors
TLDR
This work describes an efficient implementation of SpMV on the IntelR Xeon PhiTM Coprocessor, codenamed Knights Corner (KNC), that exploits the salient architectural features of KNC, such as large caches and hardware support for irregular memory accesses.
View on ACM
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
TLDR
RecNMP is proposed which provides a scalable solution to improve system throughput, supporting a broad range of sparse embedding models, and is specifically tailored to production environments with heavy co-location of operators on a single server.
View on ACM
qHiPSTER: The Quantum High Performance Software Testing Environment
We present qHiPSTER, the Quantum High Performance Software Testing Environment. qHiPSTER is a distributed high-performance implementation of a quantum simulator on a classical computer, that can…
View PDF on arXiv
Convergence of Recognition, Mining, and Synthesis Workloads and Its Implications
TLDR
A diverse set of emerging RMS applications from market segments like graphics, gaming, media-mining, unstructured information management, financial analytics, and interactive virtual communities presents a relatively focused, highly overlapping set of common platform challenges.
View on IEEE
...
...
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy, Terms of Service, and Dataset License