M. Smelyanskiy

Publications
Influence

On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima

N. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, P. Tang
Computer Science, Mathematics
ICLR
15 September 2016

We present numerical evidence that large-batch methods tend to converge to sharp minimizers of the training and testing functions - and as is well known, sharp minima lead to poorer generalization. Expand

1,161
124
PDF

Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU

V. Lee, C. Kim, +9 authors P. Dubey
Computer Science
ISCA
19 June 2010

An analysis of a set of important throughput computing kernels shows that there is an ample amount of parallelism in these kernels which makes them suitable for today's multi-core CPUs and GPUs. Expand

Efficient sparse matrix-vector multiplication on x86-based many-core processors

X. Liu, M. Smelyanskiy, E. Chow, P. Dubey
Computer Science
ICS '13
10 June 2013

We identify and address several bottlenecks which may limit performance even before memory bandwidth: (a) low SIMD efficiency due to sparsity, (b) overhead due to irregular memory accesses, and (c) load-imbalance due to non-uniform matrix structures. Expand

qHiPSTER: The Quantum High Performance Software Testing Environment

M. Smelyanskiy, Nicolas P. D. Sawaya, Alán Aspuru-Guzik
Physics, Computer Science
ArXiv
26 January 2016

We present qHiPSTER, the Quantum High Performance Software Testing Environment. Expand

Convergence of Recognition, Mining, and Synthesis Workloads and Its Implications

Y. Chen, J. Chhugani, +6 authors M. Smelyanskiy
Computer Science
Proceedings of the IEEE
15 April 2008

This paper examines the growing need for a general-purpose ldquoanalytics enginerdquo that can enable next-generation processing platforms to effectively model events, objects, and concepts based on end-user input, and accessible datasets, along with an ability to iteratively refine the model in real-time. Expand

Design and Implementation of the Linpack Benchmark for Single and Multi-node Systems Based on Intel® Xeon Phi Coprocessor

A. Heinecke, K. Vaidyanathan, +6 authors P. Dubey
Computer Science
IEEE 27th International Symposium on Parallel and…
20 May 2013

Dense linear algebra has been traditionally used to evaluate the performance and efficiency of new architectures. Expand

The Architectural Implications of Facebook's DNN-Based Personalized Recommendation

Udit Gupta, X. Wang, +12 authors X. Zhang
Computer Science
IEEE International Symposium on High Performance…
6 June 2019

The widespread application of deep learning has changed the landscape of computation in data centers. Expand

Anatomy of High-Performance Many-Threaded Matrix Multiplication

Tyler M. Smith, R. Geijn, M. Smelyanskiy, J. Hammond, F. V. Zee
Computer Science
IEEE 28th International Parallel and Distributed…
19 May 2014

BLIS is a new framework for rapid instantiation of the BLAS. Expand

Sparsifying Synchronization for High-Performance Shared-Memory Sparse Triangular Solver

Jongsoo Park, M. Smelyanskiy, N. Sundaram, P. Dubey
Computer Science
ISC
22 June 2014

The last decade has seen rapid growth of single-chip multiprocessors CMPs, which have been leveraging Moore's law to deliver high concurrency via increases in the number of cores and vector width. Expand

High Performance Parallel Stochastic Gradient Descent in Shared Memory

S. Sallinen, Nadathur Satish, M. Smelyanskiy, Samantika Sury, C. Ré
Computer Science
IEEE International Parallel and Distributed…
23 May 2016

In this paper, we explore several modern parallelization methods of SGD on a shared memory system, in the context of sparse and convex optimization problems, and show that their parallel efficiency is severely limited by inter-core communication. Expand

Recommended Authors