• Publications
  • Influence
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
We study a D-PSGD algorithm and provide a theoretical analysis that indicates a regime in which decentralized algorithms might outperform centralized algorithms for distributed stochastic gradient descent. Expand
Asynchronous Decentralized Parallel Stochastic Gradient Descent
We propose an asynchronous decentralized stochastic gradient decent algorithm that achieves a similar epoch-wise convergence rate as AllReduce-SGD, at an over 100-GPU scale. Expand
Incremental knowledge base construction using DeepDive
We describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems and we present techniques to make the KBC process more efficient. Expand
Communication Compression for Decentralized Training
We develop a framework of quantized, decentralized training and propose two different strategies, which we call {\em extrapolation compression} and {\em difference compression}. Expand
Heterogeneity-aware Distributed Parameter Servers
We first conduct a systematic study of existing systems running distributed stochastic gradient descent; we find that, although these systems work well in homogeneous environments, they can suffer performance degradation, sometimes up to 10x, in heterogeneous environments. Expand
D2: Decentralized Training over Decentralized Data
We present D$^2$, a novel decentralized parallel stochastic gradient descent algorithm designed for large data variance among workers, which significantly outperforms D-PSGD. Expand
Asynchrony begets momentum, with an application to deep learning
Asynchronous methods are widely used in deep learning, but have limited theoretical justification when applied to non-convex problems. Expand
An object-based convolutional neural network (OCNN) for urban land use classification
A novel object-based convolutional neural network is proposed for urban land use classification using VFSR images. Expand
Neuroinvasion of SARS-CoV-2 in human and mouse brain
Although COVID-19 is considered to be primarily a respiratory disease, SARS-CoV-2 affects multiple organ systems including the central nervous system (CNS). Yet, there is no consensus whether theExpand
The use of categorization information in language models for question retrieval
This paper proposes a category-based framework for search in CQA archives that uses language models to exploit categories of questions for improving question-answer search. Expand