• DOI:10.1109/HPCA47549.2020.00027
Corpus ID: 202542460

PREMA: A Predictive Multi-Task Scheduling Algorithm For Preemptible Neural Processing Units

@article{Choi2020PREMAAP,
  title={PREMA: A Predictive Multi-Task Scheduling Algorithm For Preemptible Neural Processing Units},
  author={Yujeong Choi and Minsoo Rhu},
  journal={2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)},
  year={2020},
  pages={220-233}
}

• Yujeong Choi, Minsoo Rhu
• Published 2020
• Computer Science
• 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)

To amortize cost, cloud vendors providing DNN acceleration as a service to end-users employ consolidation and virtualization to share the underlying resources among multiple DNN service requests. This paper makes a case for a "preemptible" neural processing unit (NPU) and a "predictive" multi-task scheduler to meet the latency demands of high-priority inference while maintaining high throughput. We evaluate both the mechanisms that enable NPUs to be preemptible and the policies that utilize… Expand

View on IEEE

arxiv.org

Save to Library

Create Alert

Cite

Launch Research Feed