Brief announcement: deadline-aware scheduling of big-data processing jobs

@inproceedings{Bodk2014BriefAD,
  title={Brief announcement: deadline-aware scheduling of big-data processing jobs},
  author={Peter Bod{\'i}k and Ishai Menache and Joseph Naor and Jonathan Yaniv},
  booktitle={SPAA},
  year={2014}
}
This paper presents a novel algorithm for scheduling big data jobs on large compute clusters. In our model, each job is represented by a DAG consisting of several stages linked by precedence constraints. The resource allocation per stage is malleable, in the sense that the processing time of a stage depends on the resources allocated to it (the dependency can be arbitrary in general).The goal of the scheduler is to maximize the total value of completed jobs, where the value for each job depends… CONTINUE READING

Similar Papers

Loading similar papers…