FlowFlex: Malleable Scheduling for Flows of MapReduce Jobs

  title={FlowFlex: Malleable Scheduling for Flows of MapReduce Jobs},
  author={Viswanath Nagarajan and Joel L. Wolf and Andrey Balmin and Kirsten Hildrum},
We introduce FlowFlex, a highly generic and effective scheduler for flows of MapReduce jobs connected by precedence constraints. Such a flow can result, for example, from a single user-level Pig, Hive or Jaql query. Each flow is associated with an arbitrary function describing the cost incurred in completing the flow at a particular time. The overall objective is to minimize either the total cost (minisum) or the maximum cost (minimax) of the flows. Our contributions are both theoretical and… CONTINUE READING
Highly Cited
This paper has 18 citations. REVIEW CITATIONS
14 Citations
27 References
Similar Papers


Publications referenced by this paper.
Showing 1-10 of 27 references

Scheduling with Deadlines and Loss Functions

  • R. McNaughton
  • Management Science 6(1), 1–12
  • 1959
Highly Influential
4 Excerpts

Berlinska and M . Drozdowski , Scheduling Divisible MapReduce Computations

  • K. Hildrum Balmin, V. Nagarajan, J. Wolf
  • Malleable Scheduling for Flows of MapReduce Jobs
  • 2013

Malleable Scheduling for Flows of MapReduce Jobs, Research Report RC25364

  • A. Balmin, K. Hildrum, V. Nagarajan, J. Wolf
  • IBM Research
  • 2013
1 Excerpt

Similar Papers

Loading similar papers…