Hierarchical scheduling for diverse datacenter workloads
Article 2013 en
Authors
AB
Arka Bhattacharya
DC
David Culler
EF
Eric Friedman
Abstract
1 min read
There has been a recent industrial effort to develop multi-resource hierarchical schedulers. However, the existing implementations have some shortcomings in that they might leave resources unallocated or starve certain jobs. This is because the multi-resource setting introduces new challenges for hierarchical scheduling policies. We provide an algorithm, which we implement in Hadoop, that generalizes the most commonly used multi-resource scheduler, DRF [1], to support hierarchies. Our evaluation shows that our proposed algorithm, H-DRF, avoids the starvation and resource inefficiencies of the existing open-source schedulers and outperforms slot scheduling.
Discussion(0)
No comments yet. Be the first to comment.