Eunomia: A Performance-Variation-Aware Fair Job Scheduler with Placement Constraints for Heterogeneous Datacenters

Wei Zhou, K. Preston White, Hongfeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Due to hardware upgrades and server consolidation, it is not uncommon to witness a few generations of servers deployed in the same datacenters. As a result, variants of fair job schedulers are proposed to enforce fairness for constrained jobs that have hardware or software constraints on task placement. However, the other important characteristics resulted from server heterogeneity, performance variation, is unfortunately overlooked by state-of-art fair job schedulers with placement constraints. In this paper, we propose Eunomia, a performance-variation-aware fair job scheduler, to address the unfairness issue due to performance variation in heterogenous clusters. Eunomia introduces a key metric, called progress share, which is defined as the ratio between the accumulated task progress given the current allocation and the accumulated task progress if the user can monopolize the cluster. Eunomia aims to equalize progress share of jobs as much as possible, so as to achieve the same slowdown of jobs from different users due to resource sharing and placement constraints, regardless of performance variation. Evaluation results show that Eunomia is able to deliver better share fairness compared with state-of-art schedulers without performance loss.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE 24th International Conference on Parallel and Distributed Systems, ICPADS 2018
PublisherIEEE Computer Society
Pages1034-1039
Number of pages6
ISBN (Electronic)9781538673089
DOIs
StatePublished - Feb 19 2019
Event24th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2018 - Singapore, Singapore
Duration: Dec 11 2018Dec 13 2018

Publication series

NameProceedings of the International Conference on Parallel and Distributed Systems - ICPADS
Volume2018-December
ISSN (Print)1521-9097

Conference

Conference24th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2018
CountrySingapore
CitySingapore
Period12/11/1812/13/18

Keywords

  • Big Data
  • Fair Scheduler
  • Placement Constraint

ASJC Scopus subject areas

  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Eunomia: A Performance-Variation-Aware Fair Job Scheduler with Placement Constraints for Heterogeneous Datacenters'. Together they form a unique fingerprint.

  • Cite this

    Zhou, W., White, K. P., & Yu, H. (2019). Eunomia: A Performance-Variation-Aware Fair Job Scheduler with Placement Constraints for Heterogeneous Datacenters. In Proceedings - 2018 IEEE 24th International Conference on Parallel and Distributed Systems, ICPADS 2018 (pp. 1034-1039). [8645046] (Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS; Vol. 2018-December). IEEE Computer Society. https://doi.org/10.1109/PADSW.2018.8645046