TY - GEN
T1 - Eunomia
T2 - 24th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2018
AU - Zhou, Wei
AU - White, K. Preston
AU - Yu, Hongfeng
PY - 2019/2/19
Y1 - 2019/2/19
N2 - Due to hardware upgrades and server consolidation, it is not uncommon to witness a few generations of servers deployed in the same datacenters. As a result, variants of fair job schedulers are proposed to enforce fairness for constrained jobs that have hardware or software constraints on task placement. However, the other important characteristics resulted from server heterogeneity, performance variation, is unfortunately overlooked by state-of-art fair job schedulers with placement constraints. In this paper, we propose Eunomia, a performance-variation-aware fair job scheduler, to address the unfairness issue due to performance variation in heterogenous clusters. Eunomia introduces a key metric, called progress share, which is defined as the ratio between the accumulated task progress given the current allocation and the accumulated task progress if the user can monopolize the cluster. Eunomia aims to equalize progress share of jobs as much as possible, so as to achieve the same slowdown of jobs from different users due to resource sharing and placement constraints, regardless of performance variation. Evaluation results show that Eunomia is able to deliver better share fairness compared with state-of-art schedulers without performance loss.
AB - Due to hardware upgrades and server consolidation, it is not uncommon to witness a few generations of servers deployed in the same datacenters. As a result, variants of fair job schedulers are proposed to enforce fairness for constrained jobs that have hardware or software constraints on task placement. However, the other important characteristics resulted from server heterogeneity, performance variation, is unfortunately overlooked by state-of-art fair job schedulers with placement constraints. In this paper, we propose Eunomia, a performance-variation-aware fair job scheduler, to address the unfairness issue due to performance variation in heterogenous clusters. Eunomia introduces a key metric, called progress share, which is defined as the ratio between the accumulated task progress given the current allocation and the accumulated task progress if the user can monopolize the cluster. Eunomia aims to equalize progress share of jobs as much as possible, so as to achieve the same slowdown of jobs from different users due to resource sharing and placement constraints, regardless of performance variation. Evaluation results show that Eunomia is able to deliver better share fairness compared with state-of-art schedulers without performance loss.
KW - Big Data
KW - Fair Scheduler
KW - Placement Constraint
UR - http://www.scopus.com/inward/record.url?scp=85063344081&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85063344081&partnerID=8YFLogxK
U2 - 10.1109/PADSW.2018.8645046
DO - 10.1109/PADSW.2018.8645046
M3 - Conference contribution
AN - SCOPUS:85063344081
T3 - Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS
SP - 1034
EP - 1039
BT - Proceedings - 2018 IEEE 24th International Conference on Parallel and Distributed Systems, ICPADS 2018
PB - IEEE Computer Society
Y2 - 11 December 2018 through 13 December 2018
ER -