TY - GEN
T1 - A dynamic load balancing scheme for I/O-intensive applications in distributed systems
AU - Qin, Xiao
AU - Jiang, Hong
AU - Zhu, Yifeng
AU - Swanson, D. R.
N1 - Publisher Copyright:
© 2003 IEEE.
PY - 2003
Y1 - 2003
N2 - In this paper, a new I/O-aware load-balancing scheme is presented to improve overall performance of a distributed system with a general and practical workload including I/O activities. The proposed scheme dynamically detects I/O load imbalance on nodes of a distributed system and determines whether to migrate the I/O requests of some jobs from overloaded nodes to other less- or under-loaded nodes, depending on data migration cost and remote I/O access overhead. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in distributed systems, thereby maintaining the same level of performance as the existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation study show that, compared with the existing schemes that only consider CPU and memory, the proposed scheme reduces the mean slowdown by up to 54.5% (with an average of 39.9%). On the other hand, when compared to the existing approaches that only consider I/O, the proposed scheme reduces the mean slowdown by up to 57.2% (with an average of 31.6%). More importantly, the new scheme improves over a very recent algorithm found in the literature that considers all the three resources by up to 49.6% (with an average of up to 41.9%).
AB - In this paper, a new I/O-aware load-balancing scheme is presented to improve overall performance of a distributed system with a general and practical workload including I/O activities. The proposed scheme dynamically detects I/O load imbalance on nodes of a distributed system and determines whether to migrate the I/O requests of some jobs from overloaded nodes to other less- or under-loaded nodes, depending on data migration cost and remote I/O access overhead. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in distributed systems, thereby maintaining the same level of performance as the existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation study show that, compared with the existing schemes that only consider CPU and memory, the proposed scheme reduces the mean slowdown by up to 54.5% (with an average of 39.9%). On the other hand, when compared to the existing approaches that only consider I/O, the proposed scheme reduces the mean slowdown by up to 57.2% (with an average of 31.6%). More importantly, the new scheme improves over a very recent algorithm found in the literature that considers all the three resources by up to 49.6% (with an average of up to 41.9%).
KW - Application software
KW - Clustering algorithms
KW - Computational modeling
KW - Computer architecture
KW - Computer science
KW - Costs
KW - Load management
KW - Remote sensing
KW - System performance
KW - Workstations
UR - http://www.scopus.com/inward/record.url?scp=84943256060&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84943256060&partnerID=8YFLogxK
U2 - 10.1109/ICPPW.2003.1240357
DO - 10.1109/ICPPW.2003.1240357
M3 - Conference contribution
AN - SCOPUS:84943256060
T3 - Proceedings of the International Conference on Parallel Processing Workshops
SP - 79
EP - 86
BT - Proceedings - 2003 International Conference on Parallel Processing Workshops, ICPPW 2003
A2 - Huang, Chua-Huang
A2 - Ramanujam, J.
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2003 International Conference on Parallel Processing Workshops, ICPPW 2003
Y2 - 6 October 2003 through 9 October 2003
ER -