Effectively Unified optimization for Large-scale Graph Community Detection

Jianping Zeng; Hongfeng Yu

doi:10.1109/BigData47090.2019.9005481

Effectively Unified optimization for Large-scale Graph Community Detection

Jianping Zeng, Hongfeng Yu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Scopus citations

Abstract

In this paper, we present a unified graph clustering framework based on an asynchronous approach. We study the similarities among the Louvain algorithm and the Infomap algorithm. Based on their common features, we build an end-to-end optimized distributed framework for implementing both algorithms. By extending the existing asynchronous distributed framework for large-scale graphs traversal, we ensure both workload and communication balanced. Our extensive experiments show that our framework is correct and effective with different large real-world and synthetic datasets using up to 32,768 processors for the Louvain algorithm and 16,384 processors for the Infomap algorithm. The quality and the scalability of our framework are superior to the existing work.

Original language	English (US)
Title of host publication	Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019
Editors	Chaitanya Baru, Jun Huan, Latifur Khan, Xiaohua Tony Hu, Ronay Ak, Yuanyuan Tian, Roger Barga, Carlo Zaniolo, Kisung Lee, Yanfang Fanny Ye
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	475-482
Number of pages	8
ISBN (Electronic)	9781728108582
DOIs	https://doi.org/10.1109/BigData47090.2019.9005481
State	Published - Dec 2019
Externally published	Yes
Event	2019 IEEE International Conference on Big Data, Big Data 2019 - Los Angeles, United States Duration: Dec 9 2019 → Dec 12 2019

Publication series

Name	Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019

Conference

Conference	2019 IEEE International Conference on Big Data, Big Data 2019
Country/Territory	United States
City	Los Angeles
Period	12/9/19 → 12/12/19

Keywords

accuracy
community detection
graph clustering
large graph
parallel and distributed processing
scalability

ASJC Scopus subject areas

Artificial Intelligence
Computer Networks and Communications
Information Systems
Information Systems and Management

Access to Document

10.1109/BigData47090.2019.9005481

Cite this

Zeng, J., & Yu, H. (2019). Effectively Unified optimization for Large-scale Graph Community Detection. In C. Baru, J. Huan, L. Khan, X. T. Hu, R. Ak, Y. Tian, R. Barga, C. Zaniolo, K. Lee, & Y. F. Ye (Eds.), Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019 (pp. 475-482). Article 9005481 (Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData47090.2019.9005481

Effectively Unified optimization for Large-scale Graph Community Detection. / Zeng, Jianping; Yu, Hongfeng.
Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019. ed. / Chaitanya Baru; Jun Huan; Latifur Khan; Xiaohua Tony Hu; Ronay Ak; Yuanyuan Tian; Roger Barga; Carlo Zaniolo; Kisung Lee; Yanfang Fanny Ye. Institute of Electrical and Electronics Engineers Inc., 2019. p. 475-482 9005481 (Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Zeng, J & Yu, H 2019, Effectively Unified optimization for Large-scale Graph Community Detection. in C Baru, J Huan, L Khan, XT Hu, R Ak, Y Tian, R Barga, C Zaniolo, K Lee & YF Ye (eds), Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019., 9005481, Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019, Institute of Electrical and Electronics Engineers Inc., pp. 475-482, 2019 IEEE International Conference on Big Data, Big Data 2019, Los Angeles, United States, 12/9/19. https://doi.org/10.1109/BigData47090.2019.9005481

Zeng J, Yu H. Effectively Unified optimization for Large-scale Graph Community Detection. In Baru C, Huan J, Khan L, Hu XT, Ak R, Tian Y, Barga R, Zaniolo C, Lee K, Ye YF, editors, Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 475-482. 9005481. (Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019). doi: 10.1109/BigData47090.2019.9005481

Zeng, Jianping ; Yu, Hongfeng. / Effectively Unified optimization for Large-scale Graph Community Detection. Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019. editor / Chaitanya Baru ; Jun Huan ; Latifur Khan ; Xiaohua Tony Hu ; Ronay Ak ; Yuanyuan Tian ; Roger Barga ; Carlo Zaniolo ; Kisung Lee ; Yanfang Fanny Ye. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 475-482 (Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019).

@inproceedings{cf70ce98f61947928119d7773df32c23,

title = "Effectively Unified optimization for Large-scale Graph Community Detection",

abstract = "In this paper, we present a unified graph clustering framework based on an asynchronous approach. We study the similarities among the Louvain algorithm and the Infomap algorithm. Based on their common features, we build an end-to-end optimized distributed framework for implementing both algorithms. By extending the existing asynchronous distributed framework for large-scale graphs traversal, we ensure both workload and communication balanced. Our extensive experiments show that our framework is correct and effective with different large real-world and synthetic datasets using up to 32,768 processors for the Louvain algorithm and 16,384 processors for the Infomap algorithm. The quality and the scalability of our framework are superior to the existing work.",

keywords = "accuracy, community detection, graph clustering, large graph, parallel and distributed processing, scalability",

author = "Jianping Zeng and Hongfeng Yu",

note = "Funding Information: This research has been sponsored in part by the National Science Foundation through grants IIS-1423487, IIS-1652846, and ICER-1541043, and the Department of Energy through the ExaCT Center for Exascale Simulation of Combustion in Turbulence. Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 IEEE International Conference on Big Data, Big Data 2019 ; Conference date: 09-12-2019 Through 12-12-2019",

year = "2019",

month = dec,

doi = "10.1109/BigData47090.2019.9005481",

language = "English (US)",

series = "Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "475--482",

editor = "Chaitanya Baru and Jun Huan and Latifur Khan and Hu, {Xiaohua Tony} and Ronay Ak and Yuanyuan Tian and Roger Barga and Carlo Zaniolo and Kisung Lee and Ye, {Yanfang Fanny}",

booktitle = "Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019",

}

TY - GEN

T1 - Effectively Unified optimization for Large-scale Graph Community Detection

AU - Zeng, Jianping

AU - Yu, Hongfeng

N1 - Funding Information: This research has been sponsored in part by the National Science Foundation through grants IIS-1423487, IIS-1652846, and ICER-1541043, and the Department of Energy through the ExaCT Center for Exascale Simulation of Combustion in Turbulence. Publisher Copyright: © 2019 IEEE.

PY - 2019/12

Y1 - 2019/12

N2 - In this paper, we present a unified graph clustering framework based on an asynchronous approach. We study the similarities among the Louvain algorithm and the Infomap algorithm. Based on their common features, we build an end-to-end optimized distributed framework for implementing both algorithms. By extending the existing asynchronous distributed framework for large-scale graphs traversal, we ensure both workload and communication balanced. Our extensive experiments show that our framework is correct and effective with different large real-world and synthetic datasets using up to 32,768 processors for the Louvain algorithm and 16,384 processors for the Infomap algorithm. The quality and the scalability of our framework are superior to the existing work.

AB - In this paper, we present a unified graph clustering framework based on an asynchronous approach. We study the similarities among the Louvain algorithm and the Infomap algorithm. Based on their common features, we build an end-to-end optimized distributed framework for implementing both algorithms. By extending the existing asynchronous distributed framework for large-scale graphs traversal, we ensure both workload and communication balanced. Our extensive experiments show that our framework is correct and effective with different large real-world and synthetic datasets using up to 32,768 processors for the Louvain algorithm and 16,384 processors for the Infomap algorithm. The quality and the scalability of our framework are superior to the existing work.

KW - accuracy

KW - community detection

KW - graph clustering

KW - large graph

KW - parallel and distributed processing

KW - scalability

UR - http://www.scopus.com/inward/record.url?scp=85081314454&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85081314454&partnerID=8YFLogxK

U2 - 10.1109/BigData47090.2019.9005481

DO - 10.1109/BigData47090.2019.9005481

M3 - Conference contribution

AN - SCOPUS:85081314454

T3 - Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019

SP - 475

EP - 482

BT - Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019

A2 - Baru, Chaitanya

A2 - Huan, Jun

A2 - Khan, Latifur

A2 - Hu, Xiaohua Tony

A2 - Ak, Ronay

A2 - Tian, Yuanyuan

A2 - Barga, Roger

A2 - Zaniolo, Carlo

A2 - Lee, Kisung

A2 - Ye, Yanfang Fanny

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE International Conference on Big Data, Big Data 2019

Y2 - 9 December 2019 through 12 December 2019

ER -

Effectively Unified optimization for Large-scale Graph Community Detection

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this