2012 StreamingGraphPartitioningforLa

• (Stanton & Kliot, 2012) ⇒ Isabelle Stanton, and Gabriel Kliot. (2012). “Streaming Graph Partitioning for Large Distributed Graphs.” In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2012). ISBN:978-1-4503-1462-6 doi:10.1145/2339530.2339722

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222012%22+Streaming+Graph+Partitioning+for+Large+Distributed+Graphs
• http://dl.acm.org/citation.cfm?id=2339530.2339722&preflayout=flat#citedby

Quotes

Author Keywords

• Complexity measures; distributed graphs; experimental evaluation; graph algorithms; graph partitioning; performance measures; streaming algorithms

Abstract

Extracting knowledge by performing computations on graphs is becoming increasingly challenging as graphs grow in size. A standard approach distributes the graph over a cluster of nodes, but performing computations on a distributed graph is expensive if large amount of data have to be moved. Without partitioning the graph, communication quickly becomes a limiting factor in scaling the system up. Existing graph partitioning heuristics incur high computation and communication cost on large graphs, sometimes as high as the future computation itself. Observing that the graph has to be loaded into the cluster, we ask if the partitioning can be done at the same time with a lightweight streaming algorithm.

We propose natural, simple heuristics and compare their performance to hashing and METIS, a fast, offline heuristic. We show on a large collection of graph datasets that our heuristics are a significant improvement, with the best obtaining an average gain of 76%. The heuristics are scalable in the size of the graphs and the number of partitions. Using our streaming partitioning methods, we are able to speed up PageRank computations on Spark, a distributed computation system, by 18% to 39% for large social networks.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2012 StreamingGraphPartitioningforLa	Isabelle Stanton Gabriel Kliot			Streaming Graph Partitioning for Large Distributed Graphs				10.1145/2339530.2339722		2012