2011 SystemMLDeclarativeMachineLearn

Subject Headings: SystemML, Declarative ML.

Notes

MapReduce is emerging as a generic parallel programming paradigm for large clusters of machines. This trend combined with the growing need to run machine learning (ML) algorithms on massive datasets has led to an increased interest in implementing ML algorithms on MapReduce. However, the cost of implementing a large class of ML algorithms as low-level MapReduce jobs on varying data and machine cluster sizes can be prohibitive. In this paper, we propose SystemML in which ML algorithms are expressed in a higher-level language and are compiled and executed in a MapReduce environment. This higher-level language exposes several constructs including linear algebra primitives that constitute key building blocks for a broad class of supervised and unsupervised ML algorithms. The algorithms expressed in SystemML are compiled and optimized into a set of MapReduce jobs that can run on a cluster of machines. We describe and empirically evaluate a number of optimization strategies for efficiently executing these algorithms on Hadoop, an open-source MapReduce implementation. We report an extensive performance evaluation on three ML algorithms on varying data and cluster sizes.

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2011 SystemMLDeclarativeMachineLearn	Shivakumar Vaithyanathan Berthold Reinwald Amol Ghoting Edwin Pednault Rajasekar Krishnamurthy Vikas Sindhwani Shirish Tatikonda Yuanyuan Tian			SystemML: Declarative Machine Learning on MapReduce				10.1109/ICDE.2011.5767930		2011