2008 SCOPEEasyandEfficientParallelPr

• (Chaiken et al., 2008) ⇒ Ronnie Chaiken, Bob Jenkins, Per-Åke Larson, Bill Ramsey, Darren Shakib, Simon Weaver, and Jingren Zhou. (2008). “SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets.” In: Proceedings of the VLDB Endowment Journal, 1(2). doi:10.14778/1454159.1454166

Subject Headings: SCOPE.

Cited By

• http://scholar.google.com/scholar?q=%222008%22+SCOPE%3A+Easy+and+Efficient+Parallel+Processing+of+Massive+Data+Sets
• http://dl.acm.org/citation.cfm?id=1454159.1454166&preflayout=flat#citedby

Quotes

Abstract

Companies providing cloud-scale services have an increasing need to store and analyze massive data sets such as search logs and click streams. For cost and performance reasons, processing is typically done on large clusters of shared-nothing commodity machines. It is imperative to develop a programming model that hides the complexity of the underlying system but provides flexibility by allowing users to extend functionality to meet a variety of requirements.

In this paper, we present a new declarative and extensible scripting language, SCOPE (Structured Computations Optimized for Parallel Execution), targeted for this type of massive data analysis. The language is designed for ease of use with no explicit parallelism, while being amenable to efficient parallel execution on large clusters. SCOPE borrows several features from SQL. Data is modeled as sets of rows composed of typed columns. The select statement is retained with inner joins, outer joins, and aggregation allowed. Users can easily define their own functions and implement their own versions of operators: extractors (parsing and constructing rows from a file), processors (row-wise processing), reducers (group-wise processing), and combiners (combining rows from two inputs). SCOPE supports nesting of expressions but also allows a computation to be specified as a series of steps, in a manner often preferred by programmers. We also describe how scripts are compiled into efficient, parallel execution plans and executed on large clusters.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2008 SCOPEEasyandEfficientParallelPr	Ronnie Chaiken Bob Jenkins Per-Åke Larson Bill Ramsey Darren Shakib Simon Weaver Jingren Zhou			SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets				10.14778/1454159.1454166		2008