2014 ASystemtoGradeComputerProgrammi

• (Srikant & Aggarwal, 2014) ⇒ Shashank Srikant, and Varun Aggarwal. (2014). “A System to Grade Computer Programming Skills Using Machine Learning.” In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2014) Journal. ISBN:978-1-4503-2956-9 doi:10.1145/2623330.2623377

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222014%22+A+System+to+Grade+Computer+Programming+Skills+Using+Machine+Learning
• http://dl.acm.org/citation.cfm?id=2623330.2623377&preflayout=flat#citedby

Quotes

Author Keywords

• Automatic grading; feature engineering; general; learning; mooc; one-class learning; recruitment; supervised learning

Abstract

The automatic evaluation of computer programs is a nascent area of research with a potential for large-scale impact. Extant program assessment systems score mostly based on the number of test-cases passed, providing no insight into the competency of the programmer. In this paper, we present a system to grade computer programs automatically. In addition to grading a program on its programming practices and complexity, the key kernel of the system is a machine-learning based algorithm which determines closeness of the logic of the given program to a correct program. This algorithm uses a set of highly-informative features, derived from the abstract representations of a given program, that capture the program's functionality. These features are then used to learn a model to grade the programs, which are built against evaluations done by experts. We show that the regression models provide much better grading than the ubiquitous test-case-pass based grading and rivals the grading accuracy of other open-response problems such as essay grading. We also show that our novel features and significant value over and above basic keyword / expression count features. In addition to this, we propose a novel way of posing computer-program grading as a one-class modeling problem and report encouraging preliminary results. We show the value of the system through a case study in a real-world industrial deployment. To the best of the authors' knowledge, this is the first time a system using machine learning has been developed and used for grading programs. The work is timely with regard to the recent boom in Massively Online Open Courseware (MOOCs), which promises to produce a significant amount of hand-graded digitized data.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2014 ASystemtoGradeComputerProgrammi	Shashank Srikant Varun Aggarwal			A System to Grade Computer Programming Skills Using Machine Learning				10.1145/2623330.2623377		2014