Prediction Task Performance Measure
(Redirected from predictive performance measure)
Jump to navigation
Jump to search
An Prediction Task Performance Measure is a performance measure for prediction tasks (to quantify erroneous prediction rates).
- AKA: Error Rate Measure.
- Context:
- It can (often) be calculated by a Predictive Performance Measuring System.
- It can range from being a Classification Performance Measure to being a Ranking Performance Measure to being a Numeric Prediction Performance Measure.
- Example(s):
- Categorical Prediction Task Performance Measure, such as a binary prediction measure (which can range from being a Type I Error Rate to being a Type II Error Rate).
- Ordinal Prediction Task Performance Measure, such as a ranking task performance measure (e.g. NDCG)
- Numeric Prediction Task Performance Measure.
- …
- Counter-Example(s):
- See: Data Transmission, Accuracy; Confusion matrix, Bit Error Rate, Predictive Performance, Predictive Feature, Unpredictive Feature.
References
2017
- https://spark.apache.org/docs/latest/mllib-evaluation-metrics.html
- QUOTE: spark.mllib comes with a number of machine learning algorithms that can be used to learn from and make predictions on data. When these algorithms are applied to build machine learning models, there is a need to evaluate the performance of the model on some criteria, which depends on the application and its requirements. spark.mllib also provides a suite of metrics for the purpose of evaluating the performance of machine learning models.
Specific machine learning algorithms fall under broader types of machine learning applications like classification, regression, clustering, etc. Each of these types have well established metrics for performance evaluation and those metrics that are currently available in spark.mllib are detailed in this section.
- QUOTE: spark.mllib comes with a number of machine learning algorithms that can be used to learn from and make predictions on data. When these algorithms are applied to build machine learning models, there is a need to evaluate the performance of the model on some criteria, which depends on the application and its requirements. spark.mllib also provides a suite of metrics for the purpose of evaluating the performance of machine learning models.
2011
- (Kai Ming Ting, 2011b) ⇒ Kai Ming Ting. (2011). “Error Rate.” In: (Sammut & Webb, 2011) p.331
1998
- (Kohavi & Provost, 1998) ⇒ Ron Kohavi, and Foster Provost. (1998). “Glossary of Terms.” In: Machine Leanring 30(2-3).
- Error rate: See Accuracy.
1983
- (Efron, 1983) ⇒ Bradley Efron, (1983). “Estimating the error rate of a prediction rule: improvement on cross-validation.” In: Journal of the American Statistical Association, 78(382). http://www.jstor.org/stable/2288636
- QUOTE: We construct a prediction rule on the basis of some data, and then wish to estimate the error rate of this rule in classifying future observations. Cross-validation provides a nearly unbiased estimate, using only the original data. Cross-validation turns out to be related closely to the bootstrap estimate of the error rate. This article has two purposes: to understand better the theoretical basis of the prediction problem, and to investigate some related estimators, which seem to offer considerably improved estimation in small samples.