Classification Accuracy Metric

An classification accuracy metric is a classifier performance metric to assess the accuracy metric of a class prediction system (based on the proportion of the classifier's correct classifications to incorrect classifications(on labeled testing records).

• Context:
  • It can be calculated by:
    • (TP+TN)/(TP+TN+FP+FN), for a Two-Class Problem.
    • counting the correct classifications and dividing by the number of classifications made.
  • It can (typically) be the inverse of a Classification Error Measure.
  • It can be estimated by an Accuracy Estimation Process.
  • It can be reported as the rate which a case will be labeled with the right category, if the Predictive Model is a Classifier.
  • It can be reported as the average distance between the predicted label and the correct value, if the Predictive Model is an Estimator.
  • It can (typically) be required that the Test Case be unseen during the Training Phase.
  • It can be the Inverse Function to the Error Rate Function.
  • …
• Example(s):
  • A Classification System may be said to have 85.5% accuracy (to predict whether a customer responds to a promotional campaign).
  • …
• Counter-Example(s):
  • a Point Estimator Measure.
  • a True Positive Rate, or a True Negative Rate, or a False Positive Rate.
  • a Cross-Entropy Metric.
• See: Confusion Matrix; Resubstitution Accuracy; Precision; Recall; F-Measure; Error Rate; Statistical Significance; Cross-validation; Classification Task; Task Performance, Cross-Validation, Bootstrap.

---

---

References

2018

• (ML Glossary, 2018) ⇒ (2008). Accuracy. In: Machine Learning Glossary https://developers.google.com/machine-learning/glossary/ Retrieved 2018-04-22.
  • QUOTE: The fraction of predictions that a classification model got right. In multi-class classification, accuracy is defined as follows:

    [math]\displaystyle{ \text{Accuracy} =\frac{\text{Correct Predictions}} {\text{Total Number Of Examples}} }[/math]

    In binary classification, accuracy has the following definition:

    [math]\displaystyle{ \text{Accuracy} = \frac{\text{True Positives} + \text{True Negatives}}{\text{Total Number Of Examples}} }[/math]

    See true positive and true negative.

2017

• (Sammut & Webb, 2017) ⇒ (2017) Accuracy. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA
  • QUOTE: Accuracy refers to a measure of the degree to which the predictions of a model matches the reality being modeled. The term accuracy is often applied in the context of classification models. In this context, [math]\displaystyle{ accuracy = P(\lambda(X) = Y ) }[/math], where [math]\displaystyle{ XY }[/math] is a joint distribution and the classification model [math]\displaystyle{ \lambda }[/math] is a function [math]\displaystyle{ X \rightarrow Y }[/math]. Sometimes, this quantity is expressed as a percentage rather than a value between 0.0 and 1.0.

    The accuracy of a model is often assessed or estimated by applying it to test data for which the labels ([math]\displaystyle{ Y }[/math] values) are known. The accuracy of a classifier on test data may be calculated as number of correctly classified objects/total number of objects. Alternatively, a smoothing function may be applied, such as a Laplace estimate or an m-estimate.

     Accuracy is directly related to error rate, such that [math]\displaystyle{ accuracy = 1. 0 – error\; rate }[/math] (or when expressed as a percentage, [math]\displaystyle{ accuracy = 100 – error\; rate }[/math]).

2002

• (Melli, 2002) ⇒ Gabor Melli. (2002). “PredictionWorks' Data Mining Glossary.
  • Accuracy: The measure of a model's ability to correctly label a previously unseen test case. If the label is categorical (classification), accuracy is commonly reported as the rate which a case will be labeled with the right category. For example, a model may be said to predict whether a customer responds to a promotional campaign with 85.5% accuracy. If the label is continuous, accuracy is commonly reported as the average distance between the predicted label and the correct value. For example, a model may be said to predict the amount a customer will spend on a given month within $55. See also Accuracy Estimation, Classification, Estimation, Model, and Statistical Significance.

1998

• (Kohavi & Provost, 1998) ⇒ Ron Kohavi, and Foster Provost. (1998). “Glossary of Terms.” In: Machine Leanring 30(2-3).
  • Accuracy (error rate): The rate of correct (incorrect) predictions made by the model over a data set (cf. coverage). Accuracy is usually estimated by using an independent test set that was not used at any time during the learning process. More complex accuracy estimation techniques, such as cross-validation and the bootstrap, are commonly used, especially with data sets containing a small number of instances.

---