Accuracy Metric

An Accuracy Metric is a correctness quantitative performance metric that measures the degree of closeness between observed values and true values.

AKA: Accuracy Measure, Accuracy Performance Metric.
Context:
- It can typically quantify System Performance through accuracy measurement tasks.
- It can typically evaluate Predictive Models through accuracy assessment processes.
- It can typically support Model Selection Tasks through accuracy comparisons.
- It can typically inform System Improvement Decisions through accuracy analysises.
- It can typically complement Other Performance Metrics in multi-metric evaluation frameworks.
- ...
- It can often serve as Primary Evaluation Criterion in classification tasks.
- It can often guide Model Training Processes through accuracy optimizations.
- It can often detect Model Degradation through accuracy monitorings.
- It can often validate System Requirements through accuracy thresholds.
- ...
- It can range from being a Simple Accuracy Metric to being a Complex Accuracy Metric, depending on its accuracy calculation complexity.
- It can range from being a Domain-Agnostic Accuracy Metric to being a Domain-Specific Accuracy Metric, depending on its accuracy application context.
- It can range from being a Point-Estimate Accuracy Metric to being a Distributional Accuracy Metric, depending on its accuracy statistical representation.
- ...
- It can integrate with Evaluation Frameworks for comprehensive assessments.
- It can combine with Cost Functions for weighted evaluations.
- It can feed into Model Selection Algorithms for automated optimizations.
- It can support Performance Dashboards for real-time monitorings.
- ...
Example(s):
- Classification Accuracy Metrics, such as:
  - Binary Classification Accuracy Metrics, such as:
    - Balanced Accuracy Metric for imbalanced dataset evaluation.
    - Top-K Accuracy Metric for multi-class prediction tasks.
  - Multi-Class Classification Accuracy Metrics, such as:
    - Macro-Averaged Accuracy Metric for class-balanced evaluation.
    - Micro-Averaged Accuracy Metric for instance-weighted evaluation.
- Regression Accuracy Metrics, such as:
  - Mean Absolute Error (MAE) for continuous prediction tasks.
  - Root Mean Square Error (RMSE) for error magnitude assessment.
- Domain-Specific Accuracy Metrics, such as:
  - Medical Diagnosis Accuracy Metrics, such as:
    - Diagnostic Accuracy Metric for clinical decision support.
    - Screening Test Accuracy Metric for disease detection tasks.
  - Natural Language Processing Accuracy Metrics, such as:
    - Word-Level Accuracy Metric for tokenization tasks.
    - Entity Recognition Accuracy Metric for NER tasks.
- Temporal Accuracy Metrics, such as:
  - Time-Series Forecast Accuracy Metric for predictive forecasting.
  - Event Detection Accuracy Metric for temporal pattern recognition.
- ...
Counter-Example(s):
- Precision Metric, which measures correct positive predictions among all positive predictions rather than overall prediction correctness.
- Recall Metric, which measures correct positive predictions among all actual positives rather than overall prediction correctness.
- F1 Score, which balances precision and recall rather than measuring overall prediction correctness.
- Serendipity Metric, which measures unexpected valuable discoverys rather than prediction correctness.
- Efficiency Metric, which measures computational performance rather than prediction correctness.
See: Performance Metric, Evaluation Metric, Model Assessment Task, Confusion Matrix, Error Analysis, Statistical Bias, Measurement Theory.

References

2022

(Wikipedia, 2022) ⇒ https://en.wikipedia.org/wiki/Accuracy_and_precision Retrieved:2022-1-22.
- In a set of measurements, accuracy is closeness of the measurements to a specific value, while precision is the closeness of the measurements to each other.
  Accuracy has two definitions:
  1. More commonly, it is a description of systematic errors, a measure of statistical bias; low accuracy causes a difference between a result and a "true" value. ISO calls this trueness.
  2. Alternatively, ISO defines accuracy as describing a combination of both types of observational error above (random and systematic), so high accuracy requires both high precision and high trueness.
- Precision is a description of random errors, a measure of statistical variability.
  In simpler terms, given a set of data points from repeated measurements of the same quantity, the set can be said to be accurate if their average is close to the true value of the quantity being measured, while the set can be said to be precise if the values are close to each other. In the first, more common definition of "accuracy" above, the two concepts are independent of each other, so a particular set of data can be said to be either accurate, or precise, or both, or neither.

2013

(Hulley et al., 2013) ⇒ Stephen B. Hulley, Steven R. Cummings, Warren S. Browner, Deborah G. Grady, and Thomas B. Newman. (2013). “Designing Clinical Research: Fourth Edition.” Wolters Kluwer Health. ISBN: 9781469840543
- QUOTE: Accuracy: The degree to which a measurement corresponds to its true value. For example, self-reported body weight is a less accurate measurement of actual bodyweight than one made with a calibrated electronic scale.

2010

(Ge et al., 2010) ⇒ Mouzhi Ge, Carla Delgado-Battenfeld, and Dietmar Jannach. (2010). “Beyond Accuracy: Evaluating Recommender Systems by Coverage and Serendipity.” In: Proceedings of the fourth ACM conference on Recommender systems (RecSys-2010).
- QUOTE: ... Over the last decade, different recommender systems were developed and used in a variety of domains [1]. The primary goal of recommenders is to provide personalized recommendations so as to improve users’ satisfaction. As more and more recommendation techniques are proposed, researchers and practitioners are facing the problem of how to estimate the value of the recommendations. In previous evaluations, most approaches focused only on the accuracy of the generated predictions based, e.g., on the Mean Absolute Error. However, a few recent works argue that accuracy is not the only metric for evaluating recommender systems and that there are other important aspects we need to focus on in future evaluations [4, 8]. The point that the recommender community should move beyond accuracy metrics to evaluate recommenders was for example made in [8]. There, informal arguments were presented supporting that accurate recommendations may sometimes not be the most useful ones to the users, and that evaluation metrics should (1) take into account other factors which impact recommendation quality such as serendipity and (2) be applied to recommendation lists and not on individual items.