Precision Metric
A Precision Metric, [math]\displaystyle{ \operatorname{Prec}(M_i,T_j) }[/math] is a performance metric for some binary classification model on some given task that estimated the probability or any positive prediction being a correct prediction.
- AKA: Positive Predictive Value, Binary Classifier Precision, PPV.
- Context:
- It can be estimated by: TP/(TP+FP)
- It can be negatively correlated with a recall metric (at the expense of).
- It can be a Two-Class Precision Metric to being a Multi-Class Precision Metric (such as a micro-precision metric).
- ...
- Example:
- The precision for a Linear-CRF on the CoNLL 2003 Spanish NER Task is ...
- Set Intersection(Relevant Artifacts, Retrieved Artifacts) / Retrieved Artifacts.
- …
- Counter-Example(s):
- a Prediction Scoring Function, [math]\displaystyle{ \operatorname{PredScore}(M_i,T_j,\bf{o}_k) }[/math]
- a Recall Metric (True Positive Rate).
- an F-Measure.
- an Accuracy Metric (probability that a prediction is a correct prediction)
- a Negative Predictive Value.
- See: Fallout, Numeric Attribute Precision, Reproducibility Metric.
References
2015a
- (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/positive_and_negative_predictive_values Retrieved:2015-5-17.
- The positive and negative predictive values (PPV and NPV respectively) are the proportions of positive and negative results in statistics and diagnostic tests that are true positive and true negative results. The PPV and NPV describe the performance of a diagnostic test or other statistical measure. A high result can be interpreted as indicating the accuracy of such a statistic. The PPV and NPV are not intrinsic to the test; they depend also on the prevalence. The PPV can be derived using Bayes' theorem.
Although sometimes used synonymously, a positive predictive value generally refers to what is established by control groups, while a post-test probability refers to a probability for an individual. Still, if the individual's pre-test probability of the target condition is the same as the prevalence in the control group used to establish the positive predictive value, the two are numerically equal.
In information retrieval, the PPV statistic is often called the precision.
- The positive and negative predictive values (PPV and NPV respectively) are the proportions of positive and negative results in statistics and diagnostic tests that are true positive and true negative results. The PPV and NPV describe the performance of a diagnostic test or other statistical measure. A high result can be interpreted as indicating the accuracy of such a statistic. The PPV and NPV are not intrinsic to the test; they depend also on the prevalence. The PPV can be derived using Bayes' theorem.
2015b
- (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/precision_and_recall Retrieved:2015-1-20.
- In pattern recognition and information retrieval with binary classification, precision (also called positive predictive value) is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved. Both precision and recall are therefore based on an understanding and measure of relevance. Suppose a program for recognizing dogs in scenes from a video identifies 7 dogs in a scene containing 9 dogs and some cats. If 4 of the identifications are correct, but 3 are actually cats, the program's precision is 4/7 while its recall is 4/9. When a search engine returns 30 pages only 20 of which were relevant while failing to return 40 additional relevant pages, its precision is 20/30 = 2/3 while its recall is 20/60 = 1/3.
In statistics, if the null hypothesis is that all and only the relevant items are retrieved, absence of type I and type II errors corresponds respectively to maximum precision (no false positive) and maximum recall (no false negative). The above pattern recognition example contained 7 − 4 = 3 type I errors and 9 − 4 = 5 type II errors. Precision can be seen as a measure of exactness or quality, whereas recall is a measure of completeness or quantity.
In simple terms, high precision means that an algorithm returned substantially more relevant results than irrelevant, while high recall means that an algorithm returned most of the relevant results.
- In pattern recognition and information retrieval with binary classification, precision (also called positive predictive value) is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved. Both precision and recall are therefore based on an understanding and measure of relevance. Suppose a program for recognizing dogs in scenes from a video identifies 7 dogs in a scene containing 9 dogs and some cats. If 4 of the identifications are correct, but 3 are actually cats, the program's precision is 4/7 while its recall is 4/9. When a search engine returns 30 pages only 20 of which were relevant while failing to return 40 additional relevant pages, its precision is 20/30 = 2/3 while its recall is 20/60 = 1/3.
2011
- (Kai Ming Ting, 2011c) ⇒ Kai Ming Ting. (2011). “Precision.” In: (Sammut & Webb, 2011) p.780
2010
- (Wiktionary) ⇒ http://en.wiktionary.org/wiki/precision#Noun
- 1. the state of being precise or exact; exactness
2000
- 2000_SpeechAndLanguageProcessing.
- QUOTE: Precision is a measure of how much of the information that the system returned is actually correct.
1998
- (Kohavi & Provost, 1998) ⇒ Ron Kohavi, and Foster Provost. (1998). “Glossary of Terms.” In: Machine Leanring 30(2-3).
- Precision: d/(b+d).