Confidence Scoring Task
A Confidence Scoring Task is a point estimation task to produce a confidence estimate for a prediction.
- Context:
- Input: a Prediction.
- output: a Prediction Confidence Estimate.
- It can be solved by a Confidence Scoring System (that implements a Confidence Scoring algorithm).
- Example(s):
- Given predictive model [math]\displaystyle{ P(T|O) }[/math], what is the confidence score of predictions for observations [math]\displaystyle{ o_1 }[/math] and [math]\displaystyle{ o_2 }[/math]?
- Counter-Example(s):
- Given predictive model [math]\displaystyle{ P(T|O) }[/math], what is the confidence that prediction [math]\displaystyle{ p }[/math] has greater than 92% chance of being true?
- a Hypothesis Testing Task.
- See: Confidence Interval, Confidence Interval.
References
2004
- (Culotta & McCallum, 2004) ⇒ Aron Culotta, and Andrew McCallum. (2004). “Confidence Estimation for Information Extraction.” In: Proceedings of HLT-NAACL (NAACL 2004).