Information Content Measure

See: Information Entropy, Mutual Information, IC.

References

(Resnik, 1995) ⇒ Philip Resnik. (1995). “Using Information Content to Evaluate Semantic Similarity in a Taxonomy.” In: Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI 1995).
- QUOTE: This paper presents a new measure of semantic similarity in an IS-A taxonomy, based on the notion of information content. …
  … Following the standard argumentation of information theory (Ross, 1976), the information content of a concept [math]\displaystyle{ c }[/math] can be quantified as negative the log likelihood, [math]\displaystyle{ −\log p(c) }[/math]. Notice that quantifying information content in this way makes intuitive sense in this setting: as probability increases, informativeness decreases, so the more abstract a concept, the lower its information content. Moreover, if there is a unique top concept, its information content is 0. This quantitative characterization of information provides a new way to measure semantic similarity. The more information two concepts share in common, the more similar they are, and the information shared by two concepts is indicated by the information content of the concepts that subsume them in the taxonomy. Formally, define : [math]\displaystyle{ sim(c_1, c_2) = \max_{c \in S(c_1, c_2)} [−\log p(c)] }[/math] (1)