Item Relevance Measure

An Item Relevance Measure is a user-system performance measure for the relevance of an interactable item to some user population.

• Context:
  • output: Item Relevance Value (from relevant score to non-relevant score).
  • It can range from being a Retrospective Item Relevance Measure (supported by an item relevance query) to being a Predicted Item Relevance Measure.
  • It can be used to define Relevant Items (for relevance ranking tasks (relevance ranking results).
  • It can be influenced by Item Content Type, Item Popularity, Item Direct Interactions, Item Temporal Attributes, Item Social Relationships, ...
  • …
• Example(s):
  • Organization x's ...
  • Discounted Cumulative Gain.
  • a Search Engine Relevance Measure (possibly based on predicted clickthrough rate).
  • a Written Document Relevant Measure, to identify relevant written documents.
  • …
• Counter-Example(s):
  • a User Engagement Measure.
  • a Consumer LTV Measure.
• See: Item Relevance Scoring, Items Recommendation, Relevance, Information Need, Information Retrieval, Latent Semantic Analysis, Cluster Hypothesis.

---

---

References

2017

• (Wikipedia, 2017) ⇒ https://en.wikipedia.org/wiki/Relevance_(information_retrieval)#Clustering_and_relevance Retrieved:2017-6-22.
  • The cluster hypothesis, proposed by C. J. van Rijsbergen in 1979, asserts that two documents that are similar to each other have a high likelihood of being relevant to the same information need. With respect to the embedding similarity space, the cluster hypothesis can be interpreted globally or locally.F. Diaz, Autocorrelation and Regularization of Query-Based Retrieval Scores. PhD thesis, University of Massachusetts Amherst, Amherst, MA, February 2008, Chapter 3. The global interpretation assumes that there exist some fixed set of underlying topics derived from inter-document similarity. These global clusters or their representatives can then be used to relate relevance of two documents (e.g. two documents in the same cluster should both be relevant to the same request). Methods in this spirit include:
    • cluster-based information retrievalW. B. Croft, “A model of cluster searching based on classification,” Information Systems, vol. 5, pp. 189–195, 1980. A. Griffiths, H. C. Luckhurst, and P. Willett, “Using interdocument similarity information in document retrieval systems,” Journal of the American Society for Information Science, vol. 37, no. 1, pp. 3–11, 1986.
    • cluster-based document expansion such as latent semantic analysis or its language modeling equivalents.X. Liu and W. B. Croft, “Cluster-based retrieval using language models,” in SIGIR ’04: Proceedings of the 27th annual International Conference on Research and development in information retrieval, (New York, NY, USA), pp. 186–193, ACM Press, 2004. It is important to ensure that clusters – either in isolation or combination – successfully model the set of possible relevant documents.
  • A second interpretation, most notably advanced by Ellen Voorhees,E. M. Voorhees, “The cluster hypothesis revisited,” in SIGIR ’85: Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval, (New York, NY, USA), pp. 188–196, ACM Press, 1985. focuses on the local relationships between documents. The local interpretation avoids having to model the number or size of clusters in the collection and allow relevance at multiple scales. Methods in this spirit include,
    • multiple cluster retrieval
    • spreading activationS. Preece, A spreading activation network model for information retrieval. PhD thesis, University of Illinois, Urbana-Champaign, 1981. and relevance propagationT. Qin, T.-Y. Liu, X.-D. Zhang, Z. Chen, and W.-Y. Ma, “A study of relevance propagation for web search,” in SIGIR ’05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, (New York, NY, USA), pp. 408–415, ACM Press, 2005. methods
    • local document expansionA. Singhal and F. Pereira, “Document expansion for speech retrieval,” in SIGIR ’99: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, (New York, NY, USA), pp. 34–41, ACM Press, 1999.
    • score regularizationF. Diaz, “Regularizing query-based retrieval scores,” Information Retrieval, vol. 10, pp. 531–562, December 2007.
  • Local methods require an accurate and appropriate document similarity measure.

2015

• (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/relevance_(information_retrieval) Retrieved:2015-2-21.
  • In information science and information retrieval, relevance denotes how well a retrieved document or set of documents meets the information need of the user. Relevance may include concerns such as timeliness, authority or novelty of the result.

2007

• (Pazzani & Billsus, 2007) ⇒ Michael J. Pazzani, and Daniel Billsus. (2007). “Content-based Recommendation Systems.” In: The adaptive web. Springer Berlin Heidelberg, 2007.
  • QUOTE: … associated with a term is a real number that represents the importance or relevance.

---