Mean Percentage Ranking Measure

References

(Hu et al., 2008) ⇒ Yifan Hu, Yehuda Koren, and Chris Volinsky. (2008). “Collaborative Filtering for Implicit Feedback Datasets.” In: Proceedings of the 2008 Eighth IEEE International Conference on Data Mining. ISBN:978-0-7695-3502-9 doi:10.1109/ICDM.2008.22
- QUOTE: We denote by rank_{ui} the percentile-ranking of program i within the ordered list of all programs prepared for user u. This way, rankui = 0% would mean that program i is predicted to be the most desirable for user u, thus preceding all other programs in the list. On the other hand, [[rank_{ui}]] = 100% indicates that program i is predicted to be the least preferred for user u, thus placed at the end of the list. (We opted for using percentile-ranks rather than absolute ranks in order to make our discussion general and independent of the number of programs.) Our basic quality measure is the expected percentile ranking of a watching unit in the test period, which is: : [math]\displaystyle{ \bar{\text{rank}} = \frac{\Sigma_{u,i}r^t_{ui} rank_{ui}}{\Sigma_{u,i} r^t_{ui}}. (8) }[/math] Lower values of [math]\displaystyle{ \bar{\text{rank}} }[/math] are more desirable, as they indicate ranking actually watched shows closer to the top of the recommendation lists. Notice that for random predictions, the expected value of rankui is 50% (placing i in the middle of the sorted list). Thus, rank > 50% indicates an algorithm no better than random.