User Engagement Measure

A User Engagement Measure is an user measure in which engagement is quantified for application users.

• Context:
  • output: a User Engagement Score.
  • It can range from being a Retrospective User Engagement Measure (supported by a user engagement query) to being a Predicted User Engagement Measure (supported by a user engagement prediction system).
  • It can include an Awareness Measure.
  • It can range from being a Leading User Engagement Measure to being a Lagging User Engagement Measure.
  • It can range from being an Individual User-level Engagement Measure to being a User Population-level Engagement Measure.
  • It can range from being an Intra-Session User Engagement Measure to being an Inter-Session User Engagement Measure.
  • It can range from being a Distinct User Engagement Measure to being a Composite User Engagement Measure.
  • It can help to inform a User Experience Project, for user experience feature.
  • It can be negatively correlated to a User Disengagement Measure.
  • …
• Example(s):
  • a Lagging User Engagement Measure, such as: Monthly Active Users (MAU), DAU/MAU, and per month.
  • a Leading User Engagement Measure, such as: Purchases per Session, Above-Average Platform Usage per Day, ...
  • a Video Game Player Engagement Metric.
  • a Healthcare Application User Engagement Measure.
  • a Customer Churn Measure?
  • …
• Counter-Example(s):
  • a B2C Revenue-based Customer Measure, such as: customer LTV measure.
  • a Customer Satisfaction Survey, such as a Net Promoter Score.
• See: Item Relevance Measure, Engaging Application.

---

---

References

2022

• https://chat.openai.com/chat
  • QUESTION: A formal definition of a User Engagement Measure
  • ANSWER: User engagement is a measure of the extent to which users of a product, service, or system are actively involved in and invested in the experience or content being provided. It is typically used to assess the effectiveness of a product, service, or system in meeting the needs and preferences of users, and to identify opportunities for improvement.

     User engagement measures may be based on a variety of factors, including usage patterns, feedback, survey responses, and other indicators of user behavior or attitudes. These measures may be collected through a range of methods, including tracking software, surveys, and other forms of data collection.

     User engagement measures are typically used to inform decisions about how to improve the user experience and increase engagement. They may be used in a variety of contexts, including product design, marketing, customer service, and user support. Overall, user engagement measures are designed to help organizations understand and address the needs and preferences of their users, with the goal of improving the user experience and increasing engagement.

2017a

• (Fabijan et al., 2017) ⇒ Aleksander Fabijan, Pavel Dmitriev, Helena Holmström Olsson, and Jan Bosch. (2017). “The Evolution of Continuous Experimentation in Software Product Development: From Data to a Data-driven Organization at Scale.” In: Proceedings of the 39th International Conference on Software Engineering (ICSE-2017).
  • QUOTE: ... Metrics that measure user engagement and purchases are established and consist of a combination of different signals from the logs aggregated per user, session and other analysis units. ...

2017b

• (Chen et al., 2017) ⇒ Zhengxing Chen, Su Xue, John Kolen, Navid Aghdaie, Kazi A. Zaman, Yizhou Sun, and Magy Seif El-Nasr. (2017). “EOMM: An Engagement Optimized Matchmaking Framework.” In: Proceedings of the 26th International Conference on World Wide Web. ISBN:978-1-4503-4913-0 doi:10.1145/3038912.3052559
  • QUOTE: Matchmaking connects multiple players to participate in online player-versus-player games. Current matchmaking systems depend on a single core strategy: create fair games at all times. These systems pair similarly skilled players on the assumption that a fair game is best player experience. We will demonstrate, however, that this intuitive assumption sometimes fails and that matchmaking based on fairness is not optimal for engagement. ...

2015

• (Drutsa et al., 2015) ⇒ Alexey Drutsa, Gleb Gusev, and Pavel Serdyukov. (2015). “Future User Engagement Prediction and Its Application to Improve the Sensitivity of Online Experiments.” In: Proceedings of the 24th International Conference on World Wide Web. ISBN:978-1-4503-3469-3 doi:10.1145/2736277.2741116
  • QUOTE: Engagement measures: We use the logs of Yandex, one of the most popular global search engines, in order to study user engagement. For each user10, we study 6 popular engagement measures, which represent both loyalty and activity aspects of user engagement. For a considered period of time (a day, a week, a month, etc.), we study the following engagement measures calculated over this time period:
    • the number of sessions (S);
    • the number of queries (Q);
    • the number of clicks (C);
    • the presence time (PT);
    • the number of clicks per query (CpQ or CTR);
    • the absence time per session (ATPS).
  • Following common practice [12, 14, 9, 27, 3, 8], we define a session as a sequence of user actions whose dwell times are less than 30 minutes. The presence time PT is measured as the sum of all session lengths (in seconds) observed during the considered time period, while the total absence time is measured as the length of the considered time period minus the presence time. Note that the measures S, Q, C, and PT are additive with respect to the time period. The measure ATpS is calculated as the total absence time divided by the number of user sessions S11. The number of clicks per query CpQ could be regarded as the CTR of the search engine result pages as well. The measures S and ATpS represent the user loyalty [25, 14, 9, 27, 8], whereas the measures Q, C, PT, and CpQ represent the user activity [18, 25, 9, 7, 8] aspects of user engagement [25]. The set of all measures is denoted by M = fS; Q; C; PT; CpQ; ATpSg. Before proceeding to the main problems studied in this paper, we present a brief analysis of these measures. We investigate the relationships between them and their persistence across time in order to have a better interpretation of the prediction quality obtained in the next section.

2014

• (Lalmas et al., 2014) ⇒ Mounia Lalmas, Heather O'Brien, and Elad Yom-Tov. (2014). “Measuring User Engagement.” Morgan & Claypool Publishers. ISBN:1627052615, 9781627052610 doi:10.2200/S00605ED1V01Y201410ICR038
  • QUOTE: User engagement refers to the quality of the user experience that emphasizes the positive aspects of interacting with an online application and, in particular, the desire to use that application longer and repeatedly. User engagement is a key concept in the design of online applications (whether for desktop, tablet or mobile), motivated by the observation that successful applications are not just used, but are engaged with. Users invest time, attention, and emotion in their use of technology, and seek to satisfy pragmatic and hedonic needs. Measurement is critical for evaluating whether online applications are able to successfully engage users, and may inform the design of and use of applications. User engagement is a multifaceted, complex phenomenon; this gives rise to a number of potential measurement approaches. Common ways to evaluate user engagement include using self-report measures, e.g., questionnaires; observational methods, e.g. facial expression analysis, speech analysis; neuro-physiological signal processing methods, e.g., respiratory and cardiovascular accelerations and decelerations, muscle spasms; and web analytics, e.g., number of site visits, click depth. ...

2013

• (Claussen et al., 2013) ⇒ Jörg Claussen, Tobias Kretschmer, and Philip Mayrhofer. (2013). “The Effects of Rewarding User Engagement: The Case of Facebook Apps.” In: Information Systems Research Journal, 24(1). doi:10.1287/isre.1120.0467
  • QUOTE: We study the market for apps on Facebook, the dominant social networking platform, and make use of a rule change by Facebook by which highly engaging apps were rewarded with further opportunities to engage users. ...

2012

• (Lehmann et al., 2012) ⇒ Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, and Georges Dupret. (2012). “Models of User Engagement.” In: Proceedings of the 20th International Conference on User Modeling, Adaptation, and Personalization. ISBN:978-3-642-31453-7 doi:10.1007/978-3-642-31454-4_14
  • QUOTE: Our research goal is to provide a better understanding of how users engage with online services, and how to measure this engagement. We should not speak of one main approach to measure user engagement --- e.g. through one fixed set of metrics --- because engagement depends on the online services at hand. Instead, we should be talking of models of user engagement. ...

2010

• (O'Brien & Toms, 2010) ⇒ Heather L. O'Brien, and Elaine G. Toms. (2010). “The Development and Evaluation of a Survey to Measure User Engagement.” In: Journal of the American Society for Information Science and Technology, 61(1). doi:10.1002/asi.v61:1
  • QUOTE: Facilitating engaging user experiences is essential in the design of interactive systems. To accomplish this, it is necessary to understand the composition of this construct and how to evaluate it. Building on previous work that posited a theory of engagement and identified a core set of attributes that operationalized this construct, we constructed and evaluated a multidimensional scale to measure user engagement.

---