2020 AMachineLearningApproachtoUnder

• (Chien et al., 2020) ⇒ Isabel Chien, Angel Enrique, Jorge Palacios, Tim Regan, Dessie Keegan, David Carter, Sebastian Tschiatschek, Aditya Nori, Anja Thieme, Derek Richards, Gavin Dohert, and Danielle Belgrave. (2020). “A Machine Learning Approach to Understanding Patterns of Engagement with Internet-delivered Mental Health Interventions.” In: JAMA Network Open, 3(7).

Subject Headings: Internet-Based Therapy, Cognitive Behavioral Therapy, Patient Engagement Measure.

Cited By

• http://scholar.google.com/scholar?q=%222020%22+A+Machine+Learning+Approach+to+Understanding+Patterns+of+Engagement+with+Internet-delivered+Mental+Health+Interventions

Quotes

Key Points

Question

Can machine learning techniques be used to identify heterogeneity in patient engagement with internet-based cognitive behavioral therapy for symptoms of depression and anxiety?

Findings

In this cohort study using data from 54 604 individuals, 5 heterogeneous subtypes were identified based on patient engagement with the online intervention. These subtypes were associated with different patterns of patient behavior and different levels of improvement in symptoms of depression and anxiety.

Meaning

The findings of this study suggest that patterns of patient behavior may elucidate different modalities of engagement, which can help to conduct better triage for patients to provide personalized therapeutic activities, helping to improve outcomes and reduce the overall burden of mental health disorders.

Abstract

Importance

The mechanisms by which engagement with internet-delivered psychological interventions are associated with depression and anxiety symptoms are unclear.

Objective

To identify behavior types based on how people engage with an internet-based cognitive behavioral therapy (iCBT) intervention for symptoms of depression and anxiety.

Design, Setting, and Participants

Deidentified data on 54 604 adult patients assigned to the Space From Depression and Anxiety treatment program from January 31, 2015, to March 31, 2019, were obtained for probabilistic latent variable modeling using machine learning techniques to infer distinct patient subtypes, based on longitudinal heterogeneity of engagement patterns with iCBT.

Interventions

A clinician-supported iCBT-based program that follows clinical guidelines for treating depression and anxiety, delivered on a web 2.0 platform.

Main Outcomes and Measures

Log data from user interactions with the iCBT program to inform engagement patterns over time. Clinical outcomes included symptoms of depression (Patient Health Questionnaire-9 [PHQ-9]) and anxiety (Generalized Anxiety Disorder-7 [GAD-7]); PHQ-9 cut point greater than or equal to 10 and GAD-7 scores greater than or equal to 8 were used to define depression and anxiety.

Results

Patients spent a mean (SD) of 111.33 (118.92) minutes on the platform and completed 230.60 (241.21) tools. At baseline, mean PHQ-9 score was 12.96 (5.81) and GAD-7 score was 11.85 (5.14). Five subtypes of engagement were identified based on patient interaction with different program sections over 14 weeks: class 1 (low engagers, 19 930 [36.5%]), class 2 (late engagers, 11 674 [21.4%]), class 3 (high engagers with rapid disengagement, 13 936 [25.5%]), class 4 (high engagers with moderate decrease, 3258 [6.0%]), and class 5 (highest engagers, 5799 [10.6%]). Estimated mean decrease (SE) in PHQ-9 score was 6.65 (0.14) for class 3, 5.88 (0.14) for class 5, and 5.39 (0.14) for class 4; class 2 had the lowest rate of decrease at −4.41 (0.13). Compared with PHQ-9 score decrease in class 1, the Cohen d effect size (SE) was −0.46 (0.014) for class 2, −0.46 (0.014) for class 3, −0.61 (0.021) for class 4, and −0.73 (0.018) for class 5. Similar patterns were found across groups for GAD-7.

Conclusions and Relevance

The findings of this study may facilitate tailoring interventions according to specific subtypes of engagement for individuals with depression and anxiety. Informing clinical decision needs of supporters may be a route to successful adoption of machine learning insights, thus improving clinical outcomes overall.

Introduction

The World Health Organization defines health as a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity.1 Mental disorders present a substantial burden for good health as they have deleterious effects on the individual, society, and the worldwide economy,2,3,4 making their prevention and treatment a public health priority.5,6,7

Responding to the demand for accessible and sustainable mental health care services, internet-delivered psychological interventions offer access to evidence-based treatment and positive clinical outcomes while maintaining quality of care and reducing costs.8,9 Extensive research has reported possible effectiveness of these interventions for treating psychological disorders.9,10,11,12,13 However, more complete understanding of the clinical use of digital therapy programs requires further research.14,15,16 Most previous studies explored the association between use of the interventions and outcomes, relying on single metrics, such as raw use counts.17,18 Other studies suggest that single metrics are unlikely to sufficiently capture associations between engagement and outcomes, especially when compared with other factors, such as the actual level of attention or interactivity during an intervention.19,20 Thus, identifying different behavioral patterns of engagement and linking these patterns to clinical outcomes offer new opportunities for personalizing treatment delivery to reduce nonadherence to therapy and enhance possible effectiveness.20,21

The aim of this study was to examine whether different types of patient behaviors manifest in the way people engage with an internet-based cognitive behavioral therapy (iCBT) intervention for symptoms of depression and anxiety. We used machine learning to build a probabilistic graphical modeling framework to understand longitudinal patterns of engagement with iCBT.22,23,24 We hypothesized that these patterns would allow us to infer distinct, heterogeneous patient behavior subtypes. We further hypothesized that these subtypes are associated with the intervention’s success of improving mental health and that different subtypes of engagement are associated with differences in clinical outcomes.

...

References

1. World Health Organization (WHO) . A State of Complete Physical Mental and Social Well-being and Not Merely the Absence of Disease or Infirmity: Constitution of the World Health Organization Basic Documents. 45th ed. Supplement; 2006. [Google Scholar]
2. Kessler RC, Angermeyer M, Anthony JC, et al. . Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization’s World Mental Health Survey Initiative. World Psychiatry. 2007;6(3):168-176. [PMC free article] [PubMed] [Google Scholar]
3. Kessler RC, Heeringa S, Lakoma MD, et al. . Individual and societal effects of mental disorders on earnings in the United States: results from the national comorbidity survey replication. Am J Psychiatry. 2008;165(6):703-711. doi:10.1176/appi.ajp.2008.08010126 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
4. Alonso J, Angermeyer MC, Bernert S, et al. ; ESEMeD/MHEDEA 2000 Investigators, European Study of the Epidemiology of Mental Disorders (ESEMeD) Project . Disability and quality of life impact of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatr Scand Suppl. 2004;109(420):38-46. [PubMed] [Google Scholar]
5. Substance Abuse and Mental Health Services Administration . Key Substance Use and Mental Health Indicators in the United States: Results From the 2016 National Survey on Drug Use and Health. Center for Behavioral Health Statistics and Quality; Substance Abuse and Mental Health Services Administration; 2017. HHS Publication No. SMA 17-5044, NSDUH Series H-52. [Google Scholar]
6. Whiteford HA, Degenhardt L, Rehm J, et al. . Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575-1586. doi:10.1016/S0140-6736(13)61611-6 [PubMed] [CrossRef] [Google Scholar]
7. World Health Organization . Fact sheets—depression. Updated January 30, 2020. Accessed June 7, 2020. https://www.who.int/news-room/fact-sheets/detail/depression
8. Richards D, Enrique E, Palacios J. Internet-Delivered Cognitive Behaviour Therapy—The Handbook of Brief Therapies: A Practical Guide. Sage; 2019. [Google Scholar]
9. Andersson G, Titov N, Dear BF, Rozental A, Carlbring P. Internet-delivered psychological treatments: from innovation to implementation. World Psychiatry. 2019;18(1):20-28. doi:10.1002/wps.20610 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
10. Andrews G, Basu A, Cuijpers P, et al. . Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord. 2018;55:70-78. doi:10.1016/j.janxdis.2018.01.001 [PubMed] [CrossRef] [Google Scholar]
11. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329-342. doi:10.1016/j.cpr.2012.02.004 [PubMed] [CrossRef] [Google Scholar]
12. Wright JH, Owen JJ, Richards D, et al. . Computer-assisted cognitive-behavior therapy for depression: a systematic review and meta-analysis. J Clin Psychiatry. 2019;80(2):18r12188. doi:10.4088/JCP.18r12188 [PubMed] [CrossRef] [Google Scholar]
13. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018;47(1):1-18. doi:10.1080/16506073.2017.1401115 [PubMed] [CrossRef] [Google Scholar]
14. Donkin L, Hickie IB, Christensen H, et al. . Rethinking the dose-response relationship between usage and outcome in an online intervention for depression: randomized controlled trial. J Med internet Res. 2013;15(10):e231. doi:10.2196/jmir.2771 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
15. Kelders SM, Bohlmeijer ET, Van Gemert-Pijnen JE. Participants, usage, and use patterns of a web-based intervention for the prevention of depression within a randomized controlled trial. J Med internet Res. 2013;15(8):e172. doi:10.2196/jmir.2258 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
16. Van Gemert-Pijnen JE, Kelders SM, Bohlmeijer ET. Understanding the usage of content in a mental health intervention for depression: an analysis of log data. J Med internet Res. 2014;16(1):e27. doi:10.2196/jmir.2991 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
17. Fuhr K, Schröder J, Berger T, et al. . The association between adherence and outcome in an internet intervention for depression. J Affect Disord. 2018;229:443-449. doi:10.1016/j.jad.2017.12.028 [PubMed] [CrossRef] [Google Scholar]
18. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med internet Res. 2011;13(3):e52. doi:10.2196/jmir.1772 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
19. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017;7(2):254-267. doi:10.1007/s13142-016-0453-1 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

20. Baltierra NB, Muessig KE, Pike EC, LeGrand S, Bull SS, Hightow-Weidman LB. More than just tracking time: complex measures of user engagement with an internet-based health promotion intervention. J Biomed Inform. 2016;59:299-307. doi:10.1016/j.jbi.2015.12.015 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

21. Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med internet Res. 2017;19(6):e232. doi:10.2196/jmir.7126 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

22. Schulam P, Saria S. Integrative analysis using coupled latent variable models for individualizing prognoses. J Mach Learn Res. 2016;17(1):8244-8278. [Google Scholar]

23. Alaa AM, Yoon J, Hu S, van der Schaar M. Personalized risk scoring for critical care prognosis using mixtures of gaussian processes. IEEE Trans Biomed Eng. 2018;65(1):207-218. doi:10.1109/TBME.2017.2698602 [PubMed] [CrossRef] [Google Scholar]

24. Belgrave DC, Granell R, Simpson A, et al. . Developmental profiles of eczema, wheeze, and rhinitis: two population-based birth cohort studies. PLoS Med. 2014;11(10):e1001748. doi:10.1371/journal.pmed.1001748 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

25. Richards D, Timulak L, O’Brien E, et al. . A randomized controlled trial of an internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression. Behav Res Ther. 2015;75:20-31. doi:10.1016/j.brat.2015.10.005 [PubMed] [CrossRef] [Google Scholar]

26. Richards D, Duffy D, Blackburn B, et al. . Digital IAPT: the effectiveness & cost-effectiveness of internet-delivered interventions for depression and anxiety disorders in the Improving Access to Psychological Therapies programme: study protocol for a randomised control trial. BMC Psychiatry. 2018;18(1):59. doi:10.1186/s12888-018-1639-5 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

27. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606-613. doi:10.1046/j.1525-1497.2001.016009606.x [PMC free article] [PubMed] [CrossRef] [Google Scholar]

28. Löwe B, Decker O, Müller S, et al. . Validation and standardization of the Generalized Anxiety Disorder screener (GAD-7) in the general population. Med Care. 2008;46(3):266-274. doi:10.1097/MLR.0b013e318160d093 [PubMed] [CrossRef] [Google Scholar]

29. Jacobson NS, Trua P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. In: Kazdin AE, ed. Methodological Issues & Strategies in Clinical Research. American Psychological Association; 1992:631-648. doi:10.1037/10109-042 [CrossRef] [Google Scholar]

30. Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev. 2006;26(1):17-31. doi:10.1016/j.cpr.2005.07.003 [PubMed] [CrossRef] [Google Scholar]

31. Enrique A, Palacios JE, Ryan H, Richards D. Exploring the relationship between usage and outcomes of an internet-based intervention for individuals with depressive symptoms: secondary analysis of data from a randomized controlled trial. J Med internet Res. 2019;21(8):e12775. doi:10.2196/12775 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

32. Elliott M, Doane MJ. Stigma management of mental illness: effects of concealment, discrimination, and identification on well-being. Self Ident. 2015;14(6):654-674. doi:10.1080/15298868.2015.1053518 [CrossRef] [Google Scholar]

33. Chen AT, Wu S, Tomasino KN, Lattie EG, Mohr DC. A multi-faceted approach to characterizing user behavior and experience in a digital mental health intervention. J Biomed Inform. 2019;94:103187. doi:10.1016/j.jbi.2019.103187 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

34. Nathan PE, Gorman JM, eds. A Guide to Treatments That Work. Oxford University Press; 2015. [Google Scholar]

35. Duncan BL, Reese RJ. Empirically supported treatments, evidence-based treatments, and evidence-based practice. In: Weiner I, Stricker G, Widiger TA, eds. Handbook of Psychology. 2nd ed. Wiley; 2012:489-513. [Google Scholar]

36. Bellotti V, Edwards K. Intelligibility and accountability: human considerations in context-aware systems. Hum Comput Interact. 2001;16(2-4):193-212. doi:10.1207/S15327051HCI16234_05 [CrossRef] [Google Scholar]

37. Chikersal P, Belgrave D, Doherty G, et al. . Understanding client support strategies to improve clinical outcomes in an online mental health intervention. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems; Association for Computing Machinery, April 2020. doi:10.1145/3313831.3376341 [CrossRef] [Google Scholar];

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2020 AMachineLearningApproachtoUnder	Isabel Chien Angel Enrique Jorge Palacios Tim Regan Dessie Keegan David Carter Sebastian Tschiatschek Aditya Nori Anja Thieme Derek Richards Gavin Dohert Danielle Belgrave			A Machine Learning Approach to Understanding Patterns of Engagement with Internet-delivered Mental Health Interventions						2020