Machine learning in mental health – let’s bring our clinicians in the loop

In recent years, there has been an outpouring of mental health research in remote patient monitoring, digital biomarkers, and machine learning (ML) that is focused on analyzing patient device data [1] [2] [3] [4] [5] [6] [7] [8] [9]. Studies have demonstrated that analysis of these data streams from phones show promise in:

• Classifying bipolar states based on digital biomarkers like speech, activity, mood, and reported sleep [10] [11] [12] [13]

• Distinguishing children with ADHD from healthy controls based on eye movements [14]

• Predicting schizophrenic episodes prior to occurrence with screen on/off state, GPS, and symptom surveys [15]

• Estimating neurocognitive function from touchscreen swiping and tapping patterns [16]

In addition to researchers aiming to diagnose and monitor pathologies, pharmaceutical companies have also recognized the value digital biomarkers will hold in drug development and personalized medication selection, as evidenced by recent collaborations between Mindstrong and BlackThorn Therapeutics[17], and Cerebral with Alto Neuroscience [18]. These studies and collaborations are exciting, but they are small and exploratory, and the technology has not yet translated to clinical practice [19].

Review of the ML literature suggests that lack of high-quality, high-volume, diverse datasets with validated ground truth information is the major bottleneck to development [20] [21] [22] [23] [24] [25] [26]. Without this data, ML researchers cannot adequately train and test their algorithms to develop ones they know are worth evaluating in large, expensive clinical trials. Acquisition of such training data requires significant pre-trial collaboration between researchers and clinicians to obtain validated ground truth information about patient states to accompany digital biomarker streams. In other words, we need our clinicians in the loop, helping to collect data, at a large scale prior to clinical trials.

But what’s in it for clinicians? We don’t have automated interpretation of the data to inform their clinical decision-making yet, so what is their incentive to sacrifice their valuable time to participate in dataset collection? It is difficult to say because there is a lack of studies evaluating the effectiveness of putting raw digital biomarkers data in the hands of practicing clinicians, but one case series I was able to find is promising [27]. It highlights how clinicians are able to digest and act upon raw markers like sleep and simple correlations between data streams like mobility and psychosis, and between mood and medication adherence without any advanced algorithms. This isn’t surprising, as measurement-based often equates to better care, but we need more studies like this to convince clinicians to use raw digital biomarkers in practice so that they can understand patients’ metal states more longitudinally, and researchers can gather the datasets they need to unlock ML in mental health.

References:

[1] Hidalgo-Mazzei D, Young AH, Vieta E, et al. Behavioural biomarkers and mobile mental health: a new paradigm. Int J Bipolar Disord. 2018;6, 9.

[2] Coravos A, Khozin S, Mandl KD. Developing and adopting safe and effective digital biomarkers to improve patient outcomes. npj Digit. Med. 2019;2, 14.

[3] Dagum P. Digital Brain Biomarkers of Human Cognition and Mood. Digital Phenotyping and Mobile Sensing: Studies in Neuroscience, Psychology and Behavioral Economics. 2019;93-107.

[4] Birk RH, Samuel G. Can digital data diagnose mental health problems? A sociological exploration of ‘digital phenotyping’. Sociology of Health & Illness. 2020;42(8), 1873-1887.

[5] Boukhechba M, Baglione AN, Barenes LE. Leveraging Mobile Sensing and Machine Learning for Personalized Mental Health Care. SAGE Journals. 2020.

[6] Marsch LA. Digital health data-driven approaches to understand human behavior. Neuropsychopharmacol. 2021;46, 191–196.

[7] Hirschtritt ME, Insel TR. Digital Technologies in Psychiatry: Present and Future. The Journal of Lifelong Learning in Psychiatry. 2018.

[8] Liang Y, Zheng X, Zeng DD. A survey on big data-driven digital phenotyping of mental health. Information Fusion. 2019;52: 290-307.

[9] Chauvin JJ, Insel TR. Building the Thermometer for Mental Health. Cerebrum. 2018.

[10] Faurholt-Jepsen M, Busk J, Frost M, Vinberg M, Christensen EM, Winther O, et al. Voice analysis as an objective state marker in bipolar disorder. Transl Psychiatry. 2016a;6:e856.

[11] Crescenzo FD, Quested DJ. Actigraphic features of bipolar disorder: A systematic review and meta-analysis. Sleep Medicine Reviews. 2017;33: 58-69.

[12] Faurholt-Jepsen M, Busk J, Þórarinsdóttir H, Frost M, et al. Objective smartphone data as a potential diagnostic marker of bipolar disorder. Australian & New Zealand Journal of Psychiatry. 2019;53, 2, 119–28.

[13] Stanislaus S, et al. Daily self-reported and automatically generated smartphone-based sleep measurements in patients with newly diagnosed bipolar disorder, unaffected first-degree relatives and healthy control individuals. Evidence-Based Mental Health. 2020.

[14] Varela Casal P et al. Clinical validation of eye vergence as an objective marker for diagnosis of ADHD in children. J. Atten. Disord. 2018.

[15] Barnett I, Torous J, Staples P, Sandoval L, et al. Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology. 2018;43, 8, 1660–6.

[16] Dagum P. Digital biomarkers of cognitive function. NPJ Digit Med. 2018;1:10.

[17] BlackThorn Therapeutics announces innovative clinical collaboration agreement with Mindstrong Health. Business Wire. 2017. https://www.businesswire.com/news/home/20170607005392/en/BlackThorn-Therapeutics-Announces-Innovative-Clinical-Collaboration-Agreement. Accessed 2/23/2022.

[18] Mental health startup Cerebral teams with Alto Neuroscience to bring 'precision psychiatry' to patients' homes. Fierce Healthcare. 2021. https://www.fiercehealthcare.com/tech/mental-health-startup-cerebral-teams-alto-neuroscience-to-bring-precision-psychiatry-to. Accessed 02/23/2022.

[19] Marsch LA. Digital health data-driven approaches to understand human behavior. Neuropsychopharmacol. 2021;46, 191–196.

[20] Boukhechba M, Baglione AN, Barenes LE. Leveraging Mobile Sensing and Machine Learning for Personalized Mental Health Care. SAGE Journals. 2020.

[21] Marsch LA. Digital health data-driven approaches to understand human behavior. Neuropsychopharmacol. 2021;46, 191–196.

[22] Chauvin JJ, Insel TR. Building the Thermometer for Mental Health. Cerebrum. 2018.

[23] Low DM, Bently KH, Ghosh SS. Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope Investig Otolaryngol. 2020;5(1): 96-116.

[24] Tai AMY, Albuquerque A, Carmona NE, Subramanieapillai M, Cha DS, Sheko M, Lee Y, Mansur R, McIntyre RS. Machine learning and big data: Implications for disease modeling and therapeutic discovery in psychiatry. Artificial Intelligence in Medicine. 2019;99.

[25] Garcia-Ceja E, Riegler M, Nordgreen T, Jakobsen P, Oedegaard KJ, Tørresen J. Mental health monitoring with multimodal sensing and machine learning: A survey. Pervasive and Mobile Computing. 2018;51: 1-26.

[26] Thieme A, Belgrave D, Doherty G. Machine Learning in Mental Health: A Systematic Review of the HCI Literature to Support the Development of Effective and Implementable ML Systems. ACM Transactions on Computer-Human Interaction. 2020;27(5): 1-53.

[27] Wisniewski H, Henson P, Torous J. Using a smartphone app to identify clinically relevant behavior trends via symptom report, cognition scores, and exercise levels: a case series. Front Psychiatry. 2019;10:652.