This paper is in the following e-collection/theme issue:

Bipolar Disorders (55) Viewpoints and Perspectives (484)

Published on in Vol 7, No 7 (2020): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14267, first published .
Development of an Emotion-Sensitive mHealth Approach for Mood-State Recognition in Bipolar Disorder

Development of an Emotion-Sensitive mHealth Approach for Mood-State Recognition in Bipolar Disorder

Development of an Emotion-Sensitive mHealth Approach for Mood-State Recognition in Bipolar Disorder

Authors of this article:

Henning Daus1, 2 Author Orcid Image ;   Timon Bloecher3 Author Orcid Image ;   Ronny Egeler4 Author Orcid Image ;   Richard De Klerk5 Author Orcid Image ;   Wilhelm Stork6 Author Orcid Image ;   Matthias Backenstrass1, 7 Author Orcid Image
Article Authors Cited by (3) Tweetations (1) Metrics

    Viewpoint

    1Institute of Clinical Psychology, Centre for Mental Health, Klinikum Stuttgart, Stuttgart, Germany

    2Faculty of Science, Eberhard-Karls-University Tübingen, Tübingen, Germany

    3Embedded Systems and Sensors Engineering, Research Center for Information Technology, Karlsruhe, Germany

    4Sikom Software GmbH, Heidelberg, Germany

    5ITK Engineering GmbH, Ruelzheim, Germany

    6Institute for Information Processing Technologies, Karlsruhe Institute of Technology, Karlsruhe, Germany

    7Department of Clinical Psychology and Psychotherapy, Institute of Psychology, Ruprecht-Karls-University Heidelberg, Heidelberg, Germany

    Corresponding Author:

    Matthias Backenstrass, PhD

    Institute of Clinical Psychology

    Centre for Mental Health

    Klinikum Stuttgart

    Prießnitzweg 24

    Stuttgart, 70374

    Germany

    Phone: 49 711 278 22901

    Email: m.backenstrass@klinikum-stuttgart.de


    Internet- and mobile-based approaches have become increasingly significant to psychological research in the field of bipolar disorders. While research suggests that emotional aspects of bipolar disorders are substantially related to the social and global functioning or the suicidality of patients, these aspects have so far not sufficiently been considered within the context of mobile-based disease management approaches. As a multiprofessional research team, we have developed a new and emotion-sensitive assistance system, which we have adapted to the needs of patients with bipolar disorder. Next to the analysis of self-assessments, third-party assessments, and sensor data, the new assistance system analyzes audio and video data of these patients regarding their emotional content or the presence of emotional cues. In this viewpoint, we describe the theoretical and technological basis of our emotion-sensitive approach and do not present empirical data or a proof of concept. To our knowledge, the new assistance system incorporates the first mobile-based approach to analyze emotional expressions of patients with bipolar disorder. As a next step, the validity and feasibility of our emotion-sensitive approach must be evaluated. In the future, it might benefit diagnostic, prognostic, or even therapeutic purposes and complement existing systems with the help of new and intuitive interaction models.

    JMIR Ment Health 2020;7(7):e14267

    doi:10.2196/14267

    Keywords

    bipolar disorder; mood recognition; emotion recognition; monitoring; mobile apps; assistance system; mHealth 


    With a prevalence of more than 1%, bipolar disorder is one of the most common mental disorders worldwide [1]. The disease is associated with the suffering of the affected people and their relatives and poses great challenges to them in their everyday lives [2,3]. The depressive and (hypo) manic episodes can have extensive social and economic consequences for patients with bipolar disorder and their families [3]. In particular, the high relapse rates within bipolar disorder are unsettling for all parties concerned: Even with pharmacological treatment [4] and different psychological approaches [5,6], these relapses cannot completely be prevented in many cases.

    Because of the frequently severe and chronic course and the individual and social consequences, additional strategies and support options within patient care are necessary [6,7]. With the proceeding technological development and the digitalization of the health care system, increasing attention has recently been paid to internet- and mobile-based interventions in the field of bipolar disorders [8-10]. Internet-based interventions, such as psychoeducational tutorials, can help to reach a great number of patients. Mobile-based approaches often assess real-time information about illness activity or deliver time-sensitive messages to patients in an ambulatory setting. To achieve this, most systems use smartphone technology, external sensor systems, or wearable devices (portable computer systems that assess and analyze psychophysiological data). In our research project, we developed a new mobile-based assistance system for bipolar disorder. In this viewpoint, we describe the theoretical and technological basis of our approach.

    Over the past years, the use of smartphone apps or mobile programs has been investigated with samples of patients with bipolar disorder with an often good feasibility [11-16]. Because of the mobility of digital systems, for example, ambulatory self-assessments can easily be integrated in the patients’ daily routines. This benefits a better availability and can increase the adherence compared with nondigital approaches [11,12,17-19]. Furthermore, the self-assessment approach can be expanded by additional assessments of sensor data: wearable devices or internal smartphone sensors can be used to trace a patient’s mood state [20-22]. Thus, sensor data can aid in automatic recognition of mood-state changes and can support relapse prevention [23-27]. In order to improve disease management as well as treatment compliance and medication adherence in patients with bipolar disorder, self-assessments of patients can be combined with automatic feedback within certain situations [13,14,16,28]. Even simple SMS text message reminders two times per week can improve medication adherence of these patients and help them to create a more positive attitude toward their medication [29]. Interestingly enough, smartphone apps can also support the biological and social rhythms of patients with bipolar disorder. This might lead to a smaller degree of rhythmic disbalances in the long-term course of their disease [30-32]. Beyond that, several studies indicate that mobile-based approaches can reduce the symptom severity in bipolar or other mood disorders [13,14,33-36].

    However, the existing approaches neglect the emotional aspects of bipolar disorders. For example, during mood episodes there are typical patterns of experienced emotions: whereas manic states are often characterized by increased happiness or anger and fear, depressive states often show patterns of elevated sadness and disgust [37]. Bipolar disorders are further associated with a generally amplified emotionality [38,39] and difficulties in emotion processing and regulation [40-44], in emotion recognition [45-47], and in the expression of emotions [48,49]. These deficits might partially be related to the current mood state of patients [41,46,47]. Yet, they strongly affect their social and global functioning and are related to severe outcome variables such as suicidality [39,40,42-45,48]. Consequently, emotional aspects have a great impact on the patients’ everyday lives and the long-term course of bipolar disorder.

    So far, mobile-based approaches have analyzed the keyboard activity of patients with bipolar disorder [22] or even ambient sound samples [31,32,50] or voice features during phone calls [23,51]. However, to our knowledge, none of the referenced approaches have analyzed the emotional content of audio data or social interactions. Moreover, psychological research has so far focused on emotional responses of fully or partially remitted patients with bipolar disorder by analyzing their facial expressions during standardized tasks [48,49]. Yet, there does not exist any mobile-based approach that analyzes facial expressions of these patients regarding their emotional cues. In reference to the importance of emotional aspects in bipolar disorder, they should play a more important role in the design of mobile Health (mHealth) approaches too. Compared with other behavioral measures, the emotional expressions of patients with bipolar disorder could reflect their emotional reactivity more sensitively [49]. Beyond that, the ambulatory setting would allow to monitor individual changes over time and mood states in real life [52,53]. Thus, emotion-sensitive mHealth systems for bipolar disorder might even increase our understanding of the experienced and expressed emotions of patients or of their impact on the patients’ social and global functioning.


    Within the EmAsIn project (Emotion-sensitive Assistance systems for the reactive psychological Interaction with people) we developed the first emotion-sensitive, technical assistance system for patients with bipolar disorder. Because self-assessments of symptoms are the well-established basis of mood monitoring in bipolar disorder [11,12,17-19], our system also includes regular self-assessments of patients. It further analyzes automatically assessed sensor data, because physiological or behavioral data have been shown to be useful in mood-recognition approaches [23-27], and sleep data have been in the focus of bipolar research for a certain period now [54]. In addition, we incorporated third-party assessments of relatives or related parties, because some patients themselves emphasize the importance of an external point of view regarding their current condition [55]. As a consequence, some of the pressure might be taken off the constant self-monitoring of patients with bipolar disorder. The additionally assessed data could also help in individual cases or during certain periods (eg, during severe mood episodes) with less reliable or accurate self-assessments [56,57]. The importance of emotional aspects of bipolar disorders [37-49] motivated us to develop the key component of our system, the emotion-sensitive Story of the Day module. It analyzes audio and video data to explore the emotional experiences and expressions of patients. While many apps in this field are poorly investigated [58], we emphasized the importance of an empirically validated basis of our emotion-sensitive approach [59-61]. To consider the patients’ point of view, we initially started a dialogue with patients with bipolar disorder, which indicated their overall positive attitudes toward our innovative ideas [55].


    Our assistance system includes an Android smartphone app and a connected wearable device, which can be both code protected and password protected. It uses multichannel data acquisition to realize an early recognition of mood-state changes in bipolar disorder. It further intends to complement the rather technical exchange of information between systems and patients with new and intuitive interaction models. Therefore, it aims to recognize socioemotional cues in human communication behavior and hereby infer conclusions about emotional and mental states. To this end, the emotion-sensitive Story of the Day module analyzes the verbal and facial expressions of patients in short and actively user-triggered recordings with respect to their emotional content or the presence of emotional cues. Consequently, this module collects active and passive emotion-related data of patients with bipolar disorder and relies on its regular use (see “Story of the Day” section). If all the features of the assistance system are activated, it can gather information about mood states and the course of bipolar disorders with the aid of the following resources:

    • daily self-assessments of patients regarding their mood, activity level, and other relevant symptoms;
    • regular third-party assessments by relatives or other related parties regarding the most important symptoms;
    • automatic assessments of (psycho-) physiological parameters such as heart rate or resting heart rate;
    • automatic assessments of sleep duration and quality;
    • automatic assessments of several behavioral parameters such as recognized activities, movement/acceleration, steps per day, range of motion, or smartphone usage behavior (eg, used apps, number of calls per day);
    • assessments of auditive information (eg, voice, emotional content, speech duration, or breaks) as emotional cues and indicators of mood states;
    • assessments of visual information (facial expressions) as emotional cues and indicators of mood states.

    All data resources are presented in Table 1, which also indicates their mandatory or optional usage within the assistance system. Users can switch between different features and tasks by opening the menu of the app. If this feature is activated, the app reminds them of their tasks by using push notifications at a predefined time of the day. Daily self-assessments consist of six 7-point items (from –3 to 3) about symptoms that are relevant to depressive as well as to (hypo-) manic mood states. Negative values are predominantly associated with depressive symptoms, whereas positive values should reflect (hypo) manic states. In addition, as in earlier approaches [19], each user can choose from a given list of potential early warning signs (like mixed emotions or increased caffeine intake) or can create new items. These items are then incorporated into the daily self-assessments, where they are evaluated with yes or no. The third-party assessments are very similar to the self-assessments, but they are realized by using a separate and individually secured web application.

    The assistance system uses smartphone sensors to assess several of the behavioral aspects, for instance, with regard to movement or social interaction (without analyzing content information). Information about sleeping behavior and (psycho-) physiological data is continuously collected with the help of the connected wearable device, which users wear on their wrists (see Multimedia Appendix 1 for more detailed information). Whereas most of the sensor data are automatically assessed, users are asked to use the Story of the Day module on a regular basis (eg, once per day). Once information is gathered through the different sources, the assistance system integrates all data with the aid of an external server and visualizes the accessed information in the form of graphic representations over time. In addition, users can implement a digital version of their own, personal crisis plan with individual strategies for different mood states and locally stored contact information. They can also enter information about their actual medication to use the medication reminder of the system. To facilitate the handling, users can use their own and secured web application to insert and manage information.

    The system is supposed to recognize mood-state changes in patients with bipolar and to react by sending warning signals or, like other approaches [13,14,28,30], by proposing recommendations (eg, to consult a doctor) and self-management strategies. All system components are fully developed; only the interventions that depend on the automatic mood-state recognition have not been implemented at the actual stage of development. Apart from long-term analyses using big data approaches, neural networks, and machine learning approaches [62], we are pursuing rule-based evaluation models to allow for an increasing accuracy of the state recognition. To this end, patients can adjust the importance of certain parameters for their own mood-recognition approach. For example, they can assign values between 1 and 3 to each relevant factor (self-assessments, third-party assessments, behavioral, physiological and sleep data, or emotional expressions) to implicate their individual importance (with 1 being less important, 2 moderately important, and 3 very important). The system then includes the individual assignments when integrating and analyzing the assessed data. Beyond that, patients may also assign these values to the warning signs, which are then analyzed as separate factors. Figure 1 illustrates the concept of the assistance system.

    Table 1. Data resources of the assistance system.
    Information source and its componentsParametersCategory
    Sensor data


    		Smartphone



    		LocationRange of motiona, visited locationsaActivity and behavior


    		AccelerometerMovements/accelerationaActivity and behavior


    		Smartphone usageUsage durationa, number of callsa, click rateaActivity and behavior


    		Social interactionUsage of social appsa, number of messages (SMS text messages, emails, instant messengers)aSocial behavior

    		Wearable



    		VitalHeart ratea, resting heart rateaPhysiological data


    		Movement patternsSteps/distance per daya, recognized activitiesaActivity and behavior


    		SleepSleeping/wake up timebSleep duration



    		Bedtime/getting out of bedbSleep efficiency



    		Wake phasesb, activity at nightbSleep quality
    Self-assessments


    		Smartphone



    		DiarySelf-assessmentsbSelf-image
    Third-party assessments


    		Web application



    		DiaryThird-party assessmentsaPerception by others
    Story of the Day


    		Smartphone



    		MicrophoneSpeech durationb, breaksb, words per minutebActivity/urge to speak



    		Emotional wordsb, color of the voiceb, loudnessbEmotional expression


    		CameraFacial expressionsbEmotional expression

    aOptional.

    bMandatory.

    ‎
    Figure 1. Concept of the assistance system.

    Story of the Day

    As opposed to earlier approaches, which analyzed ambient sound samples or voice features without processing emotional information [23,31,32,50,51], our emotion-sensitive module analyzes intentionally recorded sequences regarding the contained auditive and visual emotional cues. When a recording is initiated on the start screen of the Story of the Day module, the app uses the smartphone camera to capture video data. In order to secure a sufficient recording quality, the users mount their smartphones in well-positioned holders before activating this feature. Furthermore, external microphones are attached to the smartphones to improve the audio quality of the recordings. At the beginning of each recording sequence, the users are asked to describe an important event of their day. After telling their story the recording must actively be ended and the users are asked if they want to save the recording. If the microphone and camera do not record any information (ie, no recognized voice or face), the recording is automatically discontinued.

    The app analyzes the assessed auditive and visual information separately. The verbal information is analyzed regarding the use of emotional words, the color of the voice, its energy level (ie, loudness), the verbal fluency, and the speech rate as well as the extent to which the story is narrated. The count of emotional words in automatic transcriptions of the used language of each recording is based on the Linguistic Inquiry and Word Count (LIWC) program [59] and includes the emotional categories of positive emotions, negative emotions, sadness, anxiety, or anger. The voice analysis follows the EmoVoice approach [60], a framework that uses acoustic signals as emotional classification units and recognizes emotional or mental states on the basis of these signals. For each audio file, the system analyzes segments of 250 ms and assigns values between 0 and 1 to the categories anger, boredom, disgust, fear, happiness, and sadness. The automatic recognition of emotions in facial expressions during the Story of the Day recordings is based on the Facial Action Coding System (FACS) [61]. In short intervals of 1 frame/second, facial expressions are examined evaluating the 4 emotions, namely, happiness, sadness, anger, and anxiety. For each emotion, the percentage frequency of its coding is calculated.


    Internet- and mobile-based approaches have become increasingly important to psychological research in the field of bipolar disorders. In particular, the aspiring mHealth approach benefits a consistent self-monitoring of patients with bipolar disorder [11,12,17-19] and allows for mood-recognition approaches based on automatically assessed sensor data [20,21,23-27]. Our new assistance system incorporates some of the well-known components of mHealth systems for bipolar disorder and combines them with the innovative features of third-party assessments and the analysis of emotional expressions.

    While the self-perception of patients with bipolar disorder is certainly the most important factor in mood monitoring, self-assessments can be less reliable in specific cases or during severe episodes [56,57]. Beyond that, some patients trust the assessments of relatives or related parties more than their own perception, when it comes to their mood states [55]. Thus, our third-party assessments could help to gain a more comprehensive view regarding the patients’ mood states. In reference to the great burden, which bipolar disorders are putting on the relationships of patients [63], the third-party assessments might even reduce some of the tension: They can shift the external feedback from possibly strained direct interactions to regular web-based assessments.

    Our Story of the Day module, as far as we know, is the first mobile-based approach to analyze the emotional expressions of patients with bipolar disorder. As opposed to the analysis of ambient sound samples or voice features during phone calls [23,31,32,50,51], the actively user-triggered Story of the Day recordings allow us to analyze visual and auditive information as well as the emotional content of the spoken language. The well-established FACS [61], LIWC [59], and EmoVoice approach [60] should provide the technical implementation of our emotion-recognition approach with some helpful framework. This development is especially promising when the effects of emotional deficits on the social and global functioning of patients are considered [39,40,42-45,48]. Consequently, our emotion-sensitive approach is not only interesting in the context of mood-state recognition but might also increase our understanding of experienced and expressed emotions of patients with bipolar disorder. The received feedback in regard to their emotional expressions might be especially informative to patients without regular or with strained social interactions. Moreover, the emotional and narrative character of our Story of the Day module might aid a less technical or distant usage experience and might motivate patients to reflect upon their daily (social) experiences and interactions.

    Of course, our new assistance system comes with its limitations. Most importantly, the predictive value of our approach concerning its mood-state recognition and its efficacy and effectiveness with respect to relapse prevention has to be addressed in empirical studies with patients with bipolar disorder. In addition, not all patients approve of the involvement of relatives or related parties in their mood-monitoring approach [55]. Our Story of the Day module must also be used on a regular basis to enable its automatic analysis of emotional expressions. Thus, like self-monitoring systems, our emotion-sensitive approach may depend on the patients’ mood state and motivation. However, as a consequence, the Story of the Day module does not automatically assess audio or video data and thus does not interfere with the patients’ privacy or personal space. Beyond that, our assistance system allows patients to activate or deactivate certain features (eg, the third-party assessments) and meets the patients’ expectations of flexible systems [55,64]. Furthermore, based on our preliminary findings, we estimate that the Story of the Day recordings should not take up more than 2 minutes per day. In the future, our Story of the Day approach might be even less effortful as it could possibly be realized in a more natural setting without smartphone holders or external microphones.

    Whereas the EmoVoice approach [60] and, in part, the LIWC approach [59] incorporate the analysis of verbally expressed disgust into our emotion-sensitive module, the Story of the Day module does not recognize this emotion in the facial expressions of patients. Because disgust is one of the more frequently experienced emotions in bipolar disorder [37], subsequent mobile-based FACS approaches [61] should possibly be programmed to include this emotion as well. Finally, our Story of the Day module does not react to suicidal statements and suicidality is not assessed during the self-assessments. The monitoring of suicidal tendencies or even time-sensitive interventions in case of severe suicidal crises with technological help comes with extensive ethical or legal considerations and can have unexpected effects [65]. Accordingly, before implementing such features into mobile-based approaches for bipolar disorder, their feasibility and effects should be examined thoroughly.

    With this in mind, there are still some issues to be dealt with in the further development of our assistance system and more research is needed to examine the clinical value of our system. However, our assistance system and its new and innovative features might improve the understanding of the patients’ mood state and could provide important information about the patients’ expressed emotions as well as their (social) interaction behavior. Considering the strong association between emotional aspects and the social and global functioning of patients with bipolar disorder, in the future, emotion-sensitive systems might be even useful during emotion-based treatment approaches in bipolar disorder [66-68].


    The mHealth approach offers many opportunities to support patients with bipolar disorder in their everyday struggle with their disease. However, the existing mobile-based approaches do not consider the importance of emotional aspects in bipolar disorder and their implications regarding the social and global functioning of patients. With our assistance system, we aim to address this issue and have therefore implemented the emotion-sensitive Story of the Day module. With the help of this module, our system analyzes the emotional experiences and expressions of patients besides regular self-assessments and third-party assessments as well as the analysis of further sensor data. In the future, emotion-sensitive approaches might not only benefit a better understanding of the patients’ emotional states, but they might also be used to complement the technical exchange of information between systems and patients with more intuitive interaction models. Moreover, they might even support emotion-based interventions in bipolar disorder.

    Acknowledgments

    This work was supported by the German Federal Ministry of Education and Research (Grant no. 16SV7357). The EmAsIn project was approved by the Ethical Committee of the Faculty of Behavioural and Cultural Studies, Ruprecht-Karls University Heidelberg.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Assessment and analysis of sleep data.

    DOCX File , 13 KB
    1. Grande I, Berk M, Birmaher B, Vieta E. Bipolar disorder. Lancet. Apr 09, 2016;387(10027):1561-1572. [CrossRef] [Medline]
    2. Murray CJL, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. Dec 15, 2012;380(9859):2197-2223. [CrossRef] [Medline]
    3. Rosa AR, Reinares M, Michalak EE, Bonnin CM, Sole B, Franco C, et al. Functional impairment and disability across mood states in bipolar disorder. Value Health. Dec 2010;13(8):984-988. [FREE Full text] [CrossRef] [Medline]
    4. Fountoulakis KN, Kasper S, Andreassen O, Blier P, Okasha A, Severus E, et al. Efficacy of pharmacotherapy in bipolar disorder: a report by the WPA section on pharmacopsychiatry. Eur Arch Psychiatry Clin Neurosci. Jun 2012;262 Suppl 1:1-48. [CrossRef] [Medline]
    5. Oud M, Mayo-Wilson E, Braidwood R, Schulte P, Jones SH, Morriss R, et al. Psychological interventions for adults with bipolar disorder: systematic review and meta-analysis. Br J Psychiatry. Mar 2016;208(3):213-222. [CrossRef] [Medline]
    6. Miziou S, Tsitsipa E, Moysidou S, Karavelas V, Dimelis D, Polyzoidou V, et al. Psychosocial treatment and interventions for bipolar disorder: a systematic review. Ann Gen Psychiatry. 2015;14:19. [FREE Full text] [CrossRef] [Medline]
    7. Catalá-López F, Gènova-Maleras R, Vieta E, Tabarés-Seisdedos R. The increasing burden of mental and neurological disorders. Eur Neuropsychopharmacol. Nov 2013;23(11):1337-1339. [CrossRef] [Medline]
    8. Hidalgo-Mazzei D, Mateu A, Reinares M, Matic A, Vieta E, Colom F. Internet-based psychological interventions for bipolar disorder: Review of the present and insights into the future. J Affect Disord. Aug 28, 2015;188:1-13. [CrossRef] [Medline]
    9. Gliddon E, Barnes SJ, Murray G, Michalak EE. Online and mobile technologies for self-management in bipolar disorder: A systematic review. Psychiatr Rehabil J. Sep 2017;40(3):309-319. [CrossRef] [Medline]
    10. Faurholt-Jepsen M, Bauer M, Kessing LV. Smartphone-based objective monitoring in bipolar disorder: status and considerations. Int J Bipolar Disord. Jan 23, 2018;6(1):6. [FREE Full text] [CrossRef] [Medline]
    11. Bardram JE, Frost M, Szántó K, Faurholt-Jepsen M, Vinberg M, Kessing LV. Designing mobile health technology for bipolar disorder: a field trial of the monarca system. New York, United States. Association for Computing Machinery; 2013. Presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; April, 2013:2627-2636; Paris, France. [CrossRef]
    12. Schwartz S, Schultz S, Reider A, Saunders EFH. Daily mood monitoring of symptoms using smartphones in bipolar disorder: A pilot study assessing the feasibility of ecological momentary assessment. J Affect Disord. Feb 2016;191:88-93. [CrossRef] [Medline]
    13. Wenze SJ, Armey MF, Miller IW. Feasibility and Acceptability of a Mobile Intervention to Improve Treatment Adherence in Bipolar Disorder: A Pilot Study. Behav Modif. Jan 8, 2014;38(4):497-515. [CrossRef] [Medline]
    14. Depp CA, Ceglowski J, Wang VC, Yaghouti F, Mausbach BT, Thompson WK, et al. Augmenting psychoeducation with a mobile intervention for bipolar disorder: a randomized controlled trial. J Affect Disord. Mar 15, 2015;174:23-30. [CrossRef] [Medline]
    15. Hidalgo-Mazzei D, Mateu A, Reinares M, Murru A, Del MBC, Varo C, et al. Psychoeducation in bipolar disorder with a SIMPLe smartphone application: Feasibility, acceptability and satisfaction. J Affect Disord. Aug 2016;200:58-66. [CrossRef] [Medline]
    16. Miklowitz DJ, Price J, Holmes EA, Rendell J, Bell S, Budge K, et al. Facilitated Integrated Mood Management for adults with bipolar disorder. Bipolar Disord. Mar 2012;14(2):185-197. [FREE Full text] [CrossRef] [Medline]
    17. Schärer LO, Krienke UJ, Graf S, Meltzer K, Langosch JM. Validation of life-charts documented with the personal life-chart app - a self-monitoring tool for bipolar disorder. BMC Psychiatry. Mar 14, 2015;15:49. [FREE Full text] [CrossRef] [Medline]
    18. Bopp JM, Miklowitz DJ, Goodwin GM, Stevens W, Rendell JM, Geddes JR. The longitudinal course of bipolar disorder as revealed through weekly text messaging: a feasibility study. Bipolar Disord. May 2010;12(3):327-334. [FREE Full text] [CrossRef] [Medline]
    19. Faurholt-Jepsen M, Torri E, Cobo J, Yazdanyar D, Palao D, Cardoner N, et al. Smartphone-based self-monitoring in bipolar disorder: evaluation of usability and feasibility of two systems. Int J Bipolar Disord. Jan 04, 2019;7(1):1. [FREE Full text] [CrossRef] [Medline]
    20. Mühlbauer E, Bauer M, Ebner-Priemer U, Ritter P, Hill H, Beier F, et al. Effectiveness of smartphone-based ambulatory assessment (SBAA-BD) including a predicting system for upcoming episodes in the long-term treatment of patients with bipolar disorders: study protocol for a randomized controlled single-blind trial. BMC Psychiatry. Oct 26, 2018;18(1):349. [FREE Full text] [CrossRef] [Medline]
    21. Cochran A, Belman-Wells L, McInnis M. Engagement Strategies for Self-Monitoring Symptoms of Bipolar Disorder With Mobile and Wearable Technology: Protocol for a Randomized Controlled Trial. JMIR Res Protoc. May 10, 2018;7(5):e130. [FREE Full text] [CrossRef] [Medline]
    22. Zulueta J, Piscitello A, Rasic M, Easter R, Babu P, Langenecker SA, et al. Predicting Mood Disturbance Severity with Mobile Phone Keystroke Metadata: A BiAffect Digital Phenotyping Study. J Med Internet Res. Dec 20, 2018;20(7):e241. [FREE Full text] [CrossRef] [Medline]
    23. Faurholt-Jepsen M, Vinberg M, Frost M, Christensen EM, Bardram J, Kessing LV. Daily electronic monitoring of subjective and objective measures of illness activity in bipolar disorder using smartphones--the MONARCA II trial protocol: a randomized controlled single-blind parallel-group trial. BMC Psychiatry. 2014;14:309. [FREE Full text] [CrossRef] [Medline]
    24. Grünerbl A, Muaremi A, Osmani V, Bahle G, Ohler S, Tröster G, et al. Smartphone-based recognition of states and state changes in bipolar disorder patients. IEEE J Biomed Health Inform. Jan 2015;19(1):140-148. [CrossRef] [Medline]
    25. Javelot H, Spadazzi A, Weiner L, Garcia S, Gentili C, Kosel M, et al. Telemonitoring with respect to mood disorders and information and communication technologies: overview and presentation of the PSYCHE project. Biomed Res Int. 2014;2014:104658. [FREE Full text] [CrossRef] [Medline]
    26. Merikangas KR, Swendsen J, Hickie IB, Cui L, Shou H, Merikangas AK, et al. Real-time Mobile Monitoring of the Dynamic Associations Among Motor Activity, Energy, Mood, and Sleep in Adults With Bipolar Disorder. JAMA Psychiatry. Mar 01, 2019;76(2):190-198. [CrossRef] [Medline]
    27. Faurholt-Jepsen M, Busk J, Þórarinsdóttir H, Frost M, Bardram JE, Vinberg M, et al. Objective smartphone data as a potential diagnostic marker of bipolar disorder. Aust N Z J Psychiatry. Feb 2019;53(2):119-128. [CrossRef] [Medline]
    28. Hidalgo-Mazzei D, Reinares M, Mateu A, Nikolova VL, Bonnín CDM, Samalin L, et al. OpenSIMPLe: A real-world implementation feasibility study of a smartphone-based psychoeducation programme for bipolar disorder. J Affect Disord. Dec 01, 2018;241:436-445. [CrossRef] [Medline]
    29. Menon V, Selvakumar N, Kattimani S, Andrade C. Therapeutic effects of mobile-based text message reminders for medication adherence in bipolar I disorder: Are they maintained after intervention cessation? J Psychiatr Res. Sep 2018;104:163-168. [CrossRef] [Medline]
    30. Hidalgo-Mazzei D, Reinares M, Mateu A, Juruena MF, Young AH, Pérez-Sola V, et al. Is a SIMPLe smartphone application capable of improving biological rhythms in bipolar disorder? J Affect Disord. Dec 01, 2017;223:10-16. [CrossRef] [Medline]
    31. Matthews M, Abdullah S, Murnane E, Voida S, Choudhury T, Gay G, et al. Development and Evaluation of a Smartphone-Based Measure of Social Rhythms for Bipolar Disorder. Assessment. Aug 2016;23(4):472-483. [FREE Full text] [CrossRef] [Medline]
    32. Abdullah S, Matthews M, Frank E, Doherty G, Gay G, Choudhury T. Automatic detection of social rhythms in bipolar disorder. J Am Med Inform Assoc. May 2016;23(3):538-543. [CrossRef] [Medline]
    33. Ben-Zeev D, Buck B, Chu PV, Razzano L, Pashka N, Hallgren KA. Transdiagnostic Mobile Health: Smartphone Intervention Reduces Depressive Symptoms in People With Mood and Psychotic Disorders. JMIR Ment Health. Apr 12, 2019;6(4):e13202. [FREE Full text] [CrossRef] [Medline]
    34. Depp CA, Mausbach B, Granholm E, Cardenas V, Ben-Zeev D, Patterson TL, et al. Mobile interventions for severe mental illness: design and preliminary data from three approaches. J Nerv Ment Dis. Oct 2010;198(10):715-721. [FREE Full text] [CrossRef] [Medline]
    35. Dahne J, Lejuez CW, Diaz VA, Player MS, Kustanowitz J, Felton JW, et al. Pilot Randomized Trial of a Self-Help Behavioral Activation Mobile App for Utilization in Primary Care. Behav Ther. Jul 2019;50(4):817-827. [CrossRef] [Medline]
    36. Josephine K, Josefine L, Philipp D, David E, Harald B. Internet- and mobile-based depression interventions for people with diagnosed depression: A systematic review and meta-analysis. J Affect Disord. Dec 01, 2017;223:28-40. [CrossRef] [Medline]
    37. Carolan LA, Power MJ. What basic emotions are experienced in bipolar disorder? Clin Psychol Psychother. 2011;18(5):366-378. [CrossRef] [Medline]
    38. Gruber J, Kogan A, Mennin D, Murray G. Real-world emotion? An experience-sampling approach to emotion experience and regulation in bipolar I disorder. J Abnorm Psychol. Nov 2013;122(4):971-983. [CrossRef] [Medline]
    39. Lima IMM, Peckham AD, Johnson SL. Cognitive deficits in bipolar disorders: Implications for emotion. Clin Psychol Rev. Feb 2018;59:126-136. [FREE Full text] [CrossRef] [Medline]
    40. Paris M, Mahajan Y, Kim J, Meade T. Emotional speech processing deficits in bipolar disorder: The role of mismatch negativity and P3a. J Affect Disord. Jul 2018;234:261-269. [CrossRef] [Medline]
    41. Bilderbeck AC, Reed ZE, McMahon HC, Atkinson LZ, Price J, Geddes JR, et al. Associations between mood instability and emotional processing in a large cohort of bipolar patients. Psychol Med. Dec 2016;46(15):3151-3160. [CrossRef] [Medline]
    42. Johnson SL, Carver CS, Tharp JA. Suicidality in Bipolar Disorder: The Role of Emotion-Triggered Impulsivity. Suicide Life Threat Behav. Apr 2017;47(2):177-192. [FREE Full text] [CrossRef] [Medline]
    43. Johnson SL, Tharp JA, Peckham AD, McMaster KJ. Emotion in bipolar I disorder: Implications for functional and symptom outcomes. J Abnorm Psychol. Jan 2016;125(1):40-52. [FREE Full text] [CrossRef] [Medline]
    44. Aparicio A, Santos JL, Jiménez-López E, Bagney A, Rodríguez-Jiménez R, Sánchez-Morla EM. Emotion processing and psychosocial functioning in euthymic bipolar disorder. Acta Psychiatr Scand. Apr 2017;135(4):339-350. [CrossRef] [Medline]
    45. Branco LD, Cotrena C, Ponsoni A, Salvador-Silva R, Vasconcellos SJL, Fonseca RP. Identification and Perceived Intensity of Facial Expressions of Emotion in Bipolar Disorder and Major Depression. Arch Clin Neuropsychol. Jun 01, 2018;33(4):491-501. [CrossRef] [Medline]
    46. Gray J, Venn H, Montagne B, Murray L, Burt M, Frigerio E, et al. Bipolar patients show mood-congruent biases in sensitivity to facial expressions of emotion when exhibiting depressed symptoms, but not when exhibiting manic symptoms. Cogn Neuropsychiatry. Nov 2006;11(6):505-520. [CrossRef] [Medline]
    47. Venn HR, Gray JM, Montagne B, Murray LK, Michael Burt D, Frigerio E, et al. Perception of facial expressions of emotion in bipolar disorder. Bipolar Disord. Aug 2004;6(4):286-293. [CrossRef] [Medline]
    48. Bersani G, Polli E, Valeriani G, Zullo D, Melcore C, Capra E, et al. Facial expression in patients with bipolar disorder and schizophrenia in response to emotional stimuli: a partially shared cognitive and social deficit of the two disorders. Neuropsychiatr Dis Treat. 2013;9:1137-1144. [FREE Full text] [CrossRef] [Medline]
    49. Broch-Due I, Kjærstad HL, Kessing LV, Miskowiak K. Subtle behavioural responses during negative emotion reactivity and down-regulation in bipolar disorder: A facial expression and eye-tracking study. Psychiatry Res. Aug 2018;266:152-159. [CrossRef] [Medline]
    50. Or F, Torous J, Onnela J. High potential but limited evidence: Using voice data from smartphones to monitor and diagnose mood disorders. Psychiatr Rehabil J. Sep 2017;40(3):320-324. [CrossRef] [Medline]
    51. Pan Z, Gui C, Zhang J, Zhu J, Cui D. Detecting Manic State of Bipolar Disorder Based on Support Vector Machine and Gaussian Mixture Model Using Spontaneous Speech. Psychiatry Investig. Jul 2018;15(7):695-700. [FREE Full text] [CrossRef] [Medline]
    52. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess. Dec 2009;21(4):463-475. [CrossRef] [Medline]
    53. Trull TJ, Ebner-Priemer U. Ambulatory assessment. Annu Rev Clin Psychol. 2013;9:151-176. [FREE Full text] [CrossRef] [Medline]
    54. Bauer M, Glenn T, Grof P, Rasgon N, Alda M, Marsh W, et al. Comparison of sleep/wake parameters for self-monitoring bipolar disorder. J Affect Disord. Aug 2009;116(3):170-175. [CrossRef] [Medline]
    55. Daus H, Kislicyn N, Heuer S, Backenstrass M. Disease management apps and technical assistance systems for bipolar disorder: Investigating the patients´ point of view. J Affect Disord. Dec 15, 2018;229:351-357. [CrossRef] [Medline]
    56. Bentall RP, Kinderman P, Manson K. Self-discrepancies in bipolar disorder: comparison of manic, depressed, remitted and normal participants. Br J Clin Psychol. Nov 2005;44(Pt 4):457-473. [CrossRef] [Medline]
    57. Inder ML, Crowe MT, Joyce PR, Moor S, Carter JD, Luty SE. "I really don't know whether it is still there": ambivalent acceptance of a diagnosis of bipolar disorder. Psychiatr Q. Jun 2010;81(2):157-165. [CrossRef] [Medline]
    58. Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality. J Med Internet Res. 2015;17(8):e198. [FREE Full text] [CrossRef] [Medline]
    59. Pennebaker JW, Booth RJ, Francis ME. Linguistic Inquiry and Word Count: LIWC2007. Austin, Texas, USA. LIWC.net; 2007.
    60. Vogt T, André E, Bee N. EmoVoice — A Framework for Online Recognition of Emotions from Voice. In: André E, Dybkjær L, Minker W, Neumann H, Pieraccini R, Weber M, editors. Perception in Multimodal Dialogue Systems. PIT 2008. Lecture Notes in Computer Science. Berlin, Heidelberg. Springer; 2008.
    61. Ekman P, Friesen WV. Manual for the Facial Action Coding System. Palo Alto, CA. Consulting Psychologists Press; 1978.
    62. Librenza-Garcia D, Kotzian BJ, Yang J, Mwangi B, Cao B, Pereira Lima LN, et al. The impact of machine learning techniques in the study of bipolar disorder: A systematic review. Neurosci Biobehav Rev. Sep 2017;80:538-554. [CrossRef] [Medline]
    63. Gitlin MJ, Miklowitz DJ. The difficult lives of individuals with bipolar disorder: A review of functional outcomes and their implications for treatment. J Affect Disord. Mar 2017;209:147-154. [CrossRef] [Medline]
    64. Saunders KEA, Bilderbeck AC, Panchal P, Atkinson LZ, Geddes JR, Goodwin GM. Experiences of remote mood and activity monitoring in bipolar disorder: A qualitative study. Eur Psychiatry. Mar 2017;41:115-121. [CrossRef] [Medline]
    65. O'Toole MS, Arendt MB, Pedersen CM. Testing an App-Assisted Treatment for Suicide Prevention in a Randomized Controlled Trial: Effects on Suicide Risk and Depression. Behav Ther. Mar 2019;50(2):421-429. [CrossRef] [Medline]
    66. Power M, Schmidt S. Emotion-focused treatment of unipolar and bipolar mood disorders. Clin. Psychol. Psychother. Jan 30, 2004;11(1):44-57. [CrossRef]
    67. Wolkenstein L, Püschel S, Schuster A, Hautzinger M. Emotionsregulationstraining bei bipolaren Patienten – Erste Befunde einer Pilotstudie. Zeitschrift für Psychiatrie, Psychologie und Psychotherapie. Oct 2014;62(4):255-263. [CrossRef]
    68. Eisner L, Eddie D, Harley R, Jacobo M, Nierenberg AA, Deckersbach T. Dialectical Behavior Therapy Group Skills Training for Bipolar Disorder. Behav Ther. Jul 2017;48(4):557-566. [FREE Full text] [CrossRef] [Medline]

    ‎
    FACS: Facial Action Coding System
    LIWC: Linguistic Inquiry and Word Count

    Edited by G Eysenbach; submitted 05.04.19; peer-reviewed by S Sperry, C Păsărelu, V Rocío; comments to author 16.08.19; revised version received 30.11.19; accepted 26.01.20; published 03.07.20.

    Copyright

    ©Henning Daus, Timon Bloecher, Ronny Egeler, Richard De Klerk, Wilhelm Stork, Matthias Backenstrass. Originally published in JMIR Mental Health (http://mental.jmir.org), 03.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.