Published on in Vol 7, No 7 (2020): July

Preprints (earlier versions) of this paper are available at, first published .
Development of an Emotion-Sensitive mHealth Approach for Mood-State Recognition in Bipolar Disorder

Development of an Emotion-Sensitive mHealth Approach for Mood-State Recognition in Bipolar Disorder

Development of an Emotion-Sensitive mHealth Approach for Mood-State Recognition in Bipolar Disorder


1Institute of Clinical Psychology, Centre for Mental Health, Klinikum Stuttgart, Stuttgart, Germany

2Faculty of Science, Eberhard-Karls-University Tübingen, Tübingen, Germany

3Embedded Systems and Sensors Engineering, Research Center for Information Technology, Karlsruhe, Germany

4Sikom Software GmbH, Heidelberg, Germany

5ITK Engineering GmbH, Ruelzheim, Germany

6Institute for Information Processing Technologies, Karlsruhe Institute of Technology, Karlsruhe, Germany

7Department of Clinical Psychology and Psychotherapy, Institute of Psychology, Ruprecht-Karls-University Heidelberg, Heidelberg, Germany

Corresponding Author:

Matthias Backenstrass, PhD

Institute of Clinical Psychology

Centre for Mental Health

Klinikum Stuttgart

Prießnitzweg 24

Stuttgart, 70374


Phone: 49 711 278 22901


Internet- and mobile-based approaches have become increasingly significant to psychological research in the field of bipolar disorders. While research suggests that emotional aspects of bipolar disorders are substantially related to the social and global functioning or the suicidality of patients, these aspects have so far not sufficiently been considered within the context of mobile-based disease management approaches. As a multiprofessional research team, we have developed a new and emotion-sensitive assistance system, which we have adapted to the needs of patients with bipolar disorder. Next to the analysis of self-assessments, third-party assessments, and sensor data, the new assistance system analyzes audio and video data of these patients regarding their emotional content or the presence of emotional cues. In this viewpoint, we describe the theoretical and technological basis of our emotion-sensitive approach and do not present empirical data or a proof of concept. To our knowledge, the new assistance system incorporates the first mobile-based approach to analyze emotional expressions of patients with bipolar disorder. As a next step, the validity and feasibility of our emotion-sensitive approach must be evaluated. In the future, it might benefit diagnostic, prognostic, or even therapeutic purposes and complement existing systems with the help of new and intuitive interaction models.

JMIR Ment Health 2020;7(7):e14267



With a prevalence of more than 1%, bipolar disorder is one of the most common mental disorders worldwide [1]. The disease is associated with the suffering of the affected people and their relatives and poses great challenges to them in their everyday lives [2,3]. The depressive and (hypo) manic episodes can have extensive social and economic consequences for patients with bipolar disorder and their families [3]. In particular, the high relapse rates within bipolar disorder are unsettling for all parties concerned: Even with pharmacological treatment [4] and different psychological approaches [5,6], these relapses cannot completely be prevented in many cases.

Because of the frequently severe and chronic course and the individual and social consequences, additional strategies and support options within patient care are necessary [6,7]. With the proceeding technological development and the digitalization of the health care system, increasing attention has recently been paid to internet- and mobile-based interventions in the field of bipolar disorders [8-10]. Internet-based interventions, such as psychoeducational tutorials, can help to reach a great number of patients. Mobile-based approaches often assess real-time information about illness activity or deliver time-sensitive messages to patients in an ambulatory setting. To achieve this, most systems use smartphone technology, external sensor systems, or wearable devices (portable computer systems that assess and analyze psychophysiological data). In our research project, we developed a new mobile-based assistance system for bipolar disorder. In this viewpoint, we describe the theoretical and technological basis of our approach.

Over the past years, the use of smartphone apps or mobile programs has been investigated with samples of patients with bipolar disorder with an often good feasibility [11-16]. Because of the mobility of digital systems, for example, ambulatory self-assessments can easily be integrated in the patients’ daily routines. This benefits a better availability and can increase the adherence compared with nondigital approaches [11,12,17-19]. Furthermore, the self-assessment approach can be expanded by additional assessments of sensor data: wearable devices or internal smartphone sensors can be used to trace a patient’s mood state [20-22]. Thus, sensor data can aid in automatic recognition of mood-state changes and can support relapse prevention [23-27]. In order to improve disease management as well as treatment compliance and medication adherence in patients with bipolar disorder, self-assessments of patients can be combined with automatic feedback within certain situations [13,14,16,28]. Even simple SMS text message reminders two times per week can improve medication adherence of these patients and help them to create a more positive attitude toward their medication [29]. Interestingly enough, smartphone apps can also support the biological and social rhythms of patients with bipolar disorder. This might lead to a smaller degree of rhythmic disbalances in the long-term course of their disease [30-32]. Beyond that, several studies indicate that mobile-based approaches can reduce the symptom severity in bipolar or other mood disorders [13,14,33-36].

However, the existing approaches neglect the emotional aspects of bipolar disorders. For example, during mood episodes there are typical patterns of experienced emotions: whereas manic states are often characterized by increased happiness or anger and fear, depressive states often show patterns of elevated sadness and disgust [37]. Bipolar disorders are further associated with a generally amplified emotionality [38,39] and difficulties in emotion processing and regulation [40-44], in emotion recognition [45-47], and in the expression of emotions [48,49]. These deficits might partially be related to the current mood state of patients [41,46,47]. Yet, they strongly affect their social and global functioning and are related to severe outcome variables such as suicidality [39,40,42-45,48]. Consequently, emotional aspects have a great impact on the patients’ everyday lives and the long-term course of bipolar disorder.

So far, mobile-based approaches have analyzed the keyboard activity of patients with bipolar disorder [22] or even ambient sound samples [31,32,50] or voice features during phone calls [23,51]. However, to our knowledge, none of the referenced approaches have analyzed the emotional content of audio data or social interactions. Moreover, psychological research has so far focused on emotional responses of fully or partially remitted patients with bipolar disorder by analyzing their facial expressions during standardized tasks [48,49]. Yet, there does not exist any mobile-based approach that analyzes facial expressions of these patients regarding their emotional cues. In reference to the importance of emotional aspects in bipolar disorder, they should play a more important role in the design of mobile Health (mHealth) approaches too. Compared with other behavioral measures, the emotional expressions of patients with bipolar disorder could reflect their emotional reactivity more sensitively [49]. Beyond that, the ambulatory setting would allow to monitor individual changes over time and mood states in real life [52,53]. Thus, emotion-sensitive mHealth systems for bipolar disorder might even increase our understanding of the experienced and expressed emotions of patients or of their impact on the patients’ social and global functioning.

Within the EmAsIn project (Emotion-sensitive Assistance systems for the reactive psychological Interaction with people) we developed the first emotion-sensitive, technical assistance system for patients with bipolar disorder. Because self-assessments of symptoms are the well-established basis of mood monitoring in bipolar disorder [11,12,17-19], our system also includes regular self-assessments of patients. It further analyzes automatically assessed sensor data, because physiological or behavioral data have been shown to be useful in mood-recognition approaches [23-27], and sleep data have been in the focus of bipolar research for a certain period now [54]. In addition, we incorporated third-party assessments of relatives or related parties, because some patients themselves emphasize the importance of an external point of view regarding their current condition [55]. As a consequence, some of the pressure might be taken off the constant self-monitoring of patients with bipolar disorder. The additionally assessed data could also help in individual cases or during certain periods (eg, during severe mood episodes) with less reliable or accurate self-assessments [56,57]. The importance of emotional aspects of bipolar disorders [37-49] motivated us to develop the key component of our system, the emotion-sensitive Story of the Day module. It analyzes audio and video data to explore the emotional experiences and expressions of patients. While many apps in this field are poorly investigated [58], we emphasized the importance of an empirically validated basis of our emotion-sensitive approach [59-61]. To consider the patients’ point of view, we initially started a dialogue with patients with bipolar disorder, which indicated their overall positive attitudes toward our innovative ideas [55].

Our assistance system includes an Android smartphone app and a connected wearable device, which can be both code protected and password protected. It uses multichannel data acquisition to realize an early recognition of mood-state changes in bipolar disorder. It further intends to complement the rather technical exchange of information between systems and patients with new and intuitive interaction models. Therefore, it aims to recognize socioemotional cues in human communication behavior and hereby infer conclusions about emotional and mental states. To this end, the emotion-sensitive Story of the Day module analyzes the verbal and facial expressions of patients in short and actively user-triggered recordings with respect to their emotional content or the presence of emotional cues. Consequently, this module collects active and passive emotion-related data of patients with bipolar disorder and relies on its regular use (see “Story of the Day” section). If all the features of the assistance system are activated, it can gather information about mood states and the course of bipolar disorders with the aid of the following resources:

  • daily self-assessments of patients regarding their mood, activity level, and other relevant symptoms;
  • regular third-party assessments by relatives or other related parties regarding the most important symptoms;
  • automatic assessments of (psycho-) physiological parameters such as heart rate or resting heart rate;
  • automatic assessments of sleep duration and quality;
  • automatic assessments of several behavioral parameters such as recognized activities, movement/acceleration, steps per day, range of motion, or smartphone usage behavior (eg, used apps, number of calls per day);
  • assessments of auditive information (eg, voice, emotional content, speech duration, or breaks) as emotional cues and indicators of mood states;
  • assessments of visual information (facial expressions) as emotional cues and indicators of mood states.

All data resources are presented in Table 1, which also indicates their mandatory or optional usage within the assistance system. Users can switch between different features and tasks by opening the menu of the app. If this feature is activated, the app reminds them of their tasks by using push notifications at a predefined time of the day. Daily self-assessments consist of six 7-point items (from –3 to 3) about symptoms that are relevant to depressive as well as to (hypo-) manic mood states. Negative values are predominantly associated with depressive symptoms, whereas positive values should reflect (hypo) manic states. In addition, as in earlier approaches [19], each user can choose from a given list of potential early warning signs (like mixed emotions or increased caffeine intake) or can create new items. These items are then incorporated into the daily self-assessments, where they are evaluated with yes or no. The third-party assessments are very similar to the self-assessments, but they are realized by using a separate and individually secured web application.

The assistance system uses smartphone sensors to assess several of the behavioral aspects, for instance, with regard to movement or social interaction (without analyzing content information). Information about sleeping behavior and (psycho-) physiological data is continuously collected with the help of the connected wearable device, which users wear on their wrists (see Multimedia Appendix 1 for more detailed information). Whereas most of the sensor data are automatically assessed, users are asked to use the Story of the Day module on a regular basis (eg, once per day). Once information is gathered through the different sources, the assistance system integrates all data with the aid of an external server and visualizes the accessed information in the form of graphic representations over time. In addition, users can implement a digital version of their own, personal crisis plan with individual strategies for different mood states and locally stored contact information. They can also enter information about their actual medication to use the medication reminder of the system. To facilitate the handling, users can use their own and secured web application to insert and manage information.

The system is supposed to recognize mood-state changes in patients with bipolar and to react by sending warning signals or, like other approaches [13,14,28,30], by proposing recommendations (eg, to consult a doctor) and self-management strategies. All system components are fully developed; only the interventions that depend on the automatic mood-state recognition have not been implemented at the actual stage of development. Apart from long-term analyses using big data approaches, neural networks, and machine learning approaches [62], we are pursuing rule-based evaluation models to allow for an increasing accuracy of the state recognition. To this end, patients can adjust the importance of certain parameters for their own mood-recognition approach. For example, they can assign values between 1 and 3 to each relevant factor (self-assessments, third-party assessments, behavioral, physiological and sleep data, or emotional expressions) to implicate their individual importance (with 1 being less important, 2 moderately important, and 3 very important). The system then includes the individual assignments when integrating and analyzing the assessed data. Beyond that, patients may also assign these values to the warning signs, which are then analyzed as separate factors. Figure 1 illustrates the concept of the assistance system.

Table 1. Data resources of the assistance system.
Information source and its componentsParametersCategory
Sensor data


LocationRange of motiona, visited locationsaActivity and behavior

AccelerometerMovements/accelerationaActivity and behavior

Smartphone usageUsage durationa, number of callsa, click rateaActivity and behavior

Social interactionUsage of social appsa, number of messages (SMS text messages, emails, instant messengers)aSocial behavior


VitalHeart ratea, resting heart rateaPhysiological data

Movement patternsSteps/distance per daya, recognized activitiesaActivity and behavior

SleepSleeping/wake up timebSleep duration

Bedtime/getting out of bedbSleep efficiency

Wake phasesb, activity at nightbSleep quality


Third-party assessments

Web application

DiaryThird-party assessmentsaPerception by others
Story of the Day


MicrophoneSpeech durationb, breaksb, words per minutebActivity/urge to speak

Emotional wordsb, color of the voiceb, loudnessbEmotional expression

CameraFacial expressionsbEmotional expression



Figure 1. Concept of the assistance system.
View this figure

Story of the Day

As opposed to earlier approaches, which analyzed ambient sound samples or voice features without processing emotional information [23,31,32,50,51], our emotion-sensitive module analyzes intentionally recorded sequences regarding the contained auditive and visual emotional cues. When a recording is initiated on the start screen of the Story of the Day module, the app uses the smartphone camera to capture video data. In order to secure a sufficient recording quality, the users mount their smartphones in well-positioned holders before activating this feature. Furthermore, external microphones are attached to the smartphones to improve the audio quality of the recordings. At the beginning of each recording sequence, the users are asked to describe an important event of their day. After telling their story the recording must actively be ended and the users are asked if they want to save the recording. If the microphone and camera do not record any information (ie, no recognized voice or face), the recording is automatically discontinued.

The app analyzes the assessed auditive and visual information separately. The verbal information is analyzed regarding the use of emotional words, the color of the voice, its energy level (ie, loudness), the verbal fluency, and the speech rate as well as the extent to which the story is narrated. The count of emotional words in automatic transcriptions of the used language of each recording is based on the Linguistic Inquiry and Word Count (LIWC) program [59] and includes the emotional categories of positive emotions, negative emotions, sadness, anxiety, or anger. The voice analysis follows the EmoVoice approach [60], a framework that uses acoustic signals as emotional classification units and recognizes emotional or mental states on the basis of these signals. For each audio file, the system analyzes segments of 250 ms and assigns values between 0 and 1 to the categories anger, boredom, disgust, fear, happiness, and sadness. The automatic recognition of emotions in facial expressions during the Story of the Day recordings is based on the Facial Action Coding System (FACS) [61]. In short intervals of 1 frame/second, facial expressions are examined evaluating the 4 emotions, namely, happiness, sadness, anger, and anxiety. For each emotion, the percentage frequency of its coding is calculated.

Internet- and mobile-based approaches have become increasingly important to psychological research in the field of bipolar disorders. In particular, the aspiring mHealth approach benefits a consistent self-monitoring of patients with bipolar disorder [11,12,17-19] and allows for mood-recognition approaches based on automatically assessed sensor data [20,21,23-27]. Our new assistance system incorporates some of the well-known components of mHealth systems for bipolar disorder and combines them with the innovative features of third-party assessments and the analysis of emotional expressions.

While the self-perception of patients with bipolar disorder is certainly the most important factor in mood monitoring, self-assessments can be less reliable in specific cases or during severe episodes [56,57]. Beyond that, some patients trust the assessments of relatives or related parties more than their own perception, when it comes to their mood states [55]. Thus, our third-party assessments could help to gain a more comprehensive view regarding the patients’ mood states. In reference to the great burden, which bipolar disorders are putting on the relationships of patients [63], the third-party assessments might even reduce some of the tension: They can shift the external feedback from possibly strained direct interactions to regular web-based assessments.

Our Story of the Day module, as far as we know, is the first mobile-based approach to analyze the emotional expressions of patients with bipolar disorder. As opposed to the analysis of ambient sound samples or voice features during phone calls [23,31,32,50,51], the actively user-triggered Story of the Day recordings allow us to analyze visual and auditive information as well as the emotional content of the spoken language. The well-established FACS [61], LIWC [59], and EmoVoice approach [60] should provide the technical implementation of our emotion-recognition approach with some helpful framework. This development is especially promising when the effects of emotional deficits on the social and global functioning of patients are considered [39,40,42-45,48]. Consequently, our emotion-sensitive approach is not only interesting in the context of mood-state recognition but might also increase our understanding of experienced and expressed emotions of patients with bipolar disorder. The received feedback in regard to their emotional expressions might be especially informative to patients without regular or with strained social interactions. Moreover, the emotional and narrative character of our Story of the Day module might aid a less technical or distant usage experience and might motivate patients to reflect upon their daily (social) experiences and interactions.

Of course, our new assistance system comes with its limitations. Most importantly, the predictive value of our approach concerning its mood-state recognition and its efficacy and effectiveness with respect to relapse prevention has to be addressed in empirical studies with patients with bipolar disorder. In addition, not all patients approve of the involvement of relatives or related parties in their mood-monitoring approach [55]. Our Story of the Day module must also be used on a regular basis to enable its automatic analysis of emotional expressions. Thus, like self-monitoring systems, our emotion-sensitive approach may depend on the patients’ mood state and motivation. However, as a consequence, the Story of the Day module does not automatically assess audio or video data and thus does not interfere with the patients’ privacy or personal space. Beyond that, our assistance system allows patients to activate or deactivate certain features (eg, the third-party assessments) and meets the patients’ expectations of flexible systems [55,64]. Furthermore, based on our preliminary findings, we estimate that the Story of the Day recordings should not take up more than 2 minutes per day. In the future, our Story of the Day approach might be even less effortful as it could possibly be realized in a more natural setting without smartphone holders or external microphones.

Whereas the EmoVoice approach [60] and, in part, the LIWC approach [59] incorporate the analysis of verbally expressed disgust into our emotion-sensitive module, the Story of the Day module does not recognize this emotion in the facial expressions of patients. Because disgust is one of the more frequently experienced emotions in bipolar disorder [37], subsequent mobile-based FACS approaches [61] should possibly be programmed to include this emotion as well. Finally, our Story of the Day module does not react to suicidal statements and suicidality is not assessed during the self-assessments. The monitoring of suicidal tendencies or even time-sensitive interventions in case of severe suicidal crises with technological help comes with extensive ethical or legal considerations and can have unexpected effects [65]. Accordingly, before implementing such features into mobile-based approaches for bipolar disorder, their feasibility and effects should be examined thoroughly.

With this in mind, there are still some issues to be dealt with in the further development of our assistance system and more research is needed to examine the clinical value of our system. However, our assistance system and its new and innovative features might improve the understanding of the patients’ mood state and could provide important information about the patients’ expressed emotions as well as their (social) interaction behavior. Considering the strong association between emotional aspects and the social and global functioning of patients with bipolar disorder, in the future, emotion-sensitive systems might be even useful during emotion-based treatment approaches in bipolar disorder [66-68].

The mHealth approach offers many opportunities to support patients with bipolar disorder in their everyday struggle with their disease. However, the existing mobile-based approaches do not consider the importance of emotional aspects in bipolar disorder and their implications regarding the social and global functioning of patients. With our assistance system, we aim to address this issue and have therefore implemented the emotion-sensitive Story of the Day module. With the help of this module, our system analyzes the emotional experiences and expressions of patients besides regular self-assessments and third-party assessments as well as the analysis of further sensor data. In the future, emotion-sensitive approaches might not only benefit a better understanding of the patients’ emotional states, but they might also be used to complement the technical exchange of information between systems and patients with more intuitive interaction models. Moreover, they might even support emotion-based interventions in bipolar disorder.


This work was supported by the German Federal Ministry of Education and Research (Grant no. 16SV7357). The EmAsIn project was approved by the Ethical Committee of the Faculty of Behavioural and Cultural Studies, Ruprecht-Karls University Heidelberg.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Assessment and analysis of sleep data.

DOCX File , 13 KB

  1. Grande I, Berk M, Birmaher B, Vieta E. Bipolar disorder. Lancet 2016 Apr 09;387(10027):1561-1572. [CrossRef] [Medline]
  2. Murray CJL, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012 Dec 15;380(9859):2197-2223. [CrossRef] [Medline]
  3. Rosa AR, Reinares M, Michalak EE, Bonnin CM, Sole B, Franco C, et al. Functional impairment and disability across mood states in bipolar disorder. Value Health 2010 Dec;13(8):984-988 [FREE Full text] [CrossRef] [Medline]
  4. Fountoulakis KN, Kasper S, Andreassen O, Blier P, Okasha A, Severus E, et al. Efficacy of pharmacotherapy in bipolar disorder: a report by the WPA section on pharmacopsychiatry. Eur Arch Psychiatry Clin Neurosci 2012 Jun;262 Suppl 1:1-48. [CrossRef] [Medline]
  5. Oud M, Mayo-Wilson E, Braidwood R, Schulte P, Jones SH, Morriss R, et al. Psychological interventions for adults with bipolar disorder: systematic review and meta-analysis. Br J Psychiatry 2016 Mar;208(3):213-222. [CrossRef] [Medline]
  6. Miziou S, Tsitsipa E, Moysidou S, Karavelas V, Dimelis D, Polyzoidou V, et al. Psychosocial treatment and interventions for bipolar disorder: a systematic review. Ann Gen Psychiatry 2015;14:19 [FREE Full text] [CrossRef] [Medline]
  7. Catalá-López F, Gènova-Maleras R, Vieta E, Tabarés-Seisdedos R. The increasing burden of mental and neurological disorders. Eur Neuropsychopharmacol 2013 Nov;23(11):1337-1339. [CrossRef] [Medline]
  8. Hidalgo-Mazzei D, Mateu A, Reinares M, Matic A, Vieta E, Colom F. Internet-based psychological interventions for bipolar disorder: Review of the present and insights into the future. J Affect Disord 2015 Aug 28;188:1-13. [CrossRef] [Medline]
  9. Gliddon E, Barnes SJ, Murray G, Michalak EE. Online and mobile technologies for self-management in bipolar disorder: A systematic review. Psychiatr Rehabil J 2017 Sep;40(3):309-319. [CrossRef] [Medline]
  10. Faurholt-Jepsen M, Bauer M, Kessing LV. Smartphone-based objective monitoring in bipolar disorder: status and considerations. Int J Bipolar Disord 2018 Jan 23;6(1):6 [FREE Full text] [CrossRef] [Medline]
  11. Bardram JE, Frost M, Szántó K, Faurholt-Jepsen M, Vinberg M, Kessing LV. Designing mobile health technology for bipolar disorder: a field trial of the monarca system. New York, United States: Association for Computing Machinery; 2013 Presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; April, 2013; Paris, France p. 2627-2636. [CrossRef]
  12. Schwartz S, Schultz S, Reider A, Saunders EFH. Daily mood monitoring of symptoms using smartphones in bipolar disorder: A pilot study assessing the feasibility of ecological momentary assessment. J Affect Disord 2016 Feb;191:88-93. [CrossRef] [Medline]
  13. Wenze SJ, Armey MF, Miller IW. Feasibility and Acceptability of a Mobile Intervention to Improve Treatment Adherence in Bipolar Disorder: A Pilot Study. Behav Modif 2014 Jan 8;38(4):497-515. [CrossRef] [Medline]
  14. Depp CA, Ceglowski J, Wang VC, Yaghouti F, Mausbach BT, Thompson WK, et al. Augmenting psychoeducation with a mobile intervention for bipolar disorder: a randomized controlled trial. J Affect Disord 2015 Mar 15;174:23-30. [CrossRef] [Medline]
  15. Hidalgo-Mazzei D, Mateu A, Reinares M, Murru A, Del MBC, Varo C, et al. Psychoeducation in bipolar disorder with a SIMPLe smartphone application: Feasibility, acceptability and satisfaction. J Affect Disord 2016 Aug;200:58-66. [CrossRef] [Medline]
  16. Miklowitz DJ, Price J, Holmes EA, Rendell J, Bell S, Budge K, et al. Facilitated Integrated Mood Management for adults with bipolar disorder. Bipolar Disord 2012 Mar;14(2):185-197 [FREE Full text] [CrossRef] [Medline]
  17. Schärer LO, Krienke UJ, Graf S, Meltzer K, Langosch JM. Validation of life-charts documented with the personal life-chart app - a self-monitoring tool for bipolar disorder. BMC Psychiatry 2015 Mar 14;15:49 [FREE Full text] [CrossRef] [Medline]
  18. Bopp JM, Miklowitz DJ, Goodwin GM, Stevens W, Rendell JM, Geddes JR. The longitudinal course of bipolar disorder as revealed through weekly text messaging: a feasibility study. Bipolar Disord 2010 May;12(3):327-334 [FREE Full text] [CrossRef] [Medline]
  19. Faurholt-Jepsen M, Torri E, Cobo J, Yazdanyar D, Palao D, Cardoner N, et al. Smartphone-based self-monitoring in bipolar disorder: evaluation of usability and feasibility of two systems. Int J Bipolar Disord 2019 Jan 04;7(1):1 [FREE Full text] [CrossRef] [Medline]
  20. Mühlbauer E, Bauer M, Ebner-Priemer U, Ritter P, Hill H, Beier F, et al. Effectiveness of smartphone-based ambulatory assessment (SBAA-BD) including a predicting system for upcoming episodes in the long-term treatment of patients with bipolar disorders: study protocol for a randomized controlled single-blind trial. BMC Psychiatry 2018 Oct 26;18(1):349 [FREE Full text] [CrossRef] [Medline]
  21. Cochran A, Belman-Wells L, McInnis M. Engagement Strategies for Self-Monitoring Symptoms of Bipolar Disorder With Mobile and Wearable Technology: Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2018 May 10;7(5):e130 [FREE Full text] [CrossRef] [Medline]
  22. Zulueta J, Piscitello A, Rasic M, Easter R, Babu P, Langenecker SA, et al. Predicting Mood Disturbance Severity with Mobile Phone Keystroke Metadata: A BiAffect Digital Phenotyping Study. J Med Internet Res 2018 Dec 20;20(7):e241 [FREE Full text] [CrossRef] [Medline]
  23. Faurholt-Jepsen M, Vinberg M, Frost M, Christensen EM, Bardram J, Kessing LV. Daily electronic monitoring of subjective and objective measures of illness activity in bipolar disorder using smartphones--the MONARCA II trial protocol: a randomized controlled single-blind parallel-group trial. BMC Psychiatry 2014;14:309 [FREE Full text] [CrossRef] [Medline]
  24. Grünerbl A, Muaremi A, Osmani V, Bahle G, Ohler S, Tröster G, et al. Smartphone-based recognition of states and state changes in bipolar disorder patients. IEEE J Biomed Health Inform 2015 Jan;19(1):140-148. [CrossRef] [Medline]
  25. Javelot H, Spadazzi A, Weiner L, Garcia S, Gentili C, Kosel M, et al. Telemonitoring with respect to mood disorders and information and communication technologies: overview and presentation of the PSYCHE project. Biomed Res Int 2014;2014:104658 [FREE Full text] [CrossRef] [Medline]
  26. Merikangas KR, Swendsen J, Hickie IB, Cui L, Shou H, Merikangas AK, et al. Real-time Mobile Monitoring of the Dynamic Associations Among Motor Activity, Energy, Mood, and Sleep in Adults With Bipolar Disorder. JAMA Psychiatry 2019 Mar 01;76(2):190-198. [CrossRef] [Medline]
  27. Faurholt-Jepsen M, Busk J, Þórarinsdóttir H, Frost M, Bardram JE, Vinberg M, et al. Objective smartphone data as a potential diagnostic marker of bipolar disorder. Aust N Z J Psychiatry 2019 Feb;53(2):119-128. [CrossRef] [Medline]
  28. Hidalgo-Mazzei D, Reinares M, Mateu A, Nikolova VL, Bonnín CDM, Samalin L, et al. OpenSIMPLe: A real-world implementation feasibility study of a smartphone-based psychoeducation programme for bipolar disorder. J Affect Disord 2018 Dec 01;241:436-445. [CrossRef] [Medline]
  29. Menon V, Selvakumar N, Kattimani S, Andrade C. Therapeutic effects of mobile-based text message reminders for medication adherence in bipolar I disorder: Are they maintained after intervention cessation? J Psychiatr Res 2018 Sep;104:163-168. [CrossRef] [Medline]
  30. Hidalgo-Mazzei D, Reinares M, Mateu A, Juruena MF, Young AH, Pérez-Sola V, et al. Is a SIMPLe smartphone application capable of improving biological rhythms in bipolar disorder? J Affect Disord 2017 Dec 01;223:10-16. [CrossRef] [Medline]
  31. Matthews M, Abdullah S, Murnane E, Voida S, Choudhury T, Gay G, et al. Development and Evaluation of a Smartphone-Based Measure of Social Rhythms for Bipolar Disorder. Assessment 2016 Aug;23(4):472-483 [FREE Full text] [CrossRef] [Medline]
  32. Abdullah S, Matthews M, Frank E, Doherty G, Gay G, Choudhury T. Automatic detection of social rhythms in bipolar disorder. J Am Med Inform Assoc 2016 May;23(3):538-543. [CrossRef] [Medline]
  33. Ben-Zeev D, Buck B, Chu PV, Razzano L, Pashka N, Hallgren KA. Transdiagnostic Mobile Health: Smartphone Intervention Reduces Depressive Symptoms in People With Mood and Psychotic Disorders. JMIR Ment Health 2019 Apr 12;6(4):e13202 [FREE Full text] [CrossRef] [Medline]
  34. Depp CA, Mausbach B, Granholm E, Cardenas V, Ben-Zeev D, Patterson TL, et al. Mobile interventions for severe mental illness: design and preliminary data from three approaches. J Nerv Ment Dis 2010 Oct;198(10):715-721 [FREE Full text] [CrossRef] [Medline]
  35. Dahne J, Lejuez CW, Diaz VA, Player MS, Kustanowitz J, Felton JW, et al. Pilot Randomized Trial of a Self-Help Behavioral Activation Mobile App for Utilization in Primary Care. Behav Ther 2019 Jul;50(4):817-827. [CrossRef] [Medline]
  36. Josephine K, Josefine L, Philipp D, David E, Harald B. Internet- and mobile-based depression interventions for people with diagnosed depression: A systematic review and meta-analysis. J Affect Disord 2017 Dec 01;223:28-40. [CrossRef] [Medline]
  37. Carolan LA, Power MJ. What basic emotions are experienced in bipolar disorder? Clin Psychol Psychother 2011;18(5):366-378. [CrossRef] [Medline]
  38. Gruber J, Kogan A, Mennin D, Murray G. Real-world emotion? An experience-sampling approach to emotion experience and regulation in bipolar I disorder. J Abnorm Psychol 2013 Nov;122(4):971-983. [CrossRef] [Medline]
  39. Lima IMM, Peckham AD, Johnson SL. Cognitive deficits in bipolar disorders: Implications for emotion. Clin Psychol Rev 2018 Feb;59:126-136 [FREE Full text] [CrossRef] [Medline]
  40. Paris M, Mahajan Y, Kim J, Meade T. Emotional speech processing deficits in bipolar disorder: The role of mismatch negativity and P3a. J Affect Disord 2018 Jul;234:261-269. [CrossRef] [Medline]
  41. Bilderbeck AC, Reed ZE, McMahon HC, Atkinson LZ, Price J, Geddes JR, et al. Associations between mood instability and emotional processing in a large cohort of bipolar patients. Psychol Med 2016 Dec;46(15):3151-3160. [CrossRef] [Medline]
  42. Johnson SL, Carver CS, Tharp JA. Suicidality in Bipolar Disorder: The Role of Emotion-Triggered Impulsivity. Suicide Life Threat Behav 2017 Apr;47(2):177-192 [FREE Full text] [CrossRef] [Medline]
  43. Johnson SL, Tharp JA, Peckham AD, McMaster KJ. Emotion in bipolar I disorder: Implications for functional and symptom outcomes. J Abnorm Psychol 2016 Jan;125(1):40-52 [FREE Full text] [CrossRef] [Medline]
  44. Aparicio A, Santos JL, Jiménez-López E, Bagney A, Rodríguez-Jiménez R, Sánchez-Morla EM. Emotion processing and psychosocial functioning in euthymic bipolar disorder. Acta Psychiatr Scand 2017 Apr;135(4):339-350. [CrossRef] [Medline]
  45. Branco LD, Cotrena C, Ponsoni A, Salvador-Silva R, Vasconcellos SJL, Fonseca RP. Identification and Perceived Intensity of Facial Expressions of Emotion in Bipolar Disorder and Major Depression. Arch Clin Neuropsychol 2018 Jun 01;33(4):491-501. [CrossRef] [Medline]
  46. Gray J, Venn H, Montagne B, Murray L, Burt M, Frigerio E, et al. Bipolar patients show mood-congruent biases in sensitivity to facial expressions of emotion when exhibiting depressed symptoms, but not when exhibiting manic symptoms. Cogn Neuropsychiatry 2006 Nov;11(6):505-520. [CrossRef] [Medline]
  47. Venn HR, Gray JM, Montagne B, Murray LK, Michael Burt D, Frigerio E, et al. Perception of facial expressions of emotion in bipolar disorder. Bipolar Disord 2004 Aug;6(4):286-293. [CrossRef] [Medline]
  48. Bersani G, Polli E, Valeriani G, Zullo D, Melcore C, Capra E, et al. Facial expression in patients with bipolar disorder and schizophrenia in response to emotional stimuli: a partially shared cognitive and social deficit of the two disorders. Neuropsychiatr Dis Treat 2013;9:1137-1144 [FREE Full text] [CrossRef] [Medline]
  49. Broch-Due I, Kjærstad HL, Kessing LV, Miskowiak K. Subtle behavioural responses during negative emotion reactivity and down-regulation in bipolar disorder: A facial expression and eye-tracking study. Psychiatry Res 2018 Aug;266:152-159. [CrossRef] [Medline]
  50. Or F, Torous J, Onnela J. High potential but limited evidence: Using voice data from smartphones to monitor and diagnose mood disorders. Psychiatr Rehabil J 2017 Sep;40(3):320-324. [CrossRef] [Medline]
  51. Pan Z, Gui C, Zhang J, Zhu J, Cui D. Detecting Manic State of Bipolar Disorder Based on Support Vector Machine and Gaussian Mixture Model Using Spontaneous Speech. Psychiatry Investig 2018 Jul;15(7):695-700 [FREE Full text] [CrossRef] [Medline]
  52. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess 2009 Dec;21(4):463-475. [CrossRef] [Medline]
  53. Trull TJ, Ebner-Priemer U. Ambulatory assessment. Annu Rev Clin Psychol 2013;9:151-176 [FREE Full text] [CrossRef] [Medline]
  54. Bauer M, Glenn T, Grof P, Rasgon N, Alda M, Marsh W, et al. Comparison of sleep/wake parameters for self-monitoring bipolar disorder. J Affect Disord 2009 Aug;116(3):170-175. [CrossRef] [Medline]
  55. Daus H, Kislicyn N, Heuer S, Backenstrass M. Disease management apps and technical assistance systems for bipolar disorder: Investigating the patients´ point of view. J Affect Disord 2018 Dec 15;229:351-357. [CrossRef] [Medline]
  56. Bentall RP, Kinderman P, Manson K. Self-discrepancies in bipolar disorder: comparison of manic, depressed, remitted and normal participants. Br J Clin Psychol 2005 Nov;44(Pt 4):457-473. [CrossRef] [Medline]
  57. Inder ML, Crowe MT, Joyce PR, Moor S, Carter JD, Luty SE. "I really don't know whether it is still there": ambivalent acceptance of a diagnosis of bipolar disorder. Psychiatr Q 2010 Jun;81(2):157-165. [CrossRef] [Medline]
  58. Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality. J Med Internet Res 2015;17(8):e198 [FREE Full text] [CrossRef] [Medline]
  59. Pennebaker JW, Booth RJ, Francis ME. Linguistic Inquiry and Word Count: LIWC2007. Austin, Texas, USA:; 2007.
  60. Vogt T, André E, Bee N. EmoVoice — A Framework for Online Recognition of Emotions from Voice. In: André E, Dybkjær L, Minker W, Neumann H, Pieraccini R, Weber M, editors. Perception in Multimodal Dialogue Systems. PIT 2008. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 2008.
  61. Ekman P, Friesen WV. Manual for the Facial Action Coding System. Palo Alto, CA: Consulting Psychologists Press; 1978.
  62. Librenza-Garcia D, Kotzian BJ, Yang J, Mwangi B, Cao B, Pereira Lima LN, et al. The impact of machine learning techniques in the study of bipolar disorder: A systematic review. Neurosci Biobehav Rev 2017 Sep;80:538-554. [CrossRef] [Medline]
  63. Gitlin MJ, Miklowitz DJ. The difficult lives of individuals with bipolar disorder: A review of functional outcomes and their implications for treatment. J Affect Disord 2017 Mar;209:147-154. [CrossRef] [Medline]
  64. Saunders KEA, Bilderbeck AC, Panchal P, Atkinson LZ, Geddes JR, Goodwin GM. Experiences of remote mood and activity monitoring in bipolar disorder: A qualitative study. Eur Psychiatry 2017 Mar;41:115-121. [CrossRef] [Medline]
  65. O'Toole MS, Arendt MB, Pedersen CM. Testing an App-Assisted Treatment for Suicide Prevention in a Randomized Controlled Trial: Effects on Suicide Risk and Depression. Behav Ther 2019 Mar;50(2):421-429. [CrossRef] [Medline]
  66. Power M, Schmidt S. Emotion-focused treatment of unipolar and bipolar mood disorders. Clin. Psychol. Psychother 2004 Jan 30;11(1):44-57. [CrossRef]
  67. Wolkenstein L, Püschel S, Schuster A, Hautzinger M. Emotionsregulationstraining bei bipolaren Patienten – Erste Befunde einer Pilotstudie. Zeitschrift für Psychiatrie, Psychologie und Psychotherapie 2014 Oct;62(4):255-263. [CrossRef]
  68. Eisner L, Eddie D, Harley R, Jacobo M, Nierenberg AA, Deckersbach T. Dialectical Behavior Therapy Group Skills Training for Bipolar Disorder. Behav Ther 2017 Jul;48(4):557-566 [FREE Full text] [CrossRef] [Medline]

FACS: Facial Action Coding System
LIWC: Linguistic Inquiry and Word Count

Edited by G Eysenbach; submitted 05.04.19; peer-reviewed by S Sperry, C Păsărelu, V Rocío; comments to author 16.08.19; revised version received 30.11.19; accepted 26.01.20; published 03.07.20


©Henning Daus, Timon Bloecher, Ronny Egeler, Richard De Klerk, Wilhelm Stork, Matthias Backenstrass. Originally published in JMIR Mental Health (, 03.07.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.