%0 Journal Article %@ 2561-326X %I JMIR Publications %V 9 %N %P e59261 %T Analysis of Social Performance and Action Units During Social Skills Training: Focus Group Study of Adults With Autism Spectrum Disorder and Schizophrenia %A Tanaka,Hiroki %A Miyamoto,Kana %A Hamet Bagnou,Jennifer %A Prigent,Elise %A Clavel,Céline %A Martin,Jean-Claude %A Nakamura,Satoshi %K social performance rating scale %K social skills training %K autism spectrum disorder %K schizophrenia %K facial expressions %K social %K autism %K training %K communication %K trainers %K tool %K neurological %D 2025 %7 10.1.2025 %9 %J JMIR Form Res %G English %X Background: Social communication is a crucial factor influencing human social life. Quantifying the degree of difficulty faced in social communication is necessary for understanding developmental and neurological disorders and for creating systems used in automatic symptom screening and assistive methods such as social skills training (SST). SST by a human trainer is a well-established method. Previous SST used a modified roleplay test to evaluate human social communication skills. However, there are no widely accepted evaluation criteria or social behavioral markers to quantify social performance during SST. Objective: This paper has 2 objectives. First, we propose applying the Social Performance Rating Scale (SPRS) to SST data to measure social communication skills. We constructed a Japanese version of the SPRS already developed in English and French. Second, we attempt to quantify action units during SST for people with autism spectrum disorder (ASD) or schizophrenia. Methods: We used videos of interactions between trainers, adults with ASD (n=16) or schizophrenia (n=15), and control participants (n=19) during SST sessions. Two raters applied the proposed scale to annotate the collected data. We investigated the differences between roleplay tasks and participant groups (ASD, schizophrenia, and control). Furthermore, the intensity of action units on the OpenFace toolkit was measured in terms of mean and SD during SST roleplaying. Results: We found significantly greater gaze scores in adults with ASD than in adults with schizophrenia. Differences were also found between the ratings of different tasks in the adults with schizophrenia and the control participants. Action units numbered AU06 and AU12 were significantly deactivated in people with schizophrenia compared with the control group. Moreover, AU02 was significantly activated in people with ASD compared with the other groups. Conclusions: The results suggest that the SPRS can be a useful tool for assessing social communication skills in different cultures and different pathologies when used with the modified roleplay test. Furthermore, facial expressions could provide effective social and behavioral markers to characterize psychometric properties. Possible future directions include using the SPRS for assessing social behavior during interaction with a digital agent. %R 10.2196/59261 %U https://formative.jmir.org/2025/1/e59261 %U https://doi.org/10.2196/59261 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 11 %N %P e60003 %T Balancing Between Privacy and Utility for Affect Recognition Using Multitask Learning in Differential Privacy–Added Federated Learning Settings: Quantitative Study %A Benouis,Mohamed %A Andre,Elisabeth %A Can,Yekta Said %K privacy preservation %K multitask learning %K federated learning %K privacy %K physiological signals %K affective computing %K wearable sensors %K sensitive data %K empathetic sensors %K data privacy %K digital mental health %K wearables %K ethics %K emotional well-being %D 2024 %7 23.12.2024 %9 %J JMIR Ment Health %G English %X Background: The rise of wearable sensors marks a significant development in the era of affective computing. Their popularity is continuously increasing, and they have the potential to improve our understanding of human stress. A fundamental aspect within this domain is the ability to recognize perceived stress through these unobtrusive devices. Objective: This study aims to enhance the performance of emotion recognition using multitask learning (MTL), a technique extensively explored across various machine learning tasks, including affective computing. By leveraging the shared information among related tasks, we seek to augment the accuracy of emotion recognition while confronting the privacy threats inherent in the physiological data captured by these sensors. Methods: To address the privacy concerns associated with the sensitive data collected by wearable sensors, we proposed a novel framework that integrates differential privacy and federated learning approaches with MTL. This framework was designed to efficiently identify mental stress while preserving private identity information. Through this approach, we aimed to enhance the performance of emotion recognition tasks while preserving user privacy. Results: Comprehensive evaluations of our framework were conducted using 2 prominent public datasets. The results demonstrate a significant improvement in emotion recognition accuracy, achieving a rate of 90%. Furthermore, our approach effectively mitigates privacy risks, as evidenced by limiting reidentification accuracies to 47%. Conclusions: This study presents a promising approach to advancing emotion recognition capabilities while addressing privacy concerns in the context of empathetic sensors. By integrating MTL with differential privacy and federated learning, we have demonstrated the potential to achieve high levels of accuracy in emotion recognition while ensuring the protection of user privacy. This research contributes to the ongoing efforts to use affective computing in a privacy-aware and ethical manner. %R 10.2196/60003 %U https://mental.jmir.org/2024/1/e60003 %U https://doi.org/10.2196/60003 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 11 %N %P e57415 %T Exploring the Effects of Variety and Amount of Mindfulness Practices on Depression, Anxiety, and Stress Symptoms: Longitudinal Study on a Mental Health–Focused eHealth System for Patients With Breast or Prostate Cancer %A Malandrone,Francesca %A Urru,Sara %A Berchialla,Paola %A Rossini,Pierre Gilbert %A Oliva,Francesco %A Bianchi,Silvia %A Ottaviano,Manuel %A Gonzalez-Martinez,Sergio %A Carli,Vladimir %A Valenza,Gaetano %A Scilingo,Enzo Pasquale %A Carletto,Sara %A Ostacoli,Luca %K depression %K anxiety %K stress %K internet-based %K mental health %K mindfulness %K breast cancer %K prostate cancer %K cancer-related mental distress %K emotional distress %K psychological distress %K mindfulness-based interventions %K MBI %K e-MBI %K dispositional mindfulness %K self-compassion %K mental wellbeing %K mobile phone %D 2024 %7 21.11.2024 %9 %J JMIR Ment Health %G English %X Background: Patients with cancer often face depression and anxiety, and mindfulness-based interventions, including internet-based versions, can effectively reduce these symptoms and improve their quality of life. This study aims to investigate the impact of internet-based mindfulness-based interventions (e-MBIs) on anxiety, depression, and stress symptoms in patients with prostate or breast cancer. Objective: The primary aims are to assess the association between the amount and variety of e-MBI practices and symptom reduction. Second, this study aims to examine how baseline information such as sociodemographic characteristics, dispositional mindfulness (DM), and dispositional self-compassion (DSC) correlate with both app usage and symptom reduction. Methods: Participants included 107 patients with cancer (68 women with breast cancer and 38 men with prostate cancer) enrolled in a hospital setting. They were assigned to the intervention group of the NEVERMIND project, using the e-BMI module via the NEVERMIND app. A longitudinal design involved Pearson correlation analysis to determine the relationship between the amount and duration of e-MBI practices. Linear regression analysis was conducted to gauge the dose-response effect, evaluating the impact of DM and DSC on depression, anxiety, and stress. Negative binomial regression was conudcted to study sociodemographic factors’ influence on the amount of practice in e-MBIs. Results: The participants with more diverse and sustained mindfulness practices experienced significant reductions in depression, anxiety, and stress. A high correlation (0.94) between e-MBI practices and symptom reduction was also highlighted. Male, married, and highly educated patients were more likely to engage in mindfulness. Even if DM and DSC did not impact the amount or variety of practices correlated, they were correlated with symptom reduction, showing that higher levels were associated with significant reductions in depression, anxiety, and stress. Conclusions: While more e-MBI practice is linked to reduced anxiety, depression, and stress, this study emphasizes the crucial role of variety of practice over amount. DM and DSC are key in shaping intervention effectiveness and may act as protectors against psychological distress. Using app log data, our research provides a unique perspective on e-MBI impact, contributing to cancer care understanding and guiding future studies. %R 10.2196/57415 %U https://mental.jmir.org/2024/1/e57415 %U https://doi.org/10.2196/57415 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e55761 %T Establishing the Foundations of Emotional Intelligence in Care Companion Robots to Mitigate Agitation Among High-Risk Patients With Dementia: Protocol for an Empathetic Patient-Robot Interaction Study %A Nyamathi,Adeline %A Dutt,Nikil %A Lee,Jung-Ah %A Rahmani,Amir M %A Rasouli,Mahkameh %A Krogh,Donna %A Krogh,Erik %A Sultzer,David %A Rashid,Humayun %A Liaqat,Hamza %A Jawad,Riyam %A Azhar,Farhan %A Ahmad,Ali %A Qamar,Bilal %A Bhatti,Taha Yasin %A Khay,Chet %A Ludlow,Jocelyn %A Gibbs,Lisa %A Rousseau,Julie %A Abbasian,Mahyar %A Song,Yutong %A Jeong,Cheonkam %A Brunswicker,Sabine %+ Sue & Bill Gross School of Nursing, University of California Irvine, 854 Health Sciences Rd, Irvine, CA, 92697, United States, 1 9498248932, anyamath@hs.uci.edu %K persons with dementia %K empathy-based care companion robot %K agitation %K fall risk %K artificial intelligence %K AI %D 2024 %7 4.10.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: An estimated 6.7 million persons are living with dementia in the United States, a number expected to double by 2060. Persons experiencing moderate to severe dementia are 4 to 5 times more likely to fall than those without dementia, due to agitation and unsteady gait. Socially assistive robots fail to address the changing emotional states associated with agitation, and it is unclear how emotional states change, how they impact agitation and gait over time, and how social robots can best respond by showing empathy. Objective: This study aims to design and validate a foundational model of emotional intelligence for empathetic patient-robot interaction that mitigates agitation among those at the highest risk: persons experiencing moderate to severe dementia. Methods: A design science approach will be adopted to (1) collect and store granular, personal, and chronological data using Personicle (an open-source software platform developed to automatically collect data from phones and other devices), incorporating real-time visual, audio, and physiological sensing technologies in a simulation laboratory and at board and care facilities; (2) develop statistical models to understand and forecast the emotional state, agitation level, and gait pattern of persons experiencing moderate to severe dementia in real time using machine learning and artificial intelligence and Personicle; (3) design and test an empathy-focused conversation model, focused on storytelling; and (4) test and evaluate this model for a care companion robot (CCR) in the community. Results: The study was funded in October 2023. For aim 1, architecture development for Personicle data collection began with a search for existing open-source data in January 2024. A community advisory board was formed and met in December 2023 to provide feedback on the use of CCRs and provide personal stories. Full institutional review board approval was received in March 2024 to place cameras and CCRs at the sites. In March 2024, atomic marker development was begun. For aim 2, after a review of open-source data on patients with dementia, the development of an emotional classifier was begun. Data labeling was started in April 2024 and completed in June 2024 with ongoing validation. Moreover, the team established a baseline multimodal model trained and validated on healthy-person data sets, using transformer architecture in a semisupervised manner, and later retrained on the labeled data set of patients experiencing moderate to severe dementia. In April 2024, empathy alignment of large language models was initiated using prompt engineering and reinforcement learning. Conclusions: This innovative caregiving approach is designed to recognize the signs of agitation and, upon recognition, intervene with empathetic verbal communication. This proposal has the potential to have a significant impact on an emerging field of computational dementia science by reducing unnecessary agitation and falls of persons experiencing moderate to severe dementia, while reducing caregiver burden. International Registered Report Identifier (IRRID): PRR1-10.2196/55761 %M 39365656 %R 10.2196/55761 %U https://www.researchprotocols.org/2024/1/e55761 %U https://doi.org/10.2196/55761 %U http://www.ncbi.nlm.nih.gov/pubmed/39365656 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 11 %N %P e62679 %T Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study %A Shen,Jocelyn %A DiPaola,Daniella %A Ali,Safinah %A Sap,Maarten %A Park,Hae Won %A Breazeal,Cynthia %+ MIT Media Lab, 75 Amherst Street, Cambridge, MA, 02139, United States, 1 3109802254, joceshen@mit.edu %K empathy %K large language models %K ethics %K transparency %K crowdsourcing %K human-computer interaction %D 2024 %7 25.9.2024 %9 Original Paper %J JMIR Ment Health %G English %X Background: Empathy is a driving force in our connection to others, our mental well-being, and resilience to challenges. With the rise of generative artificial intelligence (AI) systems, mental health chatbots, and AI social support companions, it is important to understand how empathy unfolds toward stories from human versus AI narrators and how transparency plays a role in user emotions. Objective: We aim to understand how empathy shifts across human-written versus AI-written stories, and how these findings inform ethical implications and human-centered design of using mental health chatbots as objects of empathy. Methods: We conducted crowd-sourced studies with 985 participants who each wrote a personal story and then rated empathy toward 2 retrieved stories, where one was written by a language model, and another was written by a human. Our studies varied disclosing whether a story was written by a human or an AI system to see how transparent author information affects empathy toward the narrator. We conducted mixed methods analyses: through statistical tests, we compared user’s self-reported state empathy toward the stories across different conditions. In addition, we qualitatively coded open-ended feedback about reactions to the stories to understand how and why transparency affects empathy toward human versus AI storytellers. Results: We found that participants significantly empathized with human-written over AI-written stories in almost all conditions, regardless of whether they are aware (t196=7.07, P<.001, Cohen d=0.60) or not aware (t298=3.46, P<.001, Cohen d=0.24) that an AI system wrote the story. We also found that participants reported greater willingness to empathize with AI-written stories when there was transparency about the story author (t494=–5.49, P<.001, Cohen d=0.36). Conclusions: Our work sheds light on how empathy toward AI or human narrators is tied to the way the text is presented, thus informing ethical considerations of empathetic artificial social support or mental health chatbots. %M 39321450 %R 10.2196/62679 %U https://mental.jmir.org/2024/1/e62679 %U https://doi.org/10.2196/62679 %U http://www.ncbi.nlm.nih.gov/pubmed/39321450 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 8 %P e29368 %T Understanding People’s Use of and Perspectives on Mood-Tracking Apps: Interview Study %A Schueller,Stephen M %A Neary,Martha %A Lai,Jocelyn %A Epstein,Daniel A %+ Department of Psychological Science, University of California, Irvine, 4201 Social and Behavioral Sciences Gateway, Irvine, CA, 92697, United States, 1 9498243850, s.schueller@uci.edu %K mental health %K mobile apps %K mHealth %K emotions %K affect %K self-tracking %D 2021 %7 11.8.2021 %9 Original Paper %J JMIR Ment Health %G English %X Background: Supporting mental health and wellness is of increasing interest due to a growing recognition of the prevalence and burden of mental health issues. Mood is a central aspect of mental health, and several technologies, especially mobile apps, have helped people track and understand it. However, despite formative work on and dissemination of mood-tracking apps, it is not well understood how mood-tracking apps used in real-world contexts might benefit people and what people hope to gain from them. Objective: To address this gap, the purpose of this study was to understand motivations for and experiences in using mood-tracking apps from people who used them in real-world contexts. Methods: We interviewed 22 participants who had used mood-tracking apps using a semistructured interview and card sorting task. The interview focused on their experiences using a mood-tracking app. We then conducted a card sorting task using screenshots of various data entry and data review features from mood-tracking apps. We used thematic analysis to identify themes around why people use mood-tracking apps, what they found useful about them, and where people felt these apps fell short. Results: Users of mood-tracking apps were primarily motivated by negative life events or shifts in their own mental health that prompted them to engage in tracking and improve their situation. In general, participants felt that using a mood-tracking app facilitated self-awareness and helped them to look back on a previous emotion or mood experience to understand what was happening. Interestingly, some users reported less inclination to document their negative mood states and preferred to document their positive moods. There was a range of preferences for personalization and simplicity of tracking. Overall, users also liked features in which their previous tracked emotions and moods were visualized in figures or calendar form to understand trends. One gap in available mood-tracking apps was the lack of app-facilitated recommendations or suggestions for how to interpret their own data or improve their mood. Conclusions: Although people find various features of mood-tracking apps helpful, the way people use mood-tracking apps, such as avoiding entering negative moods, tracking infrequently, or wanting support to understand or change their moods, demonstrate opportunities for improvement. Understanding why and how people are using current technologies can provide insights to guide future designs and implementations. %M 34383678 %R 10.2196/29368 %U https://mental.jmir.org/2021/8/e29368 %U https://doi.org/10.2196/29368 %U http://www.ncbi.nlm.nih.gov/pubmed/34383678