Original Paper
Abstract
Background: Mental health difficulties among university students have been rising rapidly over the last decade, and the demand for university mental health services commonly far exceeds available resources. Digital interventions are seen as one potential solution to these challenges. However, as in other mental health contexts, digital programs often face low engagement and uptake, and the field lacks usable, engaging, evidence-supported mental health interventions that may be used flexibly when students need them most.
Objective: The aim of this study is to investigate the feasibility and acceptability of a new, in situ intervention tool (Purrble) among university students experiencing anxiety. As an intervention, Purrble was designed to provide in situ support for emotion regulation (ER)—a well-known transdiagnostic construct—directly in the moments when individuals are facing emotionally challenging situations. A secondary aim is to consider the perceived impact of Purrble on youth mental health, as reported by students over a 7-week deployment.
Methods: A mixed methods open trial was conducted with 78 under- and postgraduate students at Oxford University. Participants were recruited based on moderate to high levels of anxiety measured by Generalized Anxiety Disorder-7 at baseline (mean 16.09, SD 3.03). All participants had access to Purrble for 7 weeks during the spring term with data on their perceived anxiety, emotion dysregulation, ER self-efficacy, and engagement with the intervention collected at baseline (pre), week 4 (mid), and week 8 (postintervention). Qualitative responses were also collected at the mid- and postintervention points.
Results: The findings demonstrated a sustained engagement with Purrble over the 7-week period, with the acceptability further supported by the qualitative data indicating that students accepted Purrble and that Purrble was well-integrated into their daily routines. Exploratory quantitative data analysis indicated that Purrble was associated with reductions in student anxiety (dz=0.96, 95% CI 0.62-1.29) and emotion dysregulation (dz=0.69, 95% CI 0.38-0.99), and with an increase in ER self-efficacy (dz=–0.56, 95% CI –0.86 to –0.26).
Conclusions: This is the first trial of a simple physical intervention that aims to provide ongoing ER support to university students. Both quantitative and qualitative data suggest that Purrble is an acceptable and feasible intervention among students, the engagement with which can be sustained at a stable level across a 7-week period while retaining a perceived benefit for those who use it (n=32, 61% of our sample). The consistency of use is particularly promising given that there was no clinician engagement or further support provided beyond Purrble being delivered to the students. These results show promise for an innovative intervention model, which could be complementary to the existing interventions.
doi:10.2196/46826
Keywords
Introduction
Background
Globally, the prevalence of mental health difficulties among students is a major concern [
, ]. Recent epidemiological studies estimate that about one-fifth (20.3%) of all university students are affected by mental health disorders every year [ , , ] with the incidence rates rising substantially over the last 10 years [ ]. In particular, anxiety disorders have been highlighted as a concern among student populations; for example, a recent review across 21 countries shows 11.7%-14.7% of the cross-national population being impacted [ ]. In recent years, this concern has grown as generalized anxiety disorder alone is now thought to be affecting 16.7%-18.6% of students over 12 months to lifetime prevalence [ ]. Alongside anxiety disorders, students are known to struggle with mood and substance use disorders [ - ] that can be associated with the onset of university education [ ].Traditional university services have struggled to meet the increasing needs of this population [
- ], with student demand for mental health services far exceeding changes in enrollment numbers [ , ]. Combined with difficulties such as underresourced counseling centers and lack of staff capacity, students often face extended waiting times when seeking help [ , ]. This may further exacerbate symptoms of mental ill-health and discourage future help-seeking [ ].Digital Mental Health Interventions for University Students
One potential solution to these challenges involves the use of digital technologies that are generally regarded positively by university students [
]. Prior work has shown that web-based service provision can be efficacious with a recent systematic review [ ] showing small intervention effects on anxiety (Hedges g=0.27), depression (g=0.18), and stress (g=0.20) in randomized controlled trials, amid substantial heterogeneity and inclusion of high risk of bias studies in the sample. The 48 intervention programs reviewed were typically delivered through websites (89.6%), focused on cognitive behavioral therapy (or third-wave cognitive behavioral therapy) skills training (66%), and relied on a passive control group (eg, waitlist or psychoeducation materials for behavior change) [ ]. We note, however, that these effects are smaller than those seen in similar systematic reviews of interventions for general populations (eg, for digital anxiety treatments, g=0.62 [ ]).Moreover, the implementation of digital programs within routine services remains challenging in student populations [
- ]. Pragmatic trials on web-based interventions show low engagement and uptake rates [ - ] similar to web-based mental health interventions in other domains [ ]. To overcome such a lack of participant engagement, there is a need to develop usable, engaging, evidence-supported mental health interventions that may be used flexibly based on when students need them most (eg, when stress levels are particularly high and coping skills most warrant deployment).Emotion Regulation and Emerging Digital Interventions
Emotion regulation (ER) is a well-known transdiagnostic factor for a range of mental health difficulties [
- ]. Across young people, difficulties with ER have been associated with a range of mental health difficulties, such as eating disorders [ ], anxiety and depression [ ], as well as self-harm and suicidal thoughts [ ]. In student populations particularly, difficulties with ER have been found to be moderately severe [ , ], with dysregulation related to practices such as maladaptive perfectionism [ ] and coping [ ].While the research on digitally mediated interventions targeting ER is limited to date (cf, recent review [
]), there is one emerging intervention tool—“Purrble”—which was designed to provide in-the-moment ER support in daily life [ , , ] (see Intervention section for more details). Purrble was originally developed for use with child populations, and there is emerging evidence that it could deliver significant benefits through improved ER [ , , ]. For example, Purrble has been shown to facilitate positive parental-child interactions [ , ], support children to self-soothe in the moment [ , ], and enable conversations around children’s emotions [ ]. Within interviews, parents and children recognized that Purrble was incorporated into perceived ER practices (such as disengagement or distraction), which helped to calm or settle the child in emotion-eliciting situations [ ]; the children also remained engaged with Purrble across the studies [ , , ]. From a theory-of-change perspective, Purrble is based on the Gross Process model of ER [ ]: it has been designed to impact two separate stages of the ER process, (1) the attention deployment stage, whereby an individual’s attention is shifted from the emotion-eliciting situations toward interacting with the toy; and (2) the response modulation stage, by facilitating downregulation through pleasant tactile interactions—see prior work [ ] for further details.In summary, while the emerging evidence—together with the theory of change—suggests that Purrble could support ER across a broader age range, no work has so far investigated if and how older populations such as university students might engage with Purrble during stressful periods (eg, during the college term) and how this may impact prevalent mental health concerns, such as anxiety.
This Study
The aim of this feasibility and acceptability study is to evaluate a novel digital intervention device, Purrble, designed to provide in-the-moment ER support in daily life [
] with high-anxiety students in naturalistic contexts. Therefore, the specific objectives are to determine the feasibility of Purrble as an emotion regulatory intervention by considering retention rate, engagement measures, and perceived impact on mental health; to assess the acceptability of Purrble by exploring students’ perceptions and experiences of engaging and appropriating the device using the open-text responses to mid- and postdeployment surveys; and to determine the association between Purrble deployment and students’ anxiety symptoms, ER difficulties, and ER beliefs.Methods
This study is an open trial deployment of an in situ intervention to enhance in-the-moment ER in high-anxiety university students.
Ethics Approval
Ethics approval was obtained from the University of Oxford ethics board (CUREC: 310221) prior to the start of the investigation.
Recruitment
Participants were recruited using advertisements in a university-wide newsletter that included a link to register interest in the study. To take part, interested individuals completed a short screening tool made of a battery of validated measures. To be eligible, one needed to be (1) an Oxford University student, (2) 18 years or older, (3) living in Oxford at the start of the study, and (4) score 10 or above on the Generalized Anxiety Disorder-7 (GAD-7), indicating moderate to severe anxiety symptoms. Due to the limited number of intervention devices available, additional criteria were needed for participant selection, offering priority to nonbinary or male participants due to the high volume of interest from cisgender female students and those with severe anxiety. All selected eligible students were sent consent forms to complete.
Intervention
Purrble is a small, inexpensive, web-based plush animal (
) that aims to guide the user to downregulate their unwanted emotions through a combination of sensors and haptic vibrations, with the theory-of-change targeting the attentional deployment and response modulation components of the ER process [ , ]. The smart toy’s internal state is communicated to users through vibration patterns that mimic a heartbeat, with faster rates corresponding to higher “stress” levels. The fundamental game loop is the following: whenever the smart toy wakes up from sleep, it is startled and has a rapid heartbeat. Shakes, sudden movements, or pressing its ears also wake and “startle” the toy, while calm stroking movements and hugs gradually slow the heartbeat, which eventually changes to a purring vibration, indicating a calm, happy state. The toy also makes varying gentle sounds (ie, sighs, coos, giggles, and grunts) that correspond to its internal state and complement the vibration-based feedback. The full description of the intervention including the detailed theory-of-change model is available [ ].What makes Purrble a potentially unique intervention mechanism is the combination of (1) physical form, (2) associated ongoing availability to provide support as and when needed in everyday settings, and (3) the targeting of the process of in-the-moment ER. Such ongoing in-the-moment ER support—if effective—has the potential to create a positive feedback loop and drive further engagement: for example, if a student is able to reduce their anxiety by interacting with Purrble (and thus feels better), it is more likely that Purrble will become embedded into their daily ER practice. In addition, the in-the-moment physical intervention delivery aims to remove the need for any a priori training on the part of the student and makes Purrble an interestingly complementary approach to existing cognitive interventions (which are reliant on training modules while lacking in-the-moment support for skills application).
Measures
Demographics
Information, such as university, college, student status, location, and gender, was collected as part of the screening process. These were used to ensure inclusion criteria were met.
Feasibility Measures
Digital engagement of Purrble was assessed using Twente Engagement With eHealth Technologies Scale (TWEETS) [
], which consists of 9 items measured on a 5-point Likert scale ranging from strongly disagree (0) to strongly agree (4), with higher scores indicating greater engagement. This measure has been shown to have good reliability, in this sample, Cronbach α was .88.Within the postdeployment survey, participants were asked whether they felt Purrble had helped or hurt their mental health using a 5-point Likert scale of strongly disagree (1) to strongly agree (5). This provided a bespoke measure of Purrble’s perceived impact on their own mental health.
Acceptability Measures
Exploratory open-ended questions were included in the mid- and postintervention surveys. These queried students about situations where Purrble was used, why or why not they used Purrble, perceived emotional impact, usefulness, and personal ER strategies.
Mental Health Measures
Generalized Anxiety Disorder-7
The GAD-7 [
] is a 7-item screening tool designed to assess the presence and severity of generalized anxiety disorder. Individuals are required to rate the frequency of anxiety symptoms on a 4-point Likert scale from not at all (0) to nearly every day (3). Higher scores indicate greater severity of anxiety symptoms, with a threshold of 10 suggesting moderate and 15 implying severe anxiety. In this sample, Cronbach α was .88.Difficulties in Emotion Regulation Scale-18
The Difficulties in Emotion Regulation Scale-18 (DERS-18) [
] is an 18-item questionnaire that assesses how well participants regulate their emotions. Participants are asked to rate how often each statement applies to them on a 5-point Likert scale ranging from almost never (1) to almost always (5). Higher total scores imply greater difficulties with ER. In this sample, Cronbach α was .91.Beliefs About Emotion (ER Beliefs)
Following the adaptation of the Implicit Beliefs About Emotions scale [
, ], ER beliefs was used as a 4-item measure to assess personal beliefs about the malleability of emotions. Participants are asked to score these statements using a 5-point Likert scale ranging from strongly disagree (1) to strongly agree (5). Those with higher scores are thought to hold fixed beliefs about ER, whereas lower scores suggest more malleable beliefs that may be changed. In this sample, Cronbach α was .83.Procedure
Assessments (overview in
) were conducted on the web using King’s College London Qualtrics, with the baseline assessment (anxiety and ER items) taking place the week prior to Purrble deployment and feasibility, acceptability, and mental health measures asked in mid- and postdeployment surveys. Across a 7-week deployment period, participants had access to their individual Purrble with the instructions to use it “as much or as little as they liked.” This was to ensure that any engagement was unbiased and offered an understanding of how students appropriated Purrble over time. Optional daily surveys were sent to participants each evening (~5 minutes), asking about their day and Purrble engagement ( ). Other than the bespoke feasibility measure of impact, all feasibility, acceptability, and mental health measures were asked in both mid- and postdeployment surveys.Following the intervention period, participants were invited to a semistructured interview, which took place via Zoom (Zoom Video Communications). This was to explore their perceptions and experiences of having had Purrble, whether they had engaged with the intervention, and their understanding as to whether Purrble had influenced their ER skills at all.
Survey | Predeployment | Deployment | Follow-up | |||||||
Week 0 | Week 1 | Week 2 | Week 3 | Week 4 | Week 5 | Week 6 | Week 7 | Week 8 | ||
Baseline assessment (GAD-7a, DERS-18b, ERc beliefs) | ✓ | |||||||||
Daily surveys | —d | —d | —d | —d | —d | —d | —d | |||
Middeployment survey (GAD-7, DERS-18, ER beliefs, TWEETSe, exploratory open-ended questions) | ✓ | |||||||||
Postdeployment survey (GAD-7, DERS-18, ER beliefs, TWEETS, exploratory open-ended questions) | ✓ | |||||||||
Interviews with a subset of participants | ✓ |
aGAD-7: Generalized Anxiety Disorder-7.
bDERS-18: Difficulties in Emotion Regulation Scale-18.
cER: emotion regulation.
dCollected daily.
eTWEETS: Twente Engagement With eHealth Technologies Scale.
Mixed Methods Analytic Plan
Statistical data were analyzed using RStudio (Posit) and the lmer package. For baseline measures, averages and SDs were calculated to give an insight into the characteristics of the participant sample.
Intervention feasibility was assessed in three ways by calculating descriptive statistics: (1) calculating retention rate across the study period as percentages of students who submitted mid- or postquestionnaires, (2) self-reported engagement with the intervention through a total score of the TWEETS measure for those retained in the study, and (3) perceived impact of intervention as a percentage of student responses at postdeployment survey. The daily items asking about Purrble engagement were not analyzed due to their optional status and thus high attrition (>70% by week 8 of the study).
To assess the acceptability of Purrble, iterative deductive thematic analysis was used to explore the perceptions and experiences of engaging and appropriating Purrble, as per their own needs. This was achieved by considering participant responses to open-ended questions collected during the exploratory surveys. The qualitative data were analyzed following the 6 phases outlined by Braun and Clarke [
, ]. Following survey data cleaning, all data were imported into NVivo (version 12; Lumivero), and AJW began deductively coding the data. Codes were clustered to develop a meaningful preliminary framework, which was discussed with PS. Additional considerations and reflections were discussed, leading to a revised thematic framework. This was reviewed and finalized by all authors. The semistructured interviews of this study are analyzed elsewhere [ ].To explore the association between Purrble deployment and students’ anxiety symptoms (GAD-7), ER difficulties (DERS-18), and ER beliefs, each of the outcome variables was fitted to progressively more complex linear mixed models to identify the best fit, following the analysis steps from prior analogous work [
].The starting model predicted the outcome variable as a function of time and allowed for both random slopes and intercept for individual participants (unless this reduced the model fit): <outcome> ~ timepoint + (timepoint | participant). We then included predetermined covariates, that is, engagement (TWEETS) and gender. For DERS-18 and ER beliefs, we also included baseline GAD-7 as another covariate. Restricted maximum likelihood methods were used for estimation (the default approach within lme4), which yields unbiased estimates under the assumption that the missing data mechanism is ignorable (ie, data are missing at random [
]). Given the promising trajectory for all variables, within-group effect sizes (Cohen d; 95% CI) were calculated to reflect this change from pre- (week 0) to postdeployment (week 7) on each of the outcome variables.Results
Sample Characteristics
In total, 80 (42 undergraduates, 36 postgraduates) students were enrolled in the study. Two participants did not return signed consent forms and were excluded. Of the full sample (n=78), most participants identified as female (n=48). The remaining participants identified as male (n=18) and nonbinary (n=10). Two participants did not list their gender identity. The average age was 23.5 (range 18-40) years.
At the start of the trial, participants self-reported having severe anxiety (mean 16.09, SD 3.03) alongside high scores of emotion dysregulation (mean 55.95, SD 12.76) and lower levels of beliefs that one can control their ER abilities (mean 11.24, SD 3.21). Between genders, symptoms of anxiety were higher among females (mean 18.0, SD 1.75) when compared to male (mean 14.3, SD 2.56) and nonbinary (mean 14.2, SD 2.90) counterparts.
Feasibility
In total, 300 students responded to the study call. As there was a limited number of Purrble units, 80 of the 142 students who met all inclusion criteria were invited to take part based on the additional criteria. In total, 78 participants provided valid consent forms and completed the baseline assessment; therefore, the enrollment rate was 97.5%. Of the 78 students enrolled, 57 responded to the midpoint assessment (week 4), which decreased to 52 respondents at postdeployment (week 8). Therefore, the overall retention rate across the 7-week deployment including follow-up was 66.7%. We do not see differences in baseline GAD-7 scores between those who dropped out (mean 15.9, SD 3.24; n=25) and those who remained in the study (mean 16.2, SD 2.94; n=51). However, there is some indication that female participants were less likely to drop out (6 out of 28) than male participants (13 out of 32) and nonbinary participants (3 out of 10); it is possible that these differences could be linked to the lower baseline levels of anxiety for male and nonbinary participants.
Across the 7-week deployment, self-reported engagement was measured at the midpoint (week 4) and postdeployment (week 8) through TWEETS. At both time points, students reported a moderate level of engagement with the intervention, with a limited difference from week 4 (mean 23.75, SD 5.09) to week 8 (mean 21.79, SD 6.79). Students who responded to these surveys maintained a good degree of engagement with Purrble throughout the deployment.
Purrble was indicated as a feasible intervention, as generally students reported it to have had a positive impact on their mental health. Of those students who remained in the study, 61% (n=32) reported that Purrble had helped their mental health, while only 9% (n=5) disagreed with this statement. Similarly, 93% (n=48) stated that they “strongly disagreed” or “disagreed” with the statement that Purrble had hurt their mental health, and the remaining 7% (n=4) answered neutrally.
Acceptability
Overview
To determine Purrble acceptability as a tool in students’ daily lives, the qualitative responses from the midpoint and posttrial surveys were explored considering the engagement and appropriation of the intervention. Four themes were developed: (1) primary uses for Purrble, (2) appropriation mechanisms, (3) Purrble is “just” a toy, and (4) empathetic responses (
). The themes identified are described later with example quotes. Additional illustrative quotes by subtheme can be found in .Theme and subtheme | Values, n (%) | |
Primary uses for Purrble | ||
Tool for anxiety and stress reduction | 66 (60.6) | |
Calming during night routine | 27 (24.8) | |
Prevention of mental health spiral | 6 (5.5) | |
Appropriation mechanisms | ||
Grounding | 25 (22.9) | |
Mindfulness | 13 (11.9) | |
Self-stimulation | 30 (27.5) | |
Purrble is “just” a toy | ||
Feelings of embarrassment | 11 (10.1) | |
“If levels are too high” | 15 (13.8) | |
Purrble-related anxiety | 21 (19.3) | |
Empathetic response | ||
Caring for something else helps one’s ability to self-soothe | 32 (29.4) | |
Guilt relating to “upsetting” Purrble | 8 (7.3) |
Primary Uses for Purrble
The most frequently reported use for Purrble by students was as a tool for anxiety and stress reduction. The students described using Purrble as a means to cope with anxiety or anxiety-inducing situations, with some indicating specific types of anxiety responses that Purrble was particularly useful for; for example, “Purrble is useful when I have a quality of anxiety that is panicky—by which I mean, heart palpitations and shakes, where my physiological response is in itself problematic” (R_339Y). Purrble was also often incorporated into a calming night routine, either acting as a device to replace negative sleep hygiene practices (“something to focus on that is not a screen, before I go to bed” [R_OveS]) or as a comforting object while sleeping. For a few people, Purrble was used to prevent a mental health spiral, supporting students in stopping their experiences of poor mental health from becoming more severe and difficult to cope with. Purrble was described as being particularly useful to act as a tool to “snap out of the spiraling nature of invasive thoughts” (R_2EGJ).
Together, these descriptions show students’ perceptions of Purrble as an acceptable and effective intervention for anxiety and stress as well as highlight it as a potential mechanism to break the ruminative aspect of mental health difficulties.
Appropriation Mechanism
The students also described a number of mechanisms that led to the positive effects stated earlier. Most commonly, students reported using Purrble as a tangible grounding tool—something that allowed them to center themselves when experiencing specific stressors and thus reduce their anxiety: “when I'm dissociating, I use the Purrble to ground myself” (R_1jfX). Purrble was also used alongside mindfulness processes and acted as a physical reminder and a practical tool to bring them to that mindfulness headspace. Finally, Purrble also seemed to support self-stimulation when anxious, allowing students to relax or calm down through touching or stroking the device: “The action of stroking him rhythmically is very calming, and takes the focus away from the source of panic/stress” (R_25vd).
All of these strategies can be linked to specific ER processes (attention deployment, cognitive change, and response modulation), suggesting that Purrble could have the intended impact of supporting in situ ER.
Purrble Is “Just” a Toy
While most students felt Purrble was helpful to them, some also had reservations about situations where Purrble could be used. This was commonly associated with feelings of embarrassment due to Purrble being perceived as designed for children and therefore was not always thought to be “appropriate” for adult use in public due to assumed judgment or stigma from others.
A small subset of students also felt that Purrble was not an acceptable option if their levels of distress “are too high,” either as it would not help (“knowing that it is merely a substitute and not an actual living being” [R_3Rz4]) or because it would be too difficult to engage (“it is hard to make myself reach out for him” [R_10ZQ]). Finally, for a few students, there were moments when Purrble interaction could heighten the student’s sensory perceptions, leading to Purrble-related anxiety (“the rare/occasional times where Purrble's heartbeat/sounds trigger my overstimulation/anxiety” [R_28Z2]).
Empathetic Response
Across students, Purrble appeared to induce empathetic responses: the participants sought to comfort Purrble when it was “distressed.” The “need” and ability to care for something else helped self-soothing: it allowed the students to detach themselves from their distress and promoted a focus on Purrble, which encouraged students to relax. “It made me feel quite parental and allowed me to get out of my own head for a minute” (R_2Cba). However, the same empathic response sometimes induced feelings of guilt relating to “upsetting” Purrble by waking it to start an interaction.
Exploring Associations of Purrble Deployment and Mental Health Measures
Findings indicate consistent and statistically significant reductions in all 3 outcome measures when comparing baseline with midterm (week 3) and end-of-term (week 7) observations (
and ). Additionally, these improvements over time are consistent across all models regardless of whether correlates such as TWEETS or GAD-7 baseline are factored in.First, there were large reductions of anxiety (dz=0.96, 95% CI 0.62-1.29 and dav=1.22, 95% CI 0.85-1.57) with post–week 7 anxiety showing an 83% chance of being lower than baseline. Second, the results show medium changes to difficulties with ER (dz=0.69, 95% CI 0.38-0.99 and dav=0.56, 95% CI 0.26-0.85) with postdeployment scores showing a 75% chance of being lower than baseline. Finally, the outcomes showed medium changes to ER beliefs (dz=–0.56, 95% CI –0.86 to –0.26 and dav=–0.51, 95% CI –0.80 to –0.21) with postdeployment showing a 71% chance of being lower than baseline difficulties with ER.
Midpoint engagement score (TWEETS) was not significantly associated with a reduction in anxiety or difficulties with ER scores but was associated with ER beliefs outcomes (with higher TWEETs at baseline leading to a stronger increase in ER beliefs). In contrast, higher baseline anxiety was associated with a lower reduction in difficulties in ER but not with changes in ER beliefs.
Dependent variable: generalized anxiety disorder | |||
Mid–time point | –4.18a (0.63) | ||
Post–time point | –5.38a (0.67) | ||
TWEETSb: mid–time point | –0.10 (0.09) | ||
Gender | |||
Nonbinary | –0.45 (1.48) | ||
Not listed | 3.44 (1.76) | ||
Female | 2.22c (1.06) | ||
Constant | 17.64a (2.44) | ||
Observations | 159 | ||
Log likelihood | –441.71 | ||
Akaike information criterion | 901.41 | ||
Bayesian information criterion | 929.03 |
aP<.001.
bTWEETS: Twente Engagement With eHealth Technologies Scale.
cP<.05.
Dependent variable | ||
Difficulties in Emotion Regulation | Beliefs About Emotion | |
Mid–time point | –3.42a (1.32) | 1.31b (0.36) |
Post–time point | –7.56b (1.41) | 1.59b (0.38) |
TWEETSc: mid–time point | –0.51 (0.30) | 0.235b (0.07) |
GAD-7d: baseline | 1.69a (0.52) | –0.05 (0.12) |
Constant | 41.39b (10.95) | 6.03e (2.59) |
Observations | 159 | 159 |
Log likelihood | –584.41 | –375.42 |
Akaike information criterion | 1182.81 | 764.83 |
Bayesian information criterion | 1204.29 | 786.32 |
aP<.01.
bP<.001.
cTWEETS: Twente Engagement With eHealth Technologies Scale.
dGAD-7: Generalized Anxiety Disorder-7.
eP<.05.
Discussion
Principal Findings
This study aimed to determine whether a socially assistive robot, Purrble, would be a feasible and acceptable intervention for highly anxious university students. The overall findings support the feasibility of Purrble and indicate that Purrble was acceptable as an intervention within this sample, offering insights into how students engaged and appropriated Purrble to suit their individual needs. Furthermore, the explorative analysis showed positive within-subject associations of Purrble deployment with reductions in students’ self-reported anxiety symptoms and with the perceived improvement of ER.
Around one-third of the student sample disengaged by week 4 of this study, but those who were retained used Purrble consistently across the trial period, a finding consistent across questionnaires, open-text responses, and qualitative interview data [
]. The overall number of retained participants was comparable to other studies [ - ]. Interestingly, Purrble maintained a stable level of engagement among those retained—as measured by TWEETS score—throughout the deployment with only a minor (and statistically nonsignificant) decrease in use over the 7-week deployment. This is surprising as prior research shows that engagement with digital interventions tends to drop off markedly over time [ , , ], and ongoing engagement is a key challenge in traditional therapeutic approaches [ ]. For example, prior work has reported decreasing rates of app use each week [ ], decreasing rates of program adherence over time [ ], and only 10.1% of youths completing all full 10 cognitive behavioral therapy sessions [ ]. The consistency of Purrble engagement is particularly striking given the lack of any therapist engagement (eg, clinician’s calls) at any point during the intervention that would often be deployed to encourage sustained engagement [ ]. This supports emerging arguments about the benefits of interventions that draw on extensive user-centered design approaches and focus on addressing the day-to-day needs of target populations [ , , ]. In summary, both quantitative and qualitative data suggest that Purrble is an acceptable and feasible intervention among students and can sustain a stable-level engagement across a 7-week period with acceptable dropouts while retaining a perceived benefit for those who use it.Our exploratory analysis also shows large within-subject effects on student anxiety in week 7 (d=0.96), which are akin to results from other recent web-based open trials [
, - ]. Moreover, the results also show medium effect sizes on the 2 ER constructs: beliefs about own ER self-efficacy (dz=–0.56) and difficulties with ER (dz=–0.69). This provides support for the theory-of-change underpinning Purrble, assuming that repeated in-the-moment downregulation experiences can both improve perceived self-efficacy and mindsets about ER as well as help scaffold more constructive ER practices [ ]. We note that what is particularly interesting in this work is that we have observed both changes in the proximal ER constructs and in anxiety even though Purrble intervention does not explicitly contain any cognitive components (eg, cognitive reappraisal or exposure hierarchies), which underpin traditional ER interventions for mood disorders (eg, Unified Protocol [ , ] or ER therapy [ , ]). This suggests that the effects emerging from in-the-moment experiences of Purrble use could be complementary to the traditionally more cognitively oriented interventions with the potential for mutual amplification (eg, where experiential effects of Purrble further support cognitive training and vice versa).Implications
Given the promising results found from this trial, implications for wider research are considered. First, given the large effect sizes seen in this open trial, it would be prudent to examine Purrble alongside a control treatment within a randomized controlled trial to determine whether Purrble is indeed an effective intervention.
Second, if effective, Purrble deployments could support professional clinical services at universities. One example of this would be Purrble as a complementary resource for those on waitlists. Given that waitlists are long in many countries: in the United Kingdom, for example, 54% of students reported waiting over 3 months to start treatment, and a further 12% reported waiting over a year [
]; additional support for those on waitlists is needed to aid well-being, help-seeking, and perceptions of care [ ].Third, Purrble could support emergent ER skills taught in traditional therapies, such as dialectical behavior therapy [
], serving as an additional support tool provided while on a waitlist, during or between therapeutic sessions, and potentially being of use to maintain skills once therapy is concluded. In particular, dialectical behavior therapy focuses on teaching skills, such as mindfulness, distress tolerance (distraction, reframing the moment, self-soothing, and problem-solving), interpersonal effectiveness, and ER, and seems aligned with the grounding and mindful interaction promoting effects of Purrble as reported by the students in this sample [ ].Limitations
The following limitations of this study need to be considered. First, as an exploratory open trial, the size of the effects could have been affected by a range of time effects. For example, we would expect that the alignment of the study with term time could also lead to a reduction in anxiety scores, for example, due to a reduction of examination-induced anxiety at the end of the term. Additionally, we might see potential reduction-to-mean effects, as the sample was selected on the elevated need of anxiety symptoms at the start of term. Finally, it is possible that the data were also affected by COVID-19-driven cohort effects: at the time the study was conducted, the United Kingdom had recently ended a national lockdown due to COVID-19, which had detrimental effects on young people’s—and particularly students’—mental well-being [
]. It is likely that the students were at risk of heightened stress during the study, which might have further amplified the observed effects and perceived need.Second, the study has not fully explored any gender effects in the acceptability or feasibility of Purrble. Initially, a large number of female students responded to the intervention advert, which prompted the selection of nonfemale responders and clinical need inclusion criteria. The initial disproportionate response from females may reflect a greater receptivity to the intervention concept, or potentially, this reflects a higher incidence of emotional distress, which is often presented as anxiety. For example, females (38%) present more commonly with clinically significant anxiety than males (20.3%) [
]. Alternatively, female participants are often overrepresented in psychological research [ ]. As this trial was to assess the feasibility and acceptability of Purrble, a generalization sample across gender identities was selected. Alternatively, in future trials, there may be a benefit to maintaining the clinical threshold but removing gender limits.Finally, additional mental health difficulties (eg, depression) or life factors (eg, grief) were not measured. Given that mental health difficulties are often comorbid, for example, 27.8% of a recent student sample reported both anxiety and depression [
], capturing such experiences is vital to understanding the impact and effectiveness of Purrble. Thus, it is not currently possible to examine in greater detail whether Purrble was useful for students dealing with these difficulties, which is discussed within the thematic analysis. It is also unclear whether the ER strategies induced by Purrble translate to these difficulties.Conclusions
Purrble appears to represent a feasible in situ intervention to aid ER within a university student population. In particular, Purrble was seen to be an acceptable tangible device that was useful in stressful and anxious moments, where students could downregulate their emotions. Moreover, its use was associated with a decrease in anxiety symptoms and an increase in ER competence among anxious college students. From this study, there are several research implications, such as exploring the impact of Purrble compared to a control group to better determine how Purrble is effective as an ER intervention and to what extent within highly anxious student populations. By using a longitudinal study design for this follow-up study, better evidence would be provided regarding the true impact of the intervention compared to this study, which aligned with the end of term time.
Conflicts of Interest
PS has been involved in the development of what is now Purrble as part of his postdoctoral fellowship and serves as a paid research adviser to the Committee for Children (CfC) but has no financial stake in either CfC (Purrble brand owners) or Sproutel (company manufacturing Purrble). CfC has donated the Purrble units used in the study, and Sproutel has not been involved. Neither Committee for Children nor Sproutel had access to the data or had been part of the data collection in any way nor did they approve or see the publication before it was submitted.
Optional daily survey questions.
DOCX File , 15 KBIllustrative quotes.
DOCX File , 16 KBReferences
- Auerbach RP, Alonso J, Axinn WG, Cuijpers P, Ebert DD, Green JG, et al. Mental disorders among college students in the World Health Organization world mental health surveys. Psychol Med. 2016;46(14):2955-2970. [FREE Full text] [CrossRef] [Medline]
- Jenkins PE, Ducker I, Gooding R, James M, Rutter-Eley E. Anxiety and depression in a sample of UK college students: a study of prevalence, comorbidity, and quality of life. J Am Coll Health. 2021;69(8):813-819. [CrossRef] [Medline]
- Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, et al. WHO World Mental Health Surveys International College Student Project: prevalence and distribution of mental disorders. J Abnorm Psychol. 2018;127(7):623-638. [FREE Full text] [CrossRef] [Medline]
- Blanco C, Okuda M, Wright C, Hasin DS, Grant BF, Liu SM, et al. Mental health of college students and their non-college-attending peers: results from the National Epidemiologic Study on alcohol and related conditions. Arch Gen Psychiatry. 2008;65(12):1429-1437. [FREE Full text] [CrossRef] [Medline]
- Lipson SK, Lattie EG, Eisenberg D. Increased rates of mental health service utilization by U.S. college students: 10-year population-level trends (2007-2017). Psychiatr Serv. 2019;70(1):60-63. [FREE Full text] [CrossRef] [Medline]
- Huang HCH, Ougrin D. Impact of the COVID-19 pandemic on child and adolescent mental health services. BJPsych Open. 2021;7(5):e145. [FREE Full text] [CrossRef] [Medline]
- Shaw H, Robertson S, Ranceva N. What was the impact of a global pandemic (COVID-19) lockdown period on experiences within an eating disorder service? A service evaluation of the views of patients, parents/carers and staff. J Eat Disord. 2021;9(1):14. [FREE Full text] [CrossRef] [Medline]
- Coronavirus: impact on young people with mental health needs. Young Minds. 2020. URL: https://healtheducationresources.unesco.org/library/documents/coronavirus-impact-young-people-mental-health-needs [accessed 2023-09-26]
- Duffy ME, Twenge JM, Joiner TE. Trends in mood and anxiety symptoms and suicide-related outcomes among U.S. undergraduates, 2007-2018: evidence from two national surveys. J Adolesc Health. 2019;65(5):590-598. [CrossRef] [Medline]
- King N, Pickett W, McNevin SH, Bowie CR, Rivera D, Keown-Stoneman C, et al. Mental health need of students at entry to university: baseline findings from the U-Flourish Student Well-Being and Academic Success Study. Early Interv Psychiatry. 2021;15(2):286-295. [CrossRef] [Medline]
- Xiao H, Carney DM, Youn SJ, Janis RA, Castonguay LG, Hayes JA, et al. Are we in crisis? National mental health and treatment trends in college counseling centers. Psychol Serv. 2017;14(4):407-415. [CrossRef] [Medline]
- Vidourek RA, King KA, Nabors LA, Merianos AL. Students' benefits and barriers to mental health help-seeking. Health Psychol Behav Med. 2014;2(1):1009-1022. [FREE Full text] [CrossRef] [Medline]
- Dunbar MS, Sontag-Padilla L, Kase CA, Seelam R, Stein BD. Unmet mental health treatment need and attitudes toward online mental health services among community college students. Psychiatr Serv. 2018;69(5):597-600. [FREE Full text] [CrossRef] [Medline]
- Harrer M, Adam SH, Baumeister H, Cuijpers P, Karyotaki E, Auerbach RP, et al. Internet interventions for mental health in university students: a systematic review and meta-analysis. Int J Methods Psychiatr Res. 2019;28(2):e1759. [FREE Full text] [CrossRef] [Medline]
- Pauley D, Cuijpers P, Papola D, Miguel C, Karyotaki E. Two decades of digital interventions for anxiety disorders: a systematic review and meta-analysis of treatment effectiveness. Psychol Med. 2023;53(2):567-579. [FREE Full text] [CrossRef] [Medline]
- Harrer M, Apolinário-Hagen J, Fritsche L, Salewski C, Zarski AC, Lehr D, et al. Effect of an internet- and app-based stress intervention compared to online psychoeducation in university students with depressive symptoms: results of a randomized controlled trial. Internet Interv. 2021;24:100374. [FREE Full text] [CrossRef] [Medline]
- Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ. 2015;351:h5627. [FREE Full text] [CrossRef] [Medline]
- Mohr DC, Weingardt KR, Reddy M, Schueller SM. Three problems with current digital mental health research . . . and three things we can do about them. Psychiatr Serv. 2017;68(5):427-429. [FREE Full text] [CrossRef] [Medline]
- Kaltenthaler E, Parry G, Beverley C, Ferriter M. Computerised cognitive-behavioural therapy for depression: systematic review. Br J Psychiatry. 2008;193(3):181-184. [FREE Full text] [CrossRef] [Medline]
- Lillevoll KR, Vangberg HCB, Griffiths KM, Waterloo K, Eisemann MR. Uptake and adherence of a self-directed internet-based mental health intervention with tailored e-mail reminders in senior high schools in Norway. BMC Psychiatry. 2014;14:14. [FREE Full text] [CrossRef] [Medline]
- Whiteside U, Lungu A, Richards J, Simon GE, Clingan S, Siler J, et al. Designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. J Med Internet Res. 2014;16(2):e42. [FREE Full text] [CrossRef] [Medline]
- Woodford J, Farrand P, Bessant M, Williams C. Recruitment into a guided Internet based CBT (iCBT) intervention for depression: lesson learnt from the failure of a prevalence recruitment strategy. Contemp Clin Trials. 2011;32(5):641-648. [CrossRef] [Medline]
- Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health. 2018;21(3):116-119. [FREE Full text] [CrossRef] [Medline]
- Slovák P, Theofanopoulou N, Cecchet A, Cottrell P, Bertran FA, Dagan E, et al. I just let him cry...: designing socio-technical interventions in families to prevent mental health disorders. Proc ACM Hum-Comput Interact. 2018;2(CSCW):1-34. [CrossRef]
- Gross JJ. Emotion regulation: current status and future prospects. Psychol Inq. 2015;26(1):1-26. [CrossRef]
- Sloan E, Hall K, Moulding R, Bryce S, Mildred H, Staiger PK. Emotion regulation as a transdiagnostic treatment construct across anxiety, depression, substance, eating and borderline personality disorders: a systematic review. Clin Psychol Rev. 2017;57:141-163. [FREE Full text] [CrossRef] [Medline]
- Fernandez KC, Jazaieri H, Gross JJ. Emotion regulation: a transdiagnostic perspective on a new RDoC domain. Cognit Ther Res. 2016;40(3):426-440. [FREE Full text] [CrossRef] [Medline]
- Trompeter N, Bussey K, Forbes MK, Mitchison D. Emotion dysregulation within the CBT-E model of eating disorders: a narrative review. Cogn Ther Res. 2021;45(6):1021-1036. [FREE Full text] [CrossRef]
- Young KS, Sandman CF, Craske MG. Positive and negative emotion regulation in adolescence: links to anxiety and depression. Brain Sci. 2019;9(4):76. [FREE Full text] [CrossRef] [Medline]
- Brausch AM, Clapham RB, Littlefield AK. Identifying specific emotion regulation deficits that associate with nonsuicidal self-injury and suicide ideation in adolescents. J Youth Adolesc. 2022;51(3):556-569. [FREE Full text] [CrossRef] [Medline]
- Squires LR, Hollett KB, Hesson J, Harris N. Psychological distress, emotion dysregulation, and coping behaviour: a theoretical perspective of problematic smartphone use. Int J Ment Health Addiction. 2020;19(4):1284-1299. [CrossRef]
- Kapatais A, Williams AJ, Townsend E. The mediating role of emotion regulation on self-harm among gender identity and sexual orientation minority (LGBTQ+) individuals. Arch Suicide Res. 2023;27(2):165-178. [CrossRef] [Medline]
- Dobos B, Piko BF, Mellor D. What makes university students perfectionists? The role of childhood trauma, emotional dysregulation, academic anxiety, and social support. Scand J Psychol. 2021;62(3):443-447. [FREE Full text] [CrossRef] [Medline]
- Mülder LM, Deci N, Werner AM, Reichel JL, Tibubos AN, Heller S, et al. Antecedents and moderation effects of maladaptive coping behaviors among German university students. Front Psychol. 2021;12:645087. [FREE Full text] [CrossRef] [Medline]
- Slovak P, Antle A, Theofanopoulou N, Roquet CD, Gross J, Isbister K. Designing for emotion regulation interventions: an agenda for HCI theory and research. ACM Trans Comput-Hum Interact. 2023;30(1):1-51. [CrossRef]
- Slovak P, Ford BQ, Widen S, Roquet CD, Theofanopoulou N, Gross JJ, et al. An in situ, child-led intervention to promote emotion regulation competence in middle childhood: protocol for an exploratory randomized controlled trial. JMIR Res Protoc. 2021;10(11):e28914. [FREE Full text] [CrossRef] [Medline]
- Theofanopoulou N, Slovak P. Exploring technology-mediated parental socialisation of emotion: leveraging an embodied, in-situ intervention for child emotion regulation. Presented at: CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems; April 29-May 5, 2022, 2022;1-16; New Orleans, LA, USA. URL: https://dl.acm.org/doi/proceedings/10.1145/3491102 [CrossRef]
- Theofanopoulou N, Isbister K, Edbrooke-Childs J, Slovák P. A smart toy intervention to promote emotion regulation in middle childhood: feasibility study. JMIR Ment Health. 2019;6(8):e14029. [FREE Full text] [CrossRef] [Medline]
- Gross JJ. The extended process model of emotion regulation: elaborations, applications, and future directions. Psychol Inq. 2015;26(1):130-137. [CrossRef]
- Kelders SM, Kip H, Greeff J. Psychometric evaluation of the Twente Engagement with Ehealth Technologies Scale (TWEETS): evaluation study. J Med Internet Res. 2020;22(10):e17757. [FREE Full text] [CrossRef] [Medline]
- Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097. [FREE Full text] [CrossRef] [Medline]
- Victor SE, Klonsky ED. Validation of a brief version of the Difficulties in Emotion Regulation Scale (DERS-18) in five samples. J Psychopathol Behav Assess. 2016;38(4):582-589. [CrossRef]
- Tamir M, John OP, Srivastava S, Gross JJ. Implicit theories of emotion: affective and social outcomes across a major life transition. J Pers Soc Psychol. 2007;92(4):731-744. [CrossRef] [Medline]
- De Castella K, Goldin P, Jazaieri H, Ziv M, Dweck CS, Gross JJ. Beliefs about emotion: links to emotion regulation, well-being, and psychological distress. Basic Appl Soc Psychol. 2013;35(6):497-505. [CrossRef]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101. [CrossRef] [Medline]
- Braun V, Clarke V. Reflecting on reflexive thematic analysis. Qual Res Sport Exerc Health. 2019;11(4):589-597. [CrossRef]
- Roquet CD, Theofanopoulou N, Freeman JL, Schleider J, Gross JJ, Davis K, et al. Exploring situated & embodied support for youth's mental health: design opportunities for interactive tangible device. Presented at: CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems; April 29-May 5, 2022, 2022;1-16; New Orleans, LA, USA. URL: https://dl.acm.org/doi/proceedings/10.1145/3491102 [CrossRef]
- Schleider JL, Dobias ML, Sung JY, Mullarkey MC. Future directions in single-session youth mental health interventions. J Clin Child Adolesc Psychol. 2020;49(2):264-278. [FREE Full text] [CrossRef] [Medline]
- Bates D, Kliegl R, Vasishth S, Baayen H. Parsimonious mixed models. ArXiv. Preprint posted online on May 26, 2018. [FREE Full text] [CrossRef]
- McCloud T, Jones R, Lewis G, Bell V, Tsakanikos E. Effectiveness of a mobile app intervention for anxiety and depression symptoms in university students: randomized controlled trial. JMIR Mhealth Uhealth. 2020;8(7):e15418. [FREE Full text] [CrossRef] [Medline]
- Rith-Najarian LR, Chorpita BF, Gong-Guy E, Hammons HR, Chavira DA. Feasibility of a web-based program for universal prevention of anxiety and depression in university students: an open trial. J Am Coll Health. 2022;70(8):2519-2526. [CrossRef] [Medline]
- Bantjes J, Kazdin AE, Cuijpers P, Breet E, Dunn-Coetzee M, Davids C, et al. A web-based group cognitive behavioral therapy intervention for symptoms of anxiety and depression among university students: open-label, pragmatic trial. JMIR Ment Health. 2021;8(5):e27400. [FREE Full text] [CrossRef] [Medline]
- Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. 2019;21(9):e14567. [FREE Full text] [CrossRef] [Medline]
- Ng MM, Firth J, Minen M, Torous J. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv. 2019;70(7):538-544. [FREE Full text] [CrossRef] [Medline]
- Schleider JL, Weisz JR. Little treatments, promising effects? Meta-analysis of single-session interventions for youth psychiatric problems. J Am Acad Child Adolesc Psychiatry. 2017;56(2):107-115. [CrossRef] [Medline]
- Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K, et al. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res. 2017;19(1):e10. [FREE Full text] [CrossRef] [Medline]
- Mohr DC, Riper H, Schueller SM. A solution-focused research approach to achieve an implementable revolution in digital mental health. JAMA Psychiatry. 2018;75(2):113-114. [CrossRef] [Medline]
- Graham AK, Wildes JE, Reddy M, Munson SA, Taylor CB, Mohr DC. User-centered design for technology-enabled services for eating disorders. Int J Eat Disord. 2019;52(10):1095-1107. [FREE Full text] [CrossRef] [Medline]
- Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res. 2019;21(7):e12869. [FREE Full text] [CrossRef] [Medline]
- March S, Spence SH, Donovan CL, Kenardy JA. Large-scale dissemination of internet-based cognitive behavioral therapy for youth anxiety: feasibility and acceptability study. J Med Internet Res. 2018;20(7):e234. [FREE Full text] [CrossRef] [Medline]
- Palacios JE, Richards D, Palmer R, Coudray C, Hofmann SG, Palmieri PA, et al. Supported internet-delivered cognitive behavioral therapy programs for depression, anxiety, and stress in university students: open, non-randomised trial of acceptability, effectiveness, and satisfaction. JMIR Ment Health. 2018;5(4):e11467. [FREE Full text] [CrossRef] [Medline]
- Ellard KK, Fairholme CP, Boisseau CL, Farchione TJ, Barlow DH. Unified protocol for the transdiagnostic treatment of emotional disorders: protocol development and initial outcome data. Cogn Behav Pract. 2010;17(1):88-101. [CrossRef] [Medline]
- Ehrenreich-May J, Kennedy SM, Sherman JA, Bilek EL, Buzzella BA, Bennett SM, et al. Unified Protocols for Transdiagnostic Treatment of Emotional Disorders in Children and Adolescents: Therapist Guide. Oxford. Oxford University Press; 2018.
- Mennin DS, Fresco DM. Emotion regulation therapy. In: Handbook of Emotion Regulation. New York. The Guilford Press; 2014;469-490.
- Renna ME, Quintero JM, Soffer A, Pino M, Ader L, Fresco DM, et al. A pilot study of emotion regulation therapy for generalized anxiety and depression: findings from a diverse sample of young adults. Behav Ther. 2018;49(3):403-418. [FREE Full text] [CrossRef] [Medline]
- Docherty M, Thornicroft G. Specialist mental health services in England in 2014: overview of funding, access and levels of care. Int J Ment Health Syst. 2015;9:34. [FREE Full text] [CrossRef] [Medline]
- Grünzig SD, Baumeister H, Bengel J, Ebert D, Krämer L. Correction to: effectiveness and acceptance of a web-based depression intervention during waiting time for outpatient psychotherapy: study protocol for a randomized controlled trial. Trials. 2018;19(1):393. [FREE Full text] [CrossRef] [Medline]
- Harvey LJ, Hunt C, White FA. Dialectical behaviour therapy for emotion regulation difficulties: a systematic review. Behav change. 2019;36(3):143-164. [FREE Full text] [CrossRef]
- Dickinson ER, Adelson JL, Owen J. Gender balance, representativeness, and statistical power in sexuality research using undergraduate student samples. Arch Sex Behav. 2012;41(2):325-327. [FREE Full text] [CrossRef] [Medline]
Abbreviations
DERS-18: Difficulties in Emotion Regulation Scale-18 |
ER: emotion regulation |
GAD-7: Generalized Anxiety Disorder-7 |
TWEETS: Twente Engagement With eHealth Technologies Scale |
Edited by J Torous; submitted 27.02.23; peer-reviewed by D Joshi, A Kazdin, C Laranjeira, N Hung; comments to author 01.04.23; revised version received 14.07.23; accepted 18.07.23; published 31.10.23.
Copyright©A Jess Williams, Maureen Freed, Nikki Theofanopoulou, Claudia Daudén Roquet, Predrag Klasnja, James Gross, Jessica Schleider, Petr Slovak. Originally published in JMIR Mental Health (https://mental.jmir.org), 31.10.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.