Digital health interventions (DHIs) have repeatedly been shown to be effective in improving mental well-being and relieving the symptoms of depression [1-4] and anxiety [1-3,5,6] in young people. They are efficacious in young people exhibiting elevated symptoms of anxiety and depression as well as in those who have been diagnosed with an anxiety or depressive disorder [2]. Randomized controlled trials (RCTs) have also revealed that DHIs can improve mood and promote emotional resilience in adolescents in the general populations [7,8].

DHIs may have other potential benefits for adolescents and young adults. For example, even when barriers to health systems are low (eg, health insurance coverage or free health care services) and individuals report elevated depression or anxiety symptoms, the rates of engagement with mental health support are low [9,10]. This suggests that there may be further obstacles, including attitudinal barriers, such as a preference for self-reliance and the desire to avoid perceived stigmatization [9-12], and off-putting aspects related to the delivery of the service, such as physical access issues [11], in addition to long waitlist times [12,13]. DHIs may help address these barriers, as research has shown that young people are drawn to digital health solutions [14,15], especially those on mobile phones [16].

DHIs delivered via mobile phones present a promising route for supporting the mental health of young people. For example, the ownership of mobile phones has grown enormously in recent years, with a national US survey finding that in 2019, approximately 90% of 16- to 18-year-olds possessed a smartphone, compared with 75% in 2015 [17]. There is also a high motivation to use mobile phone apps to improve mental health [18], which may stem from not only their convenience but also their control in terms of when and how much an individual chooses to engage with a DHI.

DHIs delivered by mobile phones typically take the form of “unguided” apps. Although numerous systematic reviews have found that DHIs for young people were effective when they incorporated some sort of human support [4,6,9,19], such as professional feedback or a live link to a professional, their superiority over unguided DHI approaches [20] has not been confirmed conclusively. For instance, in some studies, the effect in favor of guided DHIs was small [20] or not statistically significant when only studies with a low risk of bias were considered [19]. For smartphone-based interventions for depressive symptoms, a recent meta-analysis of RCTs by Firth et al [21] found that only apps without professional feedback produced moderate positive effects. Another meta-analysis found larger effect sizes on stress levels and quality of life when apps provided professional guidance [22].

Despite the convenience, appeal, and promise of DHIs delivered by mobile phones, evidence for their efficacy remains unsatisfactory. For example, review studies can neglect the modality of a DHI, which is important when the current bulk of studies involve comparisons of guided internet-based or computer-based cognitive behavioral therapy (CBT) with inactive controls [2,23]. Furthermore, in a recent review of 19 studies that investigated the use of serious games and virtual reality for the treatment of common mental health problems in children and young people, 10 (53%) had no comparison group, 3 (16%) used a waitlist condition, 4 (21%) used another DHI control group condition, and only 2 (11%; investigating the same DHI) used face-to-face interventions [3,4]. The effect of DHIs was largest when DHIs were compared with nonactive control conditions and was small or even nonexistent when there was an active comparator [4,22,23]. Recent meta-analyses that have specifically explored studies involving mobile phone apps for depression and anxiety have suggested that they have the potential to reduce symptoms when compared with inactive controls [21,24], and a recent meta-review by Goldberg et al [25] supports these findings. The authors noted that there are few studies involving active controls, which is important for establishing the effect of using a particular intervention.

A further limitation affecting recent studies on the efficacy of DHIs on mental health in young people (including DHIs delivered via mobile phones) was highlighted by Grist et al [4] in their meta-analysis of the efficacy of DHIs for depression and anxiety in children and adolescents. The authors found that many of the studies were underpowered. Insufficient sample sizes can promote favorable effects of particular interventions [1,2,5], reducing the likelihood that a statistically significant result reflects a true effect [26,27].

Aside from evaluation issues, a further issue troubles DHI research. In 2005, Eysenbach [28] noted that a considerable number of study participants stopped engaging with them (ie, nonuse attrition or nonadherence) or dropped out; therefore, they could not be followed up (dropout attrition). For example, Linardon et al [22] found that approximately 25% of participants using smartphone-delivered DHIs for common mental health problems dropped out before short-term follow-up (≤8 weeks) and up to 33% dropped out before long-term follow-ups. Similarly, Hollis [23] found that for DHIs targeting depression in young people, attrition ranged from 0% for a computerized in-house attention bias modification [29] to 42.5% for a synchronous chat intervention [30]. Although this is potentially problematic for evaluating the efficacy of DHIs, the lack of or diminishing engagement is troubling, given the benefits of engaging with such interventions. Developers of non–practitioner-led (unguided) DHIs face an overarching challenge: how to engage clients long enough for interventions to have an impact.

One approach to combating attrition and nonengagement in DHI design is the use of game design elements in nongaming contexts or “gamification” [31]. Although the gamification of software and technology in a behavior change context has gained traction over the past few years [32], it is still in the early stages in terms of methodology, classification, and implementation. In total, 2 relevant research streams have emerged: (1) persuasive systems and technology and (2) gamification as a tool that enables playful experiences and enhances engagement [33]. Persuasive systems are geared toward using software and technology to instigate behavioral change regarding preset goals, such as weight loss, developing beneficial psychological coping mechanisms, or even environmental sustainability actions [34]. Persuasive strategies can be implemented endogenously; software can be developed via game mechanics, which are intrinsically inherent to the gaming experience [35]. Endogenous games are designed to enhance the gaming experience to the highest degree possible; external goals of persuasive systems, where game mechanics rules are coupled with outcome goals, are likely to hinder or complicate the process. This is sometimes referred to as gamifying rather than gamification. The more commonly used gamification development procedure for persuasive games, where elements are layered on an existing framework of information exchange geared toward a predefined goal (such as weight loss), is exogenous [35]. Although it makes sense to implement evidence-based strategies, such as CBT, as the foundation of a persuasive system for a DHI, it can be at odds with the second research stream: playful experiences and enhanced engagement. Poorly implemented exogenous factors, such as badges or rewards that are not connected to the gameplay, have reportedly had the opposite effect on the playing population, leading to the rejection of the tool [36]. An example of an endogenous game would be eQuoo, where the intervention is woven into a story framework that is appealing in and of itself, whereas an exogenous element would be a badge given after a CBT exercise is successfully completed.

In addition to gamification, interventions aimed at building skills associated with improved mental health may help enhance mental health overall. As conceptualizations of mental health expand beyond the pathogenic to the salutogenic [37], researchers have started to investigate mental health in terms of both the capacity to manage difficulties and the absence of illness [38]. Resilience is widely considered to be a protective factor against negative mental health outcomes. Resilience is the capacity to overcome or adapt to adversity and thus stay mentally healthy or regain one’s mental health following significant challenges [39]. Previously thought to reflect the presence of personality characteristics, such as grit, resilience is now more commonly understood as a process in which various modifiable protective factors are drawn upon. This leads to positive outcome trajectories [40]. Protective factors exist at various systemic levels, including psychological (eg, self-efficacy and motivation) and social (eg, supportive peer relationships and a sense of community belonging) [41]. The realization that many individuals experience significant adversity during their lifetime [42] has led to an increasing demand for resilience-building programs aimed at promoting mental health [34].

In summary, research has shown that DHIs can be effective in promoting mental well-being and reducing symptoms of depression and anxiety in young people. However, the evidence base is lacking, and given their unique qualities, there is a need to specifically test the advantages of using DHIs involving gamification and through sufficiently powered 3-armed RCTs (DHIs vs active controls vs nonactive controls) that also report on trial completion and attrition.

Participants had to be aged ≥18 years and enrolled as college or university students. They had to have access to a smartphone or tablet device and an app store (App Store or Google Play) to install the eQuoo app. No further exclusion criteria were applied.

Recruitment took place in 2 waves between March and April 2021 using the UNiDAYS subscriber database, a global student discount platform. To qualify for UNiDAYS, people must provide a valid college or university email address. In the first wave, 9000 people who were then registered on the platform with their University College London affiliation were emailed an invitation to participate in a trial exploring the effects of a mental health app on resilience with a link to the trial landing page; 443 people signed up for the trial and completed the baseline assessment. This figure represented 8.6% of those emailed. On the landing page, the study goal was explained briefly, the study protocol was shared, and a consent box needed to be clicked that read “I have understood the trial terms and I consent.” This checkbox needed to be checked for participants to join the trial; without it, they could not gain access to the questionnaires and information on what to test, thus ensuring consent. In the second wave, 100,000 randomly selected UNiDAYS subscribers were emailed the same invitation; 724 participants completed the baseline assessment. This figure represented 1.6% of those emailed. The trial was completed and closed, and all participants were debriefed in June 2021.

The study (an RCT) was approved by the University College London Ethics Committee (0501/001), and the authors confirm that all ongoing and related trials are registered with the German Clinical Trials Register (DRKS00027638). This study was conducted in accordance with the principles of the Declaration of Helsinki.

An a priori power analysis using G*Power 3.1.9.4 revealed that a sample size of 207 participants was required to detect a within-between interaction for the primary outcome (input parameters: repeated measures [RM]–ANOVA, effect size f=0.10; Cronbach α=.05; power=0.95; number of groups=3; number of measurements=6; RM intercorrelation=0.50; nonsphericity correction=1). However, given the high attrition rates reported in previous research (eg, 40%-60% [28]), we aimed to enroll a minimum of 850 participants.

Participants were reminded via weekly emails to complete the questionnaires at t1 to t5. In addition to the questionnaires, the participants in the eQuoo group were asked a question to prove that they had completed their level, and the participants in the active control group were asked if they had spent 10 minutes on the Sanvello app. Data were collected using LimeSurvey, a widely used secure open-source tool.

Differences in participant characteristics between each intervention arm were first compared using chi-square tests of independence (for categorical variables) and one-way ANOVAs (for continuous variables).

Attrition rates and the number of participants who completed the levels at all time points were compared across groups. Attrition was defined as not completing the assessments past t0. This is consistent with previous studies that defined attrition as the failure to complete the study protocol associated with the intervention [22]. To compare the likelihood of attrition between intervention arms, logistic regression models were constructed, with both unadjusted and adjusted (for sex, age, living situation, and baseline measures) odds ratios (ORs) and 95% CIs reported.

A standardized outcome was then created using the last available measure, even if it was only the participant’s baseline, and this was carried forward (intention to treat) [62]. Initially, paired sample 2-tailed t tests were conducted within each intervention arm to assess statistically significant changes in the primary and secondary outcomes. Pre-post effect sizes (Cohen d) were calculated using 0.2, 0.5, and 0.8, which were used as thresholds to signify small, medium, and large effects, respectively. Differences between intervention arms in end point (t5) scores were explored using linear regression models (for each outcome) with a baseline measure, and age, sex, living situation, and study level (ie, undergraduate or postgraduate) were entered as covariates. Adjusted end point means were estimated, and the magnitude of between-group differences was explored by calculating Cohen d.

To assess differences between interventions over time points, initial analyses used RM-ANOVAs with time (6 levels) entered alongside the intervention group (3 levels). To assess the impact of listwise deletion on these models, further analysis was conducted using mixed effects models exploring changes in the primary and secondary outcomes over time, entering the intervention arm as an independent variable and age, sex, and living situation as covariates. These mixed effects models, using restricted maximum likelihood estimation, used all available data at each time point. The survey was programmed so that all fields were mandatory, that is, participants either filled out the entire questionnaire or did not participate. The data were published using the Open Science Framework [63].

Of the 1167 individuals who were recruited for the study, 2 individuals reported being aged <18 years and were excluded; therefore, the final sample comprised 1165 participants. These individuals were then randomly allocated to the study groups: 389 (33.39%) were placed in the eQuoo group, 384 (32.96%) were placed in the Sanvello group, and 392 (33.65%) were placed in the waitlist group. Figure 1 shows the participant flow diagram and the proportion of participants providing data at each time point. All participants (except 1 individual in the eQuoo group) completed baseline (t0) measures, and end point (t5) measures were available for 251 (64.5%) out of 389 participants in the eQuoo group, 77 (20%) out of 384 participants in the Sanvello group, and 101 (25.8%) out of 392 participants in the waitlist group. Of the eQuoo participants, 349 completed at least one measure after baseline (but potentially not t5), and 123 of the Sanvello participants completed at least one nonbaseline measure, as did 211 of the waitlist group.

The likelihood of attrition (defined as completing only the baseline measures) was compared for the intervention conditions. Although only 10.3% (40/389) of the participants in the eQuoo group met the criteria for attrition, the rates for the Sanvello and waitlist groups were 67.9% (261/384) and 46.2% (181/392), respectively. The odds of attrition were significantly higher in the Sanvello group than in the eQuoo group (OR 18.51, 95% CI 12.52-27.38) as well as in the waitlist group (OR 7.48, 95% CI 5.10-10.97). After adjusting for age, sex, living situation, study level, and all 3 baseline measures, Sanvello was compared with eQuoo (OR 19.27, 95% CI 12.91-28.77), as was the waitlist (OR 7.34, 95% CI 4.96-10.85).

Table 1 presents differences in participant characteristics between intervention arms. The comparative statistics suggest the groups were balanced in terms of age, sex, and baseline resilience scores but not living situation, study level, or initial depression and generalized anxiety symptom scores.

Pre- and posttest scores for resilience are presented in Table 2. There was a significant improvement (medium effect size) in resilience among eQuoo participants (t₃₈₇=18.35; P<.001; Cohen d=0.58) but not among Sanvello or waitlist participants (P=.10 and .82, respectively).

Differences in end point scores were then compared using linear regression models, controlling for baseline resilience score, age, sex, living situation, and study level. Significantly higher resilience scores were observed in the eQuoo condition compared with Sanvello (b=−4.41, 95% CI −5.09 to −3.74; P<.001) and in eQuoo compared with the waitlist condition (b=−4.78, 95% CI −5.46 to −4.10; P<.001).

RM-ANOVA models were then used to compare resilience scores at each time point between the intervention arms, and mixed effects models were constructed to explore the change between the intervention arms. A total of 174 participants (n=78, 44.8% eQuoo; n=37, 21.3% Sanvello; and n=59, 33.9% waitlist) completed the measures at all time points and were included in the RM-ANOVA. A significant effect was observed for time (F_5,855=18.51; P<.001), intervention arm (F_2,171=8.68; P<.001), and intervention interaction (F_10,855=9.69; P<.001). Mixed effects models, including age, sex, living situation, and study level as covariates led to a significant intervention-in-time interaction: scores were significantly lower over time for participants in the Sanvello arm (b=−0.86, 95% CI −1.06 to −0.66; P<.001) and waitlist arm (b=−1.25, 95% CI 1.42 to −1.08; P<.001), compared with the eQuoo arm (Figure 2).

The results suggest that using the gamified mental health app, eQuoo is an effective pathway for improving mental health and resilience. In particular, the use of eQuoo was related to increased resilience scores over time; this was not the case for the nongamified or waitlist groups. Participants using eQuoo also reported lower depression and anxiety scores at the end of the intervention compared with the other 2 groups and were significantly less likely to drop out of treatment. The findings indicated that both apps were effective in reducing anxiety and depression symptoms, although the larger effect sizes associated with the improvements suggest that eQuoo was more effective in this regard.

Mental health issues are pervasive and particularly impactful in student populations [64,65]. Unfortunately, the gap between mental health treatment needs and access to care continues to grow [66,67]. Although DHIs have been identified as meaningful alternatives to face-to-face therapy in student populations [9,11], they have significant shortcomings [68-70] in terms of motivation, interest, and engagement [69]. As the prevalence of mental health issues [69] and the demand for digital mental health solutions, such as telehealth and internet therapy [71] have increased, research investigating digital, app-based interventions is especially timely. This study contributes to the quest for solutions by presenting eQuoo as a gamified DHI that can improve mental health by reducing depression and anxiety symptoms while building resilience and compares favorably with a well-established, nongamified mental health app.

Gamification using mobile mental health apps has been suggested as a way to increase engagement in mental health services [72-74]. This study supports this suggestion by highlighting the benefits of DHIs incorporating gamification and resilience training (particularly when eQuoo was used). Although further research that distinguishes the effects of gamification and includes more diverse populations is necessary, the results of this study suggest that mobile mental health apps can assist in mental health treatment and resilience building.

Furthermore, our findings are particularly relevant, given the current events. Global factors, such as the 2019 COVID-19 pandemic and the accompanying restrictions, have been linked to an increased prevalence of mental health issues and difficulties in accessing support [75]. According to a recent World Health Organization survey, the pandemic has disrupted critical mental health services in 93% of countries worldwide while increasing demand for them [50]. Our findings suggest that DHIs may be a meaningful option in lieu of traditional mental health services for those seeking help.

The finding that resilience scores improved for eQuoo users is also promising, as this suggests that the impact and lingering effects of future adversity on mental health may be less for those with higher scores [74,76], among other benefits [77]. In other words, the inclusion of resilience training elements in DHIs such as eQuoo can facilitate a reduction in psychopathology symptoms such as depression and anxiety but may also serve a protective function against these symptoms for users when encountering difficulties later on. This would be especially important for young adults who experience many stressful transitions in different life domains [78]. Further longitudinal research is needed to explore this empirically, capturing both the long-term impact of eQuoo and the impact of subsequent stressors.

With continued research, DHIs that incorporate gamification and resilience training may play a pivotal role in improving access to mental health services and preventing mental health issues. They could be integrated into standard face-to-face treatment as part of the client’s homework, offered as a pretreatment option, or used as a mental health resource for all college students as part of an intervention and prevention strategy.

This study has several strengths. It is one of only a few follow-up RCTs involving a gamified mental health app [51]. Its large participant pool enables calculations to be performed with a large power size, thus ensuring the reliability of the outcome. This study is based on past research on the efficacy of the same gamified app that has been tested and shown to increase resilience and positive relations with others and decrease anxiety symptoms in a very different sample (employed adults) [51]. Given this (and the similarities in findings between the present and previous studies), the benefits of eQuoo may be generalized to different populations. Finally, the study included 3 treatment arms, including a treatment-as-usual active control condition using a nongamified app that has been tested [59] and shown to be efficacious [59,61]. Including this treatment-as-usual active control condition allowed the gamified app and nongamified app to be usefully compared. In addition, unlike previous studies, we have explicitly reported on dropout and nonuse using engagement data.

This study has some limitations. Although including a treatment-as-usual active control condition was important to evaluate eQuoo, the 2 apps were not structurally equivalent interventions. Ordinarily, structurally equivalent interventions are identical in terms of the number and duration of sessions, settings (group vs individual), level of therapists’ experience, and adaptability of the therapy to the client [79]. The 2 apps used in this study have different purposes and use different interventions. Although the eQuoo is designed to teach psychological skills to decrease depression and anxiety and increase resilience, and many of these skills are based on CBT principles that overlap with the structure of Sanvello, the latter relies exclusively on CBT psychoeducation and exercises [59]. In other words, there were meaningful differences between the 2 apps, in addition to gamification. A meta-analysis comparing in-person psychotherapies found no differences between programs when they had structural equivalence [79]. Thus, it may be that the differences between eQuoo and Sanvello were not solely caused by gamification. In light of this, future studies might compare structurally equivalent gamified and nongamified apps, although a robust comparison would involve a version of the experimental app with gamification features removed, to ensure that there were no other explanatory variables, such as the look or interface of the app.

Another limitation of this study is that it did not record the duration of participants’ use. Hence, it was not possible to examine whether the level of use was linked to attrition, an important phenomenon discussed in eHealth research [28]. In other words, although the attrition among participants using the gamified app was lower than that in the nongamified app and waitlist groups, it was not possible to examine the influence of active engagement. This could have been accomplished by including time-of-use data from the apps.

A further limitation was the lack of follow-up; therefore, we could not draw conclusions about the stability of the observed effects. As such, future researchers should investigate the impact of gamification on nonuse attrition, engagement, long-term attrition, and mental health.

Similarly, it was unclear which aspects of the intervention resulted in the observed effects. As the eQuoo was created as a skills training app, it may have been that the development of resilience and reduction in mental health symptomatology was the result of skills acquisition. However, specific skill acquisition was not examined. Studies in which individual skills were trained were excluded to investigate the mechanisms of the observed effects. Future researchers might, therefore, want to include pre- and postmeasures of trained skills.

The participants in this study were self-selecting; therefore, a self-selection bias may have influenced our findings. In addition, the study was fully reliant on self-report, which may have also affected the results through both response bias [80] and shared variance [81].

Given these sample-specific and method-specific limitations, future researchers might want to investigate the effects of gamification using more diverse samples and additional measures of efficacy, such as sleep duration and quality (objective measures related to depression) and clinical interviews. In addition, the authors are aware that using the traditional “gold standard” of RCTs may not be the best method for evaluating the effectiveness of an app such as eQuoo. Such studies can take years from inception to publication, and developers will have real-time feedback and data that would influence app improvements more quickly than they could be tested [82]. The authors are investigating current research possibilities that would help address this issue and will be implementing them in the coming years while documenting the results for peer review.

This study aimed to investigate the effect of the gamified mobile mental health game eQuoo on levels of resilience, anxiety, depression, and attrition (one of digital mental health’s Achilles heels [83]) in a student population. Compared with the active control and waitlist groups, we found a significant increase in resilience scores, a decrease in depression and anxiety scores, and a significantly lower attrition rate. The results suggest that eQuoo is an engaging and effective means to support students’ mental health and build their resilience.