The systematic review and meta-analysis were conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [46]. A protocol was designed and registered through the Open Science Framework after data collection and before data extraction and analysis began [47]. The quality of the studies included was assessed through the Cochrane Collaboration’s risk of bias assessment tool [48].

Studies were included if they met the following criteria: (1) studies involving human participants; (2) mental health apps targeting depression as a primary, secondary, or tertiary outcome; (3) RCTs or experimental or quasi-experimental designs with an active, wait-list, or treatment-as-usual control group; (4) published between January 1, 2005, and December 31, 2020; (5) mental health app intervention groups contain elements of CBT, acceptance and commitment therapy (ACT), behavioral activation (BA), or mindfulness; and (6) provide a measure of depressive symptoms pre- and posttreatment. Studies were excluded if they (1) were books chapters, meta-analyses, reviews, case studies, or opinion pieces; (2) were not written in English; (3) included participants younger than 18 years of age; (4) included participants with a terminal or life-threatening illness (eg, patients with cancer who had depression) to avoid potential confounds of disentangling which condition (ie, life-threatening illness or depression) influenced outcomes; and (5) included a therapist or other mental health specialist’s guidance for the mental health apps, as this could create a confound. No specific measure of depression was required, in order to allow the focus to remain on the mental health apps.

Studies were identified through a comprehensive literature search in PubMed, PsycInfo, Cochrane Clinical Trials Registry, Web of Science, and PsyArXiv (for publication bias) with no publication date restriction. The search was conducted in February 2021. Additionally, the authors conducted a manual search to locate studies that were not identified through databases. Search terms used three different concepts critical to the extant literature: app-based, mental health, and reward or gamification. Within each concept (eg, app-based), we identified multiple tags that reflected this concept (eg, “mental health app” and “MHapp”). The specific combination of operators can be found in Multimedia Appendix 1. This resulted in 129 unique combinations of search terms (eg, “MHapp-Money” and “Depression-Points”).

During the identification phase, articles were identified and collected based on the search term combinations from the five databases. After duplicate removal, two researchers (SGS and KAB) independently conducted initial screening for eligible articles by assessing titles and abstracts for inclusion or exclusion criteria (Multimedia Appendix 2). After the initial screening, both researchers independently assessed the remaining full-text articles against the inclusion and exclusion criteria. Disagreements were resolved through re-examination of the articles in question and discussion among the screeners.

Two independent reviewers independently coded the studies in a Microsoft Excel spreadsheet. The following data were extracted from each article: first author, year of publication, participants’ characteristics (ie, gender and age), population, and study length (Multimedia Appendix 3), as well as app name, app classification (ie, mobile or internet), presence of gamification elements, type of gamification element (eg, digital rewards, challenge or game, and competition or challenges), app adherence, the instrument used to measure depression, and type of therapy (eg, CBT and ACT) offered (Table 1) [31,49-85].

The conceptualization of gamification was modeled after previous research in which gamification was defined as having three components: a design feature that uses motivational affordances to influence psychological and behavioral outcomes [23]. Another literature review that focused exclusively on the health and well-being app domain has described very similar conceptualizations of gamification elements [86]. Modeled after these gamification literature reviews, gamification for this meta-analysis was defined using the following nine motivational affordance categories: points, achievements or badges, levels, narrative stories or themes, clear goals, performance-based feedback, rewards, progress metrics (eg, progress bars), and challenges [23,86]. While gamification can also include leaderboards [19], this element was excluded from the meta-analysis, as leaderboards in the context of mental health may promote social comparison, which can be counterproductive [87,88]. First authors of the studies were contacted via email to confirm conceptualization of the number of gamification elements. The number of gamification elements included in each intervention was included as a moderator in analyses.

Raw depression scores (mean and SD) at posttreatment for each study were extracted. If a study compared more than one mental health app intervention to a control group, or if more than one independent sample was examined in an article, both were included as separate comparisons. Studies used different, but convergently valid, measures of depression. If the articles met any of the exclusion criteria, specifically missing data, they were excluded from data analysis (n=12).

The quality of each study was assessed using the Cochrane Collaboration’s risk of bias assessment tool, which provided seven basic criteria: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other biases [48]. Studies were scored on a scale ranging from 0 to 2, where 0 indicates “low or no bias,” 1 indicates that the “level of bias is unclear,” and 2 indicates “high bias” (Multimedia Appendix 4). In line with previous research, if a study did not address one of the categories, it was given a 1 for the lack of explanation [89]. Total scores for individual and all studies are presented in Figure 1.

Data collection commenced in February 2021 and ended in May 2021. As of June 2021, a total of 5597 articles were identified through the literature search. After duplicate removal, 2741 eligible articles remained for title and abstract screening. Two researchers (SGS and KAB) independently identified 101 articles as potentially eligible. Screeners had an agreement rate of 98.94% (Cohen k=0.85). Full-text screening of these 101 articles was conducted. Of the reviewed articles, 38 studies with 39 different mental health apps met the inclusion criteria and were therefore included. Figure 2 presents the study selection identification, screening, and eligibility process.

A total of 8110 participants provided analyzable data (4362, 53.8%, in the interventions and 3748, 46.2%, in the control conditions) for this meta-analysis, with the majority of participants being female (n=4728, 58.3%; mean age 35.6, SD 7.9, years). Multimedia Appendix 3 shows descriptive information for each study. Treatment duration ranged from 10 days to 4 months. Most of the studies (26/38, 68%) used mental health apps with CBT [31,45-48,50,51,53,55-62,66-68,72-75,77-79], 32% (12/38) used mindfulness [49,52,57,63-67,69-71,74,76,82], 8% (3/38) used ACT [62,73,82], 8% (3/38) did not have a therapy associated with the app, and only 5% (2/38) used BA [54,70]. In terms of the depressive symptom outcome measures, 32% (12/38) of the studies used the PHQ-9, 26% (10/38) used the BDI-II, 16% (6/38) used the DASS-21, 11% (4/38) used the CES-D, 3% (1/38) used the HADS, 3% (1/38) used the IDS-SR, 3% (1/38) used the CCAPS-34, and 3% (1/38) used the PROMIS.

Of the 50 different comparisons used in the 38 different studies, 71% (27/38) contained gamification elements and 29% (11/38) did not. The number of gamification elements observed in each study are shown in Multimedia Appendix 5. Table 1 shows the type of intervention, app, and number of gamification elements for each article. Multimedia Appendix 6 reports supplemental meta-analytic results for long-term follow-up time points and control variables.

A forest plot for the postintervention differences between the mental health app intervention group and the control group is shown in Figure 3. The random effects model for all eligible studies (n=50 comparisons) revealed a small to medium effect of mental health apps in reducing depressive symptoms compared to controls (g=–0.27, 95% CI –0.36 to –0.17; P<.001). However, significant heterogeneity in the results were observed (I²=0.76, τ²=0.076; P<.001). The gamification moderator was not a significant predictor of depressive symptoms (β=–0.03, SE=0.04; P=.38); the intervention duration moderator was also not a significant predictor (β=–0.02, SE=0.04; P=.67).

A sensitivity analysis was performed for app comparisons that involved CBT-based therapy (37/50, 74%). The overall effect size was similar to the overall analysis (g=–0.30, 95% CI –0.42 to –0.17; P<.001), and there was no significant effect of gamification elements as a moderator (β=–0.04, SE=0.04; P=.31) or of study duration as a moderator (β=–0.02, SE=0.05; P=.69). Multimedia Appendix 7 show the funnel plot for this analysis.

The secondary analysis examining comparisons that included a measure of adherence (36/50, 72%) failed to show a significant effect of gamification elements (β=–1.93, SE=2.28; P=.40) on adherence rates. However, intervention duration was a significant predictor such that longer interventions were associated with higher adherence rates (β=11.33, SE=3.57; P=.003). Similarly, when examining only the CBT-based mental health apps (25/50, 50%), there was no effect of gamification elements on adherence (β=0.17, SE=2.51; P=.95), but intervention duration positively predicted adherence (β=12.23, SE=4.21; P=.008).

Examination of the funnel plot of the posttreatment effect SEs indicated heteroscedasticity. The Egger test of asymmetry was significant (Q₅₀=214.46, P<.001). A trim-and-fill analysis suggested a trending finding of three missing publications to the right, though this did not reach the level of statistical significance (β=2.45, P=.13). To be conservative, the pseudo-CIs in Figure 4 were adjusted per τ², with trim-and-fill studies as white dots. Controlling for the violations of these assumptions, the overall random effects model was significant (β=–0.26, SE=0.06; P<.001), indicating a significant improvement overall.

This meta-analysis provides a comprehensive update about the effectiveness of mental health apps in reducing depressive symptoms, and it tests whether mental health apps with gamification elements are effective for reducing depressive symptoms. Results indicated that mental health apps are effective for reducing depressive symptoms, but gamification elements within mental health apps do not seem to reduce depressive symptoms or increase the adherence of using mental health apps.

Examination of mental health apps that exclusively employed computerized CBT (excluding mindfulness and ACT) similarly failed to show an effect of gamification on depressive symptom reduction. It should be noted that some of the included studies had a high risk of bias and, as such, the results should be interpreted with caution. Nevertheless, the evidence from this meta-analysis suggests that CBT effectively aids with the control, maintenance, and reduction of depressive symptoms, but gamification and reward elements embedded in mental health apps do not significantly alleviate such symptoms.

Consistent with prior systematic reviews and meta-analyses on digital interventions for depression, we found that mental health app interventions are moderately effective in reducing depressive symptoms compared to controls [26,37,38,41,89]. Effect sizes from these prior meta-analyses ranged from 0.33 to 0.58, which is slightly higher than the small to moderate effect size observed in this meta-analysis. While this research provides strong consensus of a general positive effect of mental health apps on alleviating depressive symptoms, the specific elements that contribute to this effect remain largely elusive. Previous systematic reviews have demonstrated that guidance and support from a professional can augment the effectiveness of mental health app interventions [37,105]. Other work suggests that incorporating reminders into digital mental health interventions can promote engagement and adherence with the intervention, which may, in turn, enhance the intervention’s therapeutic benefits [38]. However, the results of this meta-analysis did not show a significant relationship between gamification and adherence rates. Understanding how such features work together to mitigate depressive symptoms is critical for improving mental health app development.

One potential explanation of why gamification elements do not moderate depressive symptoms may be reward sensitivity. Depression generally has been associated with hyposensitivity to rewards, particularly among those with anhedonic symptoms [11-13,16,17]. This diminished reward-related neural activity may decrease motivation to obtain rewards or diminish the positive reward experience. While the UGM model proposes reward as a critical part of motivation to ensure engagement, it also suggests moderating effects, such as self-efficacy and locus of control. These variables have known negative relationships with depression [106,107] and may make gamification less effective. An alternative explanation for the results may be that gamification does not engender additive benefits. The included mental health apps all used strong, evidence-based therapeutic interventions, including CBT, ACT, and mindfulness. Based on the results of this meta-analysis, these interventions appear to be sufficient in mitigating depressive symptoms.

Overall, 30 studies in this meta-analysis showed a significant effect, while nine studies failed to produce significant reductions in depressive symptoms. In terms of gamification elements, four of the apps that were associated with nonsignificant findings included at least two elements of gamification: Kokoro, Headspace, MoodGYM, and Be Good to Yourself [50,52,59,67]. In contrast, two of these apps—Headspace and MoodGYM—were used in other studies, where they produced significant positive changes in depressive symptoms [31,53,64,65,68,69]. These mixed findings support the notion that while mental health apps may have a future in telemedicine and psychological settings [89,108], more research is needed to understand which mental health app features are integral to improving mental health symptoms. From there, these facets can be properly implemented as a methodologically reliable therapeutic technique.

The push for the creation of efficient and scientifically supported mental health apps grows each year as technology becomes more ubiquitous [109]. These apps could aid therapists who are unable to accept any new clients and people who may not have access to psychological centers or counselors due to financial strains, location, or disabilities. This meta-analysis adds to the current literature by suggesting against overreliance on reward and gamification elements as major reducers of depressive symptoms. These elements may be beneficial in mental health apps, but no more so than other evidence-based therapeutic features. Many of the mental health apps currently available to the general public lack valid testing of their efficacy; thus, there is a strong need for rigorous evaluation of such apps on psychological outcomes. Indeed, in this meta-analysis, nearly half of the included studies incorporated gamification elements, yet the results suggest that such elements exert minimal therapeutic benefits. Significantly more research is needed to identify which specific mental health app features maximize therapeutic effects.

This systematic review and meta-analysis had some limitations, which were largely related to inadequacies in the studies available for analyses. Results were calculated based on aggregated samples, which may have caused a certain level of ecological bias. Due to the disparate sample of apps, heterogeneity was detected in all primary analyses. However, heterogeneity was nearly identical to past research on this topic, and we took statistical efforts to minimize this impact [38]. In addition, the meta-analysis was not exclusive to individuals with clinical levels of depression, but all experiencing depressive symptoms. No data sets were consistently available across all studies to assess clinical depression diagnoses or psychiatric comorbidities. Consequently, the findings may not generalize to individuals with severe depression or individuals with other mental health conditions.

The main findings of this study were that mental health apps are effective for reducing depressive symptoms, but gamification elements within these mental health apps do not seem to affect depressive symptoms. It is possible that mental health apps with gamification elements may influence patients managing anxiety, stress, or other conditions where anhedonia is not present. Thus, gamification may not be a promising app feature for depressive symptoms but may hold promise for other mental health conditions. Future research should consider examining the effectiveness of mental health apps with gamification on other mental health conditions. While previous research has investigated the effectiveness of mental health apps on anxiety and life satisfaction [50,56], the effect of gamification elements on these psychological factors remains underexplored. Moreover, there is a need for additional research to better characterize the usability benefits and user preferences of mental health apps. If gamification within mental health apps is not effective for individuals experiencing depressive symptoms, then it is important to identify other potential features. Designing specific features that may motivate users with depression toward continued mental health app adherence could lead to beneficial outcomes.

Mental health apps have proven to be a useful tool in reducing depressive symptoms with or without the inclusion of gamification elements. These results demonstrate that although there is a significant improvement in using mental health apps overall, there is no evidence to suggest that gamification makes outcomes significantly better or worse. Additional elements, such as personalization, motivational reminders, social support, and usability, need to be investigated. Mental health apps may provide a readily available option for global psychological care; however, supplementary research is needed on their effectiveness before reliable implementation into the health care system can occur.