We conducted this systematic review and meta-analysis and reported the results following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [25].

We searched the published literature using keywords and strategies designed by the team with the assistance of a medical librarian. These strategies were created using a combination of controlled vocabulary terms and plain keywords (Multimedia Appendix 1). Databases that were searched were Ovid MEDLINE, Embase, PsycInfo, Scopus, Cochrane Library (eg, CENTRAL), ScienceDirect, and ClinicalTrials. We limited the search to studies published from January 2012 to April 2020. All searches were completed on April 30, 2020.

Studies were included if they (1) evaluated an app-based mHealth intervention designed to treat anxiety or depression or both, (2) measured symptoms of anxiety or depression, (3) were published as original research/trials in peer-reviewed journals, and (4) were written in English. We included studies that examined interventions delivered in part via mobile apps (ie, smartphone + regular phone call). We excluded studies if they (1) evaluated interventions not delivered in real-world settings (eg, only delivered within a laboratory or clinical setting), (2) evaluated interventions not delivered through a hand-held/mobile device, (3) only measured intervention usability or adherence but not the intervention effect on anxiety or depression symptoms or outcomes, (4) only measured physical stress responses but not any psychological anxiety-related symptoms, (5) did not include a control group and an experimental/comparison group with a random allocation process, or (6) used a quasi-experimental or other study design without a random allocation process.

We used the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) [26] to assess each included study in 5 domains: (1) risk of bias arising from the randomization process, (2) risk of bias due to deviations from the intended interventions (effect of assignment to intervention), (3) risk of bias due to missing outcome data, (4) risk of bias in outcome measurement, and (5) risk of bias in selection of the reported result. The assessor rated each domain as “low risk of bias,” “some concerns,” or “high risk of bias,” which constituted an overall risk-of-bias judgment for the study. Every included study was assessed by at least 2 assessors; any discrepancies were resolved through a consensus discussion during our team meeting.

We developed a Microsoft Excel spreadsheet to facilitate systematic data extraction through iterative discussions. We extracted the following data from the included studies: study details (authors, journal, year of publication, study purposes), study design (sample size, participant eligibility criteria, control type), interventions (theoretical foundations and app components), and outcomes, including data for calculating the effect size at study endpoints and follow-ups. In addition, we obtained intervention dose design information, if available, including frequency, duration, length, and timing of delivery, to examine the minimum effective intervention dose. For outcomes, we extracted primary outcomes relevant to anxiety and depression from the included studies. If a study did not indicate the primary outcome or had multiple primary outcome measures, we used data from the most used clinically validated instruments (ie, the State-Trait Anxiety Inventory [STAI] for anxiety and the Patient Health Questionnaire-9 [PHQ-9] for depression).

To pool the effect size of the interventions for each of the depression and anxiety measurements from the included studies with various measures, we computed Hedge g by taking the difference in the mean scores (1) between the intervention and control groups at each reported time point (between-group comparison) as well as (2) between the different time points following the interventions and the preintervention (ie, baseline) for the intervention groups (pre-post comparison). These time points included any reported time points during the interventions and during the follow-up after the conclusion of the interventions. For each comparison, we pooled and analyzed these Hedge g values for the target time point using both random-effect and fixed-effect models. The between-group and pre-post comparisons were also analyzed at the conclusion of the designed study intervention for depression and anxiety, respectively. Further, we used line graphs to visualize the pooled Hedge g values by time point, including follow-ups, to facilitate the analyses of dose-dependent effects and substantiality of the interventions.

We evaluated heterogeneity between studies using I², which measures the percentage of total variance that can be explained. Study heterogeneity is considered low when I²<25%, moderate when I² ranges from 25%-75%, and high when I²>75% [27]. We also visually and statistically evaluated publication bias using funnel plots and the Egger test [28]. The pooled effect accounting for missing studies was assessed using the Duval and Tweedie trim-and-fill analysis [29]. In addition, we conducted subanalyses to compare the effect sizes generated from studies that targeted both depression and anxiety symptom reduction by pooling and analyzing Hedge g values at the end of the study.

Our search strategy yielded a total of 9837 citations from the 7 databases, including ClinicalTrials. After removing duplicates, we screened 3921 (39.9%) abstracts and excluded 3467 (88.4%) citations that did not meet our inclusion criteria. We then reviewed 454 (11.6%) full-text articles and further excluded 436 (96%) studies based on our exclusion criteria (Figure 1). Of the remaining 18 (4%) studies, 15 (83%) were included in the meta-analysis; 3 (17%) studies did not report the data for meta-analysis [30-44].

We summarize the characteristics, interventions, and primary outcomes of the studies included in our meta-analysis (N=15) in Tables 1 and 2. A total of 1942 participants were included in the 15 studies. These studies were conducted in the United States (n=4, 26%) [32,34,38,40], Germany (n=2, 13%) [36,39], Sweden (n=2, 13%) [31,43], Australia (n=1, 7%) [35], Japan (n=1, 7%) [37], Korea (n=1, 7%) [44], Switzerland (n=1, 7%) [41], Taiwan (n=1, 7%) [42], and the United Kingdom (n=1, 7%) [33]; in addition, 1 (7%) study recruited participants worldwide (the total percentage is more than 100% due to rounding) [30].

The most frequently targeted population was adults (age≥18 years) self-reporting anxiety or depression symptoms (n=6, 40%) [30,34,36,38,40,41]. Other examined populations included university students (n=2, 13%) [33,39]. Australian indigenous youth (n=1, 7%) [35], and people with a diagnosis of cancer (n=2, 13%) [32,44], social anxiety disorder (n=1, 7%) [31], major depressive disorder (n=2, 13%) [37,43], and general anxiety disorder (GAD; n=1, 7%) [31].

A total of 18 mobile apps were examined in the studies, with 8 (44%) targeting depression symptom management, 4 (22%) targeting anxiety reduction, and 6 (34%) targeting both anxiety and depression (Table 2). The majority of the mHealth apps facilitated various CBTs (n=12, 67%) [31-33,35-39,41-44]. Other therapies included ACT (n=1, 6%) [35], mindfulness and breathing relaxation techniques (n=1, 6%) [30], self-esteem and acceptance of the present (n=1, 6%) [40], and attentional bias modification (n=1, 6%) [42]. The length of intervention ranged from 4 to 12 weeks, with 4 weeks being the most commonly used length (n=5, 28%) [30,33,36,40,42]. Most apps were designed to be used on a daily basis.

Various instruments were used as primary outcome measurements. For depression, most studies used the Beck Depression Inventory-II (BDI-II) as their primary outcome measure (n=5, 33%) [38,41-44]. Other depression assessment tools included the PHQ-9 (n=4, 27%) [34-37] and the Center for Epidemiologic Studies Depression Scale (CES-D; n=2, 13%) [39,40]. There were no common anxiety assessment tools across the studies. There were a total of 8 different measurements used in the studies, including the STAI (n=2, 13%) [42,44], the 6-item short-form of the STAI (n=2, 13%) [33,39], GAD-7 (n=1, 7%) [30], the Hamilton Anxiety Rating Scale (HAM-A; n=1, 7%) [32], the Beck Anxiety Inventory (BAI; n=1, 7%) [43], the Liebowitz Social Anxiety Scale-Self Report (LSAS-SR; n=1, 7%) [31], and the Social Interaction Anxiety Scale (SIAS; n=1, 7%) [41].

Most studies (n=10, 67%) [30-32,34-36,38,39,41,42] were rated as “some concerns” for bias, and 3 (20%) [33,40,44] were rated as “high risk of bias” (Figure 2). All studies reported adequate randomization sequence generation and allocation concealment. In addition, 3 (20%) studies [33,42,44] reported unclear information concerning their approaches adjusting the effects of intervention nonadherence on outcomes, and 2 (13%) studies had a high attrition rate and provided no information about their approaches addressing missing data. Blinding of outcome assessment was not possible for most included studies due to the use of self-reported outcome assessments; thus, the results of most studies (n=13, 87%) [30-34,36-41,43,44], although unlikely, may be influenced by the awareness of the intervention received. Concerning outcome reporting, we found no evidence to suspect selective reporting for all studies. There was no evidence of publication bias according to the funnel plots and Egger test (Multimedia Appendix 2).

Of the included 15 studies, 10 (67%) [30-33,40-42] examined the effectiveness of app-based mHealth interventions in anxiety management. When compared to the preintervention, at the conclusion of the interventions, participants receiving the interventions showed a statistically significant effect on anxiety symptoms (Hedge g=–.20, 95% CI –0.31 to –0.09, heterogeneity I²=0%, P=.79); see Figure 3a. Similarly, when compared to the control groups, at the conclusion of the interventions, participants receiving the interventions showed a statistically significant effect on anxiety symptoms (Hedge g=–.10, 95% CI –0.14 to –0.05, heterogeneity I²=0%, P>.99); see Figure 3b.

Of the included 15 studies, 11 (73%) [32-36,39-44] evaluated the effectiveness of app-based mHealth interventions in depression management. When compared to the preintervention, at the conclusion of the interventions, participants receiving the interventions showed a statistically significant effect on depression symptoms (Hedge g=–.25, 95% CI –0.39 to –0.11, heterogeneity I²=3%, P=.42); see Figure 4a. However, when compared to the control groups, at the conclusion of the interventions, participants receiving the interventions did not show a statistically significant effect on depression symptoms (Hedge g=–.08, 95% CI –0.23 to 0.07, heterogeneity I²=4%, P=.41); see Figure 4b.

Our subgroup analysis included 8 (53%) studies [31,32,39-44] evaluating the effectiveness of their interventions in both depression and anxiety. The results indicated that the intervention groups showed a significant effect on both anxiety (Hedge g=–.23, 95% CI –0.36 to –0.10, heterogeneity I²=0%, P>.99) and depression (Hedge g=–.22, 95% CI –0.39 to –0.06, heterogeneity I²=15%, P=.31) compared to baseline (Figures 5a and 5b). However, compared to the control groups (waiting list), mHealth interventions showed a significant effect only on anxiety at the conclusion of the interventions (Figure 5c) but not on depression (Figure 5d). This shows that mHealth interventions are more likely to improve anxiety but not depression.

When examining the dose-dependent effects of the mHealth interventions, interventions longer than 7 weeks had larger effect sizes on anxiety reduction, with a statistically significant effect size at week 7 (Figure 6a). In contrast, the pooled effects on depression fluctuated without a clear trend of dose-dependent effects (Figure 6b). Regarding the sustainability of intervention effects, the pooled effect sizes were not significant and reduced over time during follow-ups for both anxiety and depression (Figures 6c and 6d).

We conducted a systematic review and meta-analysis to examine the existing evidence on the effectiveness of app-based mHealth interventions for anxiety and depression symptom reduction. We included a total of 15 randomized controlled trials (RCTs), with many studies [32-34,38,44] published after previous reviews on a similar topic, providing an update to the current evidence. Our meta-analysis shows that app-based mHealth interventions have a modest but significant effect on anxiety reduction, consistent with previous reviews [4,11,12]. This finding adds confidence to the further development and implementation of smartphone apps to facilitate psychological treatments for anxiety symptom management [11]. In addition, our results suggest that a longer intervention (ie, 7 weeks or longer) is more likely to result in significant anxiety reduction. This finding may explain the restricted effects in studies with less than 7 weeks of app-based mHealth interventions. To the best of our knowledge, this is the first meta-analysis to assess the relationship between the app-based mHealth intervention length and the effect of the intervention. We encourage researchers to design a longer app-based mHealth intervention for anxiety symptom control and to verify our findings regarding the length of the intervention.

With regard to depression, we found that participants receiving interventions for depression experienced little symptom reduction that was not statistically significant. The finding is inconsistent with other systematic reviews reporting that smartphone apps have small-to-moderate effect sizes on depression symptom reduction [4,10,12]. The inconsistency could result from the fact that we included 1 measure per outcome per study instead of averaging the data from studies using multiple measurements for an outcome. The inconsistency may also be because previous studies included both native smartphone and web-based apps [4,10]. Web-based apps have better accessibility by allowing participants to access the interventions via various platforms [45]. In addition, the long history of web app development led to optimal user interface design, contributing to better usability and usefulness. Usability, usefulness, and accessibility have been documented as the key factors leading to successful and effective apps for mental illness management [46]. Nevertheless, we decided to exclude web-based apps because most studies reported no information about the tools their participants used to access their apps, which diminishes the purpose of our analysis on mHealth apps.

Another possible explanation of the consistency between the results of this and previous studies can be that we included 5 [32-34,38,44] studies published after previous reviews and all of them had insignificant effect sizes in our analysis. The effect sizes from the new evidence may neutralize the effect sizes from the studies included in the previous reviews. The intervention effect heterogeneity indicated that the optimal intervention content, format, and dose designs remain unclear. This is further supported by our dose-dependent analysis revealing that there is no clear relationship between the intervention length and the effect on depression, similar to a previous study [10].

There were 3 RCTs that met our eligibility criteria but were excluded from our meta-analysis due to insufficient data reported for the analysis [47-49]. All 3 studies reported positive results toward the effects of smartphone apps facilitating CBT on mental illness. Li et al [47] conducted a 12-week RCT and reported that a CBT-based smartphone chatbot intervention is efficacious for depression symptom reduction for patients with HIV and depression at both 3 and 6 months [47]. Morbeg et al [48] conducted a 4-week RCT and found that adult people receiving a CBT-based smartphone app had significantly lower anxiety and depression symptoms. Lastly, Arean et al [49] examined 2 smartphone apps in a 4-week RCT for depression and found that both apps generated a greater reduction, although not significant, in the depression symptom score compared to the control. However, we excluded these 3 studies because they either reported statistics that cannot be used to compute Hedge g without transformation based on assumptions or did not report enough data for Hedge g calculation. Studies by Li et al [47] and Morbeg et al [48] were also not included in other previous systematic reviews. The study by Arean et al [49], after data transformation with assumptions, was included in previous reviews but showed inconsistent effects on depression symptom reduction. Therefore, it was unclear whether the inclusion of these studies would alter our results for depression. Further researchers and reviewers should emphasize the gold standard of reporting to enable better study comparison and synthesis [25,50].

Consistent with previous reviews (eg, Lui et al [9]), the majority of the included studies used mobile apps to deliver CBT for anxiety or depression or both. Cognitive behavioral therapy has been delivered by computer or web apps for the treatment of various mental illnesses [51]. Our results did not suggest that smartphone apps are not useful for facilitating CBT. Rather, our results suggest that current evidence may be insufficient to guide the app-based mHealth intervention design for effective CBT-based mental illness intervention facilitation, thus requiring more research engagement. In addition, other psychotherapies, such as ACT, may also be effective in mental illness control but received relatively less attention. More studies are needed to uncover whether smartphone apps can facilitate other psychotherapies and how effective they are.

One objective of our study was to evaluate the current dose design of existing app-based mHealth interventions for anxiety and depression for an understanding of the optimal mHealth treatment length. We found that most interventions were designed to be used on a daily basis and completed within 1.5 months [52]. However, most studies provided a paucity of information about how much time their participants were asked to spend on the interventions per day or per module/session of the interventions; in addition, most studies reported no data on how much time their participants actually spent on the interventions (the actual intervention exposure). As a result, we were only able to summarize the intervention effect by the designed intervention length and dose reported in the included studies.

Our review has several limitations that should be considered when interpreting the results. First, our literature search was restricted to English publications and resulted in a small amount of research available compared to other meta-analyses examining the evidence of smartphone-based interventions for mental illness. Second, the included studies used various outcome measures, and we extracted only the primary or secondary measures for anxiety and depression. Although this strategy was used in previous systematic reviews and meta-analyses on similar topics, we might have missed the effects detected by other measures. Both limitations might result in our findings of limited or nonexistent efficacy of the interventions and confidence reduction in our dose analysis results. Finally, we included 6 studies that delivered their interventions in part by smartphone. Although app components were the main parts of their interventions, our results may not represent the effects of stand-alone smartphone apps due to the inclusion of the studies. Nevertheless, we decided to include these studies because we considered these interventions were still app-based mHealth interventions. In addition, small effect sizes for 4 of the 6 studies suggest that the nonapp components do not seem to contribute to the primary effect. Further studies, including more studies for blended interventions (smartphone app + other intervention components), are needed to compare the effects of stand-alone smartphone apps and blended interventions on mental illness management.

Despite the limitations, this review has many strengths. First, our included studies covered several publications that were published after 2019 [32-34,38,44] to reflect updated evidence, which can support future development and use of app-based mHealth interventions for anxiety and depression. Second, we conducted several analyses assessing pooled intervention effects at various study time points to understand the effective length of app-based mHealth interventions. Finally, we computed the pooled effect size of the mHealth interventions during the follow-up period to uncover the sustainability of the intervention effects on anxiety and depression reduction, which was not revealed in previous systematic reviews focusing on a similar topic [10-12]. These analyses provide innovative insights informing the future study design of app-based mHealth interventions assessing for anxiety and depression symptom reduction.

The dose design of app-based interventions has been suggested as an important aspect that profoundly influences intervention effects [13,24,53]. However, incomplete and inconsistent reporting of the intervention dose design and exposure in the existing studies impeded our quantitative analysis exploring the optimal intervention dose design for anxiety and depression. Future studies should explore the effect of app-based mHealth interventions with various dose designs and exposures for anxiety and depression symptom management. In addition, research efforts are needed to improve the reporting of intervention doses to enable comparable data for evidence evaluation and synthesis. The use of the JITAI framework to inform intervention design, evaluation, and reporting has potential to enable high-quality evidence for future app-based mHealth interventions for mental illness [13,24]. Finally, although most studies reported that their interventions sustained over follow-up compared to baseline, our analysis indicated that the pooled between-group effects of the interventions were not significant and rapidly reduced over time for both anxiety and depression. We recommend future studies to further explore the sustainability of symptom improvements from app-based mHealth interventions for anxiety and depression at various time points, including both during the study and after study completion (follow-up).

In summary, although there is some evidence in using app-based mHealth interventions for anxiety and depression symptom reduction, clinical use cannot be recommended based on this systematic review and meta-analysis due to the small to nonexistent pooled effects found in existing studies, not to mention concerns regarding study quality/reporting of the existing studies. The effects of app-based mHealth interventions may not yet be realized, as the optimal intervention dose is still unclear. Future research should consider (1) adopting a theoretical framework, such as JITAI, to inform intervention design, evaluation, and reporting to enable high-quality evidence for app-based mHealth interventions for anxiety and depression care; (2) improving the reporting of data to enable comparable data for evidence evaluation and synthesis; and (3) exploring the sustainability of treatment benefit from the mHealth interventions.