This paper is in the following e-collection/theme issue:

Reviews in Digital Mental Health (243) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1514) Depression and Mood Disorders; Suicide Prevention (1363) Anxiety and Stress Disorders (1053) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1399) Mobile Health in Psychiatry (426)

Published on in Vol 11 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/56650, first published .
Digital Mental Health Interventions for Alleviating Depression and Anxiety During Psychotherapy Waiting Lists: Systematic Review

Digital Mental Health Interventions for Alleviating Depression and Anxiety During Psychotherapy Waiting Lists: Systematic Review

Digital Mental Health Interventions for Alleviating Depression and Anxiety During Psychotherapy Waiting Lists: Systematic Review

Authors of this article:

Sijia Huang1 Author Orcid Image ;   Yiyue Wang1 Author Orcid Image ;   Gen Li2 Author Orcid Image ;   Brian J Hall2, 3, 4 Author Orcid Image ;   Thomas J Nyman1, 2, 5 Author Orcid Image
Article Authors Cited by Tweetations (4) Metrics

    Review

    1Faculty of Arts and Sciences, New York University Shanghai, Shanghai, China

    2Center for Global Health Equity, New York University Shanghai, Shanghai, China

    3School of Global Public Health, New York University, New York, NY, United States

    4Health, Behavior and Society, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States

    5School of Psychology and Clinical Language Sciences, University of Reading, Reading, United Kingdom

    Corresponding Author:

    Thomas J Nyman, PhD

    School of Psychology and Clinical Language Sciences

    University of Reading

    Earley

    Reading, RG6 6ET

    United Kingdom

    Phone: 44 1189875123

    Email: t.nyman@reading.ac.uk


    Related ArticleThis is a corrected version. See correction statement in: https://mental.jmir.org/2024/1/e67281

    Background: Depression and anxiety have become increasingly prevalent across the globe. The rising need for treatment and the lack of clinicians has resulted in prolonged waiting times for patients to receive their first session. Responding to this gap, digital mental health interventions (DMHIs) have been found effective in treating depression and anxiety and are potentially promising pretreatments for patients who are awaiting face-to-face psychotherapy. Nevertheless, whether digital interventions effectively alleviate symptoms for patients on waiting lists for face-to-face psychotherapy remains unclear.

    Objective: This review aimed to synthesize the effectiveness of DMHIs for relieving depression and anxiety symptoms of patients on waiting lists for face-to-face therapy. This review also investigated the features, perceived credibility, and usability of DMHIs during waiting times.

    Methods: In this systematic review, we searched PubMed, PsycINFO, Cochrane, and Web of Science for research studies investigating the effectiveness of DMHIs in reducing either depression or anxiety symptoms among individuals waiting for face-to-face psychotherapy. The search was conducted in June 2024, and we have included the studies that met the inclusion criteria and were published before June 6, 2024.

    Results: Of the 9267 unique records identified, 8 studies met the eligibility criteria and were included in the systematic review. Five studies were randomized controlled trials (RCTs), and 3 studies were not. Among the RCTs, we found that digital interventions reduced depression and anxiety symptoms, but the majority of interventions were not more effective compared to the control groups where participants simply waited or received a self-help book. For the non-RCTs, the interventions also reduced symptoms, but without control groups, the interpretation of the findings is limited. Finally, participants in the included studies perceived the digital interventions to be credible and useful, but high dropout rates raised concerns about treatment adherence.

    Conclusions: Due to the lack of effective interventions among the reviewed studies, especially among the RCTs, our results suggest that waiting list DMHIs are not more effective compared to simply waiting or using a self-help book. However, more high-quality RCTs with larger sample sizes are warranted in order to draw a more robust conclusion. Additionally, as this review revealed concerns regarding the high dropout rate in digital interventions, future studies could perhaps adopt more personalized and human-centered functions in interventions to increase user engagement, with the potential to increase treatment adherence and effectiveness.

    JMIR Ment Health 2024;11:e56650

    doi:10.2196/56650

    Keywords

    digital health; digital technology; digital intervention; digital interventions; waiting list; digital mental health intervention; DMHI; digital mental health interventions; DMHIs; digital mental health; mental health intervention; mental health interventions; mental health; mental illness; mental disease; mental diseases; mental illnesses; depression; depressed; major depressive disorder; MDD; depressive disorder; depressive; anxiety; anxious; self-guided; self-guidance; self-mediated; self-mediation; systematic review; systematic reviews; mood disorder; therapy; tele-therapy; web-based therapy 


    Poor mental health is an increasing global challenge [1,2], with the prevalence of depression and anxiety reaching a peak during the COVID-19 pandemic [3-5]. Several meta-analyses have concluded that psychotherapies, such as cognitive behavioral therapy (CBT), acceptance and commitment therapy, and problem-solving therapy (PST), are as efficacious at treating depression and anxiety as pharmacotherapy, and that their effects are more enduring [6-8]. Compared to pharmacotherapy, psychotherapy is often the preferred treatment for depression and anxiety by most patients, as well as clinicians [8]. According to the latest Mental Health Gap Action Program guideline by the World Health Organization [9], psychotherapy, instead of medication, is usually recommended as a frontline treatment for common mental disorders.

    However, access to psychotherapy treatment remains limited worldwide, even in high-income countries [1]. The insufficient number of clinicians has led to prolonged waiting times before face-to-face psychotherapy can begin. For example, in Germany, waiting times are on average 12.5 weeks for an initial consultation and 23.4 weeks for the first therapy session [10]. In the Netherlands, patients wait at least 6 weeks for the first treatment session [11]. In Hong Kong, while urgent and semiurgent cases wait for 1-4 weeks, stable cases, which are 76% of the total cases, wait for an average of 40 weeks [12]. A prolonged time spent on waiting lists for therapy can have negative impacts on the well-being of patients, demonstrated by increased symptoms of depression and anxiety, remission, deterioration in life quality, and even an increase in mortality [10,13-16]. This delay is also significant for most patients as their symptoms do not usually dissipate naturally with time [10,14].

    Although waiting list interventions (ie, interventions that are implemented during the weeks or months before face-to-face treatment) are potentially important for patients who are waiting for treatment, few studies have investigated their impact on reducing depression and anxiety. However, Grünzig et al [17] conducted a systematic review of low-intensity interventions to reduce depressive symptoms before outpatient psychotherapy and found limited evidence of their effectiveness in reducing depressive symptoms. Among the reviewed studies, the interventions were a mix of face-to-face sessions, web-based training for self-help strategies, and supervised bibliotherapy with feedback via email or telephone [17]. Due to the wide range of intervention formats, Grünzig et al [17] also found that acceptance varied between the interventions.

    Responding to this treatment gap, digital mental health interventions (DMHIs) are potentially a solution to the waiting list problem because they can be implemented digitally and with minimal input from a therapist [18]. Emerging evidence suggests that DMHIs are effective in improving a wide range of mental health conditions, including depression and anxiety [19,20]. Moreover, DMHIs have adopted psychotherapies, like CBT and acceptance and commitment therapy, and these have been found to be as effective as face-to-face therapies [21,22]. Due to the advantages of DMHIs, such as being more scalable, more accessible, and more cost-effective compared to face-to-face therapies [23,24], DMHIs offer the possibility to provide patients with a pre–face-to-face therapy intervention while they are on a waiting list. In addition, DMHIs can be used in a stepped care model as the frontline treatment, which steps up to more intense and advanced care if patients do not improve with the DMHIs.

    However, although evidence shows that DMHIs are effective in treating depression and anxiety, it is unknown whether DMHIs are effective for patients on waiting lists. Unlike face-to-face psychotherapies that are usually delivered in a relatively standard procedure, DMHIs encompass diverse features, such as being available on multiple digital devices (website, computer, mobile apps) and offering many delivery formats (email, text message, virtual reality, games, self-help, or guided) [25-27]. It is unclear which of these features are critical for the creation and implementation of effective interventions. The typical factors undermining the effectiveness of DMHIs include low user engagement and low perceived credibility and usability; both of which are associated with poor treatment adherence and subsequently lower effectiveness [28-30]. In contrast, guided DMHIs have been found to be associated with higher user engagement [31], higher treatment adherence [32], and higher effectiveness compared to self-guided DMHIs [21,22]. In addition to user engagement, perceived credibility and usability are positively correlated with treatment outcomes [33]. Hence, it is important to investigate the perceived credibility and usability of DMHIs, as well as user engagement for patients on waiting lists in order to assess their effectiveness.

    To our knowledge, there are no previous systematic reviews that synthesized the effectiveness of DMHIs for relieving symptoms of depression and anxiety for patients on waiting lists for face-to-face psychotherapy. Since depression and anxiety are the most prevalent mental health disorders globally [34], this review focused on DMHI’s impacts on alleviating depression and anxiety. This review also investigated the features, perceived credibility, and usability of DMHIs during waiting times.


    Overview

    This systematic review was conducted following the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses; see Multimedia Appendix 1 for PRISMA checklist) statement [35]. The study protocol was uploaded to the Open Science Framework prior to running the initial search and updates after the search have been documented.

    Search Strategy and Selection Criteria

    Searches were conducted in the following 4 web-based databases: PubMed, PsycINFO, Cochrane, and Web of Science. Backward and forward citation searching was manually conducted using the Web of Science. The search was carried out in June 2024. Studies published before June 6, 2024, and that met the inclusion criteria were included in the review.

    The search terms used were a combination of the population (ie, patients on the waiting lists for psychotherapy), treatment (ie, intervention), and outcome (ie, depression or anxiety, or both). The detailed search terms for each domain are shown in Table 1. In addition, the language was limited to English, and the publication type was limited to peer-reviewed publications in scientific journals with full-text access.

    To fulfill the objectives of this review, the inclusion criteria were (1) randomized controlled trials (RCTs), or clinical trials, or pilot studies, or feasibility studies (2) of any kind of digital intervention (3) for individuals on waiting lists (4) for psychotherapy (5) with clear psychological outcome measurements that included depressive or anxiety symptoms.

    Table 1. Search terms.
    ConceptSearch termsa
    Concept 1: population“before psychotherapy” OR “before therapy” OR “pretreatment” OR “pretherapy” OR “before outpatient psychotherapy” OR “waiting time” OR “waiting list”
    Concept 2: intervention“digital intervention” OR “digital mental health” OR “DMH” OR “eMental health” OR “mobile mental health” OR “mobile psychiatry” OR “technology” OR “online” OR “mobile” OR “phone” OR “app” OR “web” OR “internet” OR “computer*” OR “ehealth” OR “mhealth” OR “guided” OR “self-help”
    Concept 3: outcome“depress*” OR “anxiety” OR “MDD” OR “anxious” OR “GAD” OR “generalized anxiety disorder”

    aSearch terms are based on the inclusion criteria (see eligibility criteria), targeting the population, intervention, outcome, and study type. The search terms were combinations of the keywords listed under categories 1, 2, and 3.

    Screening and Data Extraction

    Three of the coauthors (SH, GL, and TJN) independently completed the title and abstract screening of the retrieved study records, as well as the full-text reviews of the eligible studies. The web-based platform Covidence (SaaS Enterprise) was used in managing the screening process. The corresponding author resolved any screening disparities.

    Data were extracted through Covidence following the same template for each study across three domains: (1) study characteristics (ie, study title, authors, publication year, country, study type, health care settings, recruitment, sample size and demographics, and inclusion and exclusion criteria); (2) intervention characteristics (ie, therapy type, components, duration, and outcome measurements); and (3) study results (ie, primary outcome: the reduction in depression and anxiety symptoms, secondary outcome: user engagement, intervention credibility, and usability).

    Data Synthesis

    The extracted data were first synthesized by the study characteristics and the features of interventions including sample size, recruitment methods, types of psychotherapy adapted, delivery formats, and outcome measures. Next, we synthesized the effectiveness of the digital intervention by analyzing the reduction of depression and anxiety symptoms, separately for different study types (ie, RCTs and non-RCTs). Finally, reports about user engagement and perceived credibility and usability were synthesized.


    Overview

    Of the 10,066 study records retrieved from the databases, and after removing duplicates and non-English studies, a total of 9267 unique titles and abstracts were screened. Following the screening, a final 22 studies remained for full-text screening (Figure 1). After scanning the full texts, 8 studies met eligibility criteria and were included in the review.

    ‎
    Figure 1. PRISMA flow diagram outlining data set and study identification. PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

    Study Characteristics

    The study characteristics are shown in Table 2. In all the studies, the participants were informed about the digital intervention by their health care service providers and were recruited if they were interested in participating and passed screening. Five studies used an RCT design, two studies were feasibility studies, and 1 study was a nonrandomized experimental study. Studies by Kenter et al [11] and Kolovos et al [36] used the same data collection for analysis and were therefore discussed together in the current review.

    Table 2. Study characteristics.
    Reference, study type, and locationSettingRecruitmentSample sizeDemographicsInclusion and exclusion criteria
    Krämer et al (2021) [37], RCTa, GermanyOutpatient clinicsIndividuals were informed about this study and provided with a screening questionnaire.136Age: mean 36.3 (SD 11.9); 78% female; 58% in a relationshipIncluded depressive patients on the waiting lists (age 18 years and older; CES-Db>22; BSIc item 9≤1). Excluded patients indicating suicidal ideation at baseline.
    Villemaire-Krajden and Myhr (2019) [38], RCT, CanadaA university clinical teaching unit in a tertiary care hospitalParticipants were recruited from referrals for CBTd and instructions were sent by emails.67Age: mean 38.21 (SD 12.71); 67% female; 43% in a relationshipUnspecified, but included outpatients with all types of depressive disorders, anxiety, bipolar, insomnia, psychosis, adjustment disorder, personality disorders, and autism spectrum disorder.
    Kenter et al (2016) [11], RCT, The NetherlandsOutpatient clinicsRecruited patients after their registration for regular face-to-face treatments. Screened by telephone.269Age: mean 38 (SD 11.4); 53.9% female; 24.2% in a relationshipIncluded outpatients waiting for face-to-face treatment at clinics with major depressive disorder diagnosed by DSM-IVe (age 18 years and older, waiting time >8 weeks). Excluded patients with suicidal ideation and antidepressant medication.
    Kolovos et al (2016) [36], RCT, The NetherlandsOutpatient clinicsRecruited patients after their registration for regular face-to-face treatments. Screened by telephone.269Age: mean 38 (SD 11.4); 53.9% female; 24.2% in a relationshipIncluded outpatients waiting for face-to-face treatment at clinics with major depressive disorder diagnosed by DSM-IV (age 18 years and older, waiting time >8 weeks). Excluded patients with suicidal ideation and antidepressant medication.
    Twomey et al (2014) [39], RCT, IrelandA public health care provider in DublinParticipants were invited by telephone, and instructions were sent by email.149Age: mean 35.3 (SD 10.3); 73.8% female; 41.6% in a relationshipIncluded adults on waiting lists with symptoms of depression, anxiety, or stress (as shown by their initial referral). Excluded those with psychosis or cognitive impairment.
    Hentati et al (2022) [40], feasibility study, SwedenA routine psychiatric care unitParticipants were informed by the health care personnel at the psychiatric care unit and self-registered digitally.12Age: mean 34.3 (SD 11.1); 50% female; 75% in a relationshipIncluded patients with symptoms of depression or anxiety on the waiting list (age 18 years and older; PHQ-9f or GAD-7g≥5; MADRS-Sh item 9<4). Excluded patients with severe suicidal ideation, and psychiatric or somatic difficulties.
    Duffy et al (2019) [41], feasibility study, United KingdomOutpatient clinicsParticipants were informed of the study through their clinician and discussed their participation with clinicians during the assessment appointment.123Age: 28% between 17 and 24, 46% between 25 and 44, 26% between 45 and 64, and 1% between 65 and 80; 69% femaleIncluded clients with severe presentations of anxiety or depression, or both, requiring high-intensity treatment. Excluded clients requiring low-intensity treatment and those with substance abuse.
    Whitfield et al (2006) [42], nonrandomized experimental study, United KingdomOutpatient clinicsReferred participants were invited to a screening appointment.20Age: mean 38.05 (SD 12.98); 45% femaleIncluded patients with problems of depression and depression with anxiety (aged between 16 and 65, BDI-IIi<2). Excluded patients with current active suicidal intent and psychosis.

    aRCT: randomized control trial.

    bCES-D: Center for Epidemiologic Studies Depression Scale [43].

    cBSI: brief symptom inventory [44].

    dCBT: cognitive behavioral therapy.

    eDSM-IV: Diagnostic and Statistical Manual of Mental Disorders (4th Edition).

    fPHQ-9: Patient Health Questionnaire-9 [45].

    gGAD-7: Generalized Anxiety Disorder-7 [46].

    hMADRS-S: Montgomery-Åsberg Depression Rating Scale - Self Assessment [47].

    iBDI-II: Beck Depression Inventory Version II [48].

    Among the 5 RCTs, only the intervention group in Krämer et al [37] reported a significant decrease in depressive symptoms, as well as improvements in psychological symptoms and quality of life, compared to the control group. In the other 4 RCTs, 3 RCTs provided the control groups with a self-help book and did not find the internet-based interventions more effective in reducing depression and anxiety symptoms than the control groups, and 1 RCT did not provide any intervention for the control group [11,36,38,39]. The 3 non-RCT studies, which did not have a control group, reported mixed findings. Two studies found that the effectiveness of the digital interventions was beneficial and reduced depression and anxiety symptoms during the waiting time and at follow-up [41,42]; however, after the intervention in the remaining study, only a few participants reached the threshold for clinical improvement in their symptoms of depression or anxiety [40].

    Quality Assessment

    We used the RoB 2 tool (a revised Cochrane risk of bias tool for randomized trials) to assess the risk of bias of the included RCTs. The RoB 2 evaluates five domains, including the randomization process, deviations from intended interventions (effect of assignment to intervention), missing outcome data, measurement of the outcome, and selection of the reported result [49]. In addition, the ROBINS-I (Risk of Bias in Nonrandomized Studies-of Interventions) tool by Cochrane was used to assess the risk of bias of the included non-RCTs. The ROBINS-I evaluates seven domains including confounding, selection of participants, classification of interventions, deviations from intended interventions, missing data, measurement of the outcomes, and selection of the reported results [50].

    Out of the 5 included RCTs, 4 RCTs had low risk and 1 RCT had some concern with regard to the randomization process (due to significant baseline differences between the two groups). Two studies had low risk and three studies had some concerns regarding low intervention adherence. Regarding missing data, 2 studies had low risk and 3 studies had moderate to high risk due to dropouts. All 5 studies had some concerns due to it being unclear if the assessment method was double-blinded or not. Finally, all studies had a low risk of bias regarding the selection of the reported result. A detailed quality assessment for each study is presented in Table S1 in Multimedia Appendix 2.

    The 3 included non-RCTs all had moderate risks. A common serious risk factor among all studies was the consideration of confounders. None of the studies reported potential confounders, which contributed to the potential bias of the study results. In addition, intervention adherence was a concern in all studies. Due to the moderate to high dropout rates, the handling of the missing values was problematic in two of the three studies. Finally, similar to RCTs, the 3 non-RCTs also only included self-reported outcome measurements, and it was unclear if outcome assessors were aware of the intervention received by study participants. A detailed quality assessment for each study is presented in Table S2 in Multimedia Appendix 2.

    Randomized Controlled Trials

    The intervention characteristics of RCTs are shown in Tables 3 and 4. Krämer et al [37] found that the “GET.ON Mood Enhancer” that they used was effective in reducing depressive symptoms. The intervention group improved significantly after the intervention compared to the control group (F2,121.5=3.91; P<.05; see mean, SD, and between-group effect size in Table 5). The control group remained on the waiting list and did not receive any intervention during the study period. These differences were also maintained at the 5-month follow-up, where the depressive symptoms were significantly reduced in the intervention group compared to the control group. In addition, Krämer et al [37] reported a significant difference in the decrease of both psychological symptoms (F2,119.4=4.37; P<.05) and mental health quality of life (F2,119.7=3.36; P<.05), indicating that the intervention group improved more than controls.

    Table 3. Intervention characteristics of RCTa studies.
    ReferenceTherapyDelivery formatsComponentsDuration and frequency
    Krämer et al (2021) [37]CBTbInternet-based platform “GET.ON Mood Enhancer”Six consecutive modules of 45 minutes, written semistandardized feedback by an e-coach (trained psychologist), homework assignments, a digital mood diary, and an optional text message coach7 weeks
    Villemaire-Krajden and Myhr (2019) [38]CBTInternet-based website “Good Days Ahead” accessible via computerNine lessons of 30 minutes, explanations for the CBT model and its application, vignettes presenting case examples, quizzes, exercises, introduction to a few behavioral techniques such as activity monitoring and scheduling9 weekly lessons
    Kenter et al (2016) [11]PSTcInternet-based platform, email reminders if not finishing a sessionFive sessions with structured homework assignments, adopting a structured 6-step approach: identifying the problem, finding solutions, selecting one solution, creating a plan to solve the problem with this solution, executing the plan, and evaluating the plan, plus web-based feedback by a coach5 weekly sessions
    Kolovos et al (2016) [36]PSTInternet-based platform, email reminders if not finishing a sessionFive sessions with structured homework assignments, adopting a structured 6-step approach: identifying the problem, finding solutions, selecting one solution, creating a plan to solve the problem with this solution, executing the plan, and evaluating the plan, plus web-based feedback by a coach5 weekly sessions
    Twomey et al (2014) [39]CBTInternet-based platform “MoodGYM,” weekly automated email remindersA brief introductory session and five 20- to 40-minute sessions, containing written information, animations, human-centered exercises, and quizzes32 days

    aRCT: randomized controlled trial.

    bCBT: cognitive behavioral therapy.

    cPST: problem-solving therapy.

    Table 4. Control group and outcome measures of RCTa studies.
    ReferenceControl groupMeasurement


    		DepressionAnxietyOthers
    Krämer et al (2021) [37]Access to treatment as usual was not restricted, participants may be seeing a general practitioner, psychiatrist, psychotherapist, using self-help, or other inpatient or outpatient practitionerPHQ-9bNot measuredPsychological symptoms (BSIc), quality of life—mental and physical (SF-12d), need and motivation for psychotherapy, attitude toward F2F psychotherapy (ATSPPHe), attitude toward web-based interventions (adapted ATSPPH), negative effects (INEPf), outpatient psychotherapy history
    Villemaire-Krajden and Myhr (2019) [38]Provided with a workbookBDI-IIgBAIhSubjective well-being, problem or symptoms, life functioning difficulties, and risk or harm to self or others (CORE-OMi)
    Kenter et al (2016) [11]Received a self-help book without any guidance or further instructionsCES-DjHADS-AkInsomnia (ISIl), self-rated health (EQ-5D [51]), perceived control (PMSm)
    Kolovos et al (2016) [36]Received a self-help book without any guidance or further instructionsCES-DNot measuredResponse to treatment and quality of life (EQ-5D)
    Twomey et al (2014) [39]No intervention at allDASSn-DepressionDASS-AnxietyDASS-Stress, work and social functioning (WSASo)

    aRCT: randomized controlled trial.

    bPHQ-9: Patient Health Questionnaire-9 [45].

    cBSI: brief symptom inventory [44].

    dSF-12: 12-Item Short-Form Health Survey [52].

    eATSPPH: Attitude Toward Seeking Professional Psychological Help Scale [53].

    fINEP: inventory for the assessment of negative effects of psychotherapy [54].

    gBDI-II: Beck Depression Inventory Version II [48].

    hBAI: Beck Anxiety Inventory [55].

    iCORE-OM: Clinical Outcomes in Routine Evaluation Outcome Measure [56].

    jCES-D: Center for Epidemiologic Studies Depression Scale [43].

    kHADS-A: Hospital Anxiety and Depression Scale Anxiety subscale [57].

    lISI: Insomnia Severity Index questionnaire [58].

    mPMS: Pearlin Mastery Scale [59].

    nDASS-21: Depression, Anxiety, and Stress Scale-21 [60].

    oWSAS: Work and Social Adjustment Scale [61].

    Table 5. Statistical results of reviewed studies.
    ReferenceDepression symptom, mean (SD)Anxiety symptom, mean (SD)Between-group effect size:
    interaction effect, time × group

    		InterventionControlInterventionControl
    Krämer et al (2021) [37]Before treatment (ITTa): 14.6 (0.5); posttreatment (ITT): 10.5 (0.7); five-month follow-up (ITT): 9.9 (0.7)Before treatment (ITT): 15.7 (0.5); posttreatment (ITT): 13.3 (0.6); five-month follow-up (ITT): 12.7 (0.7)Not measuredNot measuredPosttreatment: P=.004, Cohen d=0.55; five month follow up: P=.005, Cohen d=0.52
    Villemaire-Krajden and Myhr (2019) [38]Before treatment: 25.50 (11.94); posttreatment: 24.14 (12.50)Before treatment: 29.27 (11.85); posttreatment: 28.59 (14.70)Before treatment: 22.33 (14.38); posttreatment: 20.00 (13.42)Before treatment: 19.86 (10.61); posttreatment: 18.32 (11.65)P>.05, partial eta squared=0 for both depressive and anxiety symptoms
    Kenter et al (2016) [11] and Kolovos et al (2016) [36]Before treatment: 37.0 (11.6); posttreatment: 27.0 (15.1)Before treatment: 35.2 (12.1); posttreatment: 25.9 (14.9)Before treatment: 12.4 (3.9); posttreatment: 10.5 (5.4)Before treatment: 12.6 (4.6); posttreatment: 10.0 (5.5)Depression: Cohen d=0.07; Anxiety: Cohen d=0.09
    Twomey et al (2014) [39]Before treatment: 18.79 (11.46); posttreatment: 14.29 (10.62)Before treatment: 16.26 (9.71); posttreatment: 16.26 (10.52)Before treatment: 12.93 (8.06); posttreatment: 11.71 (8.91)Before treatment: 10.32 (7.14); posttreatment: 11.79 (9.56)Depression: Cohen d=0.19; Anxiety: Cohen d=0.01
    Hentati et al (2022) [40]No data providedN/AbNo data providedN/AN/A
    Duffy et al (2019) [41]Before-posttreatment: decrease 3.6 pointsN/ABefore-posttreatment: decrease 3.2 pointsN/AN/A
    Whitfield et al (2006) [42]Before treatment (ITT): 28.15 (11.41); posttreatment (ITT): 20.00 (10.41); three-month follow-up (ITT): 18.95 (10.41)N/ABefore treatment (ITT): 20.30 (11.23); posttreatment (ITT): 14.55 (7.82); three-month follow-up (ITT): 9.90 (8.47)N/AN/A

    aITT: intention-to-treat.

    bNot applicable.

    Villemaire-Krajden and Myhr [38] investigated the effectiveness of using computerized CBT to decrease symptoms of distress in outpatients during a waiting period for services. They did not find significant interactions between the intervention and control group, with a very small between-group effect size (Table 5). This indicates that compared to the control group, the intervention group did not score significantly better with regard to well-being, symptom severity, functioning, or motivation for CBT. In addition, depressive symptoms, anxiety symptoms, and life functioning difficulties were not found to have changed significantly between groups from baseline to the postintervention assessment. However, for both groups, the well-being (measured by Clinical Outcomes in Routine Evaluation Outcome Measure) of the participants significantly increased (F1,37=6.31; P<.05), and the symptoms significantly decreased (F1,37=8.74; P<.05) over time.

    Both Kenter et al [11] and Kolovos et al [36] investigated the effectiveness of a web-based PST, self-help intervention in reducing depression symptoms. The intervention examined by Kenter et al [11] was effective in reducing depression and anxiety symptoms, but in comparison with the control group (who were provided with a self-help book), it was not more effective. They used intention-to-treat (ITT) analysis and found a significant reduction in depressive symptoms in both the intervention group (see Table 5, within-group effect size: Cohen d=0.75) and the control group (within-group effect size: Cohen d=0.69) from baseline to posttreatment. Despite the moderate within-group effect size for both groups, the between-group effect size was small (Table 5). The between-group difference posttreatment was not significant in the regression analysis. Similarly, for anxiety, Kenter et al [11] reported moderate within-group effect sizes (intervention group: Cohen d=0.41; control group: Cohen d=0.52) and small between-group effect size (Table 5), and they found no significant difference between the intervention group and the control group. The clinical outcomes in Kolovos et al [36] were the same as Kenter et al [11] but Kolovos et al [36] focused on estimating the cost-effectiveness of the intervention in comparison with enhanced usual care. Kolovos et al [36] found that from a societal perspective, the intervention was not cost-effective in comparison with the enhanced usual care.

    Twomey et al [39] examined the effectiveness of a self-help CBT intervention (“MoodGYM”) on reducing general psychological distress, stress, depression, anxiety, and impaired daily functioning symptoms. Twomey et al [39] found symptom improvement for the treated individuals with a within-group effect size of Cohen d=0.4 for depression and Cohen d=0.14 for anxiety. Nevertheless, similar to the studies conducted by Kenter et al [11] and Kolovos et al [36] they did not find significant differences between the intervention group and the simply waiting control group (depression: F1,64=2.99; P>.05; anxiety: F1,64=1.72; P>.05) [39]. At posttreatment, small between-group effect sizes were found for both depression and anxiety (Table 5), suggesting that “MoodGYM” was not more effective than the control condition.

    Non-RCTs

    Intervention characteristics of non-RCTs are shown in Table 6. Hentati et al [40] tested the effectiveness of a self-guided PST intervention. The results showed a 16% and 22% median for symptom improvement in depression and anxiety, respectively, from screening to posttreatment. However, from pretreatment to posttreatment, the improvement in depression was reduced by 3%, while anxiety symptoms were reduced by 3%. Hentati et al [40] concluded that there was insufficient evidence for the effectiveness of this digital intervention.

    Table 6. Intervention characteristics of non-RCTa studies.
    ReferenceTherapyDelivery formatsComponentsDuration and frequencyMeasurement





    		DepressionAnxietyOthers
    Hentati et al (2022) [40]CBTbInternet-based platform accessible via computer and mobile devicesPsychoeducational texts, treatment rationale, examples of problems and suggestions of solutions, illustrative pictures, instructions, and problem-solving exercises4 weeksPHQ-9cGAD-7dTreatment credibility (CEQe), usability (SUSf), behavioral engagement, suicidal ideation (MADRS-Sg), negative effects (NEQh), poststudy questionnaire on user experience
    Duffy et al (2019) [41]CBTInternet-based platform “SilverCloud”Eight modules including tools such as self-monitoring and thought recording, behavioral activation, cognitive restructuring, and challenging core beliefs47 days, one review every 10 to 12 days.PHQ-9GAD-7Work and social functioning (WSASi)
    Whitfield et al (2006) [42]CBTComputerized CD RomSix sessions of 45-60 minutes, including text, cartoon illustrations, animations, human-centered questions, sound, and video, as well as the offer of short support sessions by a self-help support psychiatric
    nurse    		6 weekly sessionsBDI-IIjBAIkHopelessness (BHSl), social adaptation self-evaluation (SASSm)

    aRCT: randomized controlled trial.

    bCBT: cognitive behavioral therapy.

    cPHQ-9: Patient Health Questionnaire-9 [45].

    dGAD-7: Generalized Anxiety Disorder-7 [46].

    eCEQ: Credibility/Expectancy Questionnaire [62].

    fSUS: System Usability Scale [63].

    gMADRS-S: Montgomery-Åsberg Depression Rating Scale - Self Assessment [47].

    hNEQ: Negative Effects Questionnaire [64].

    iWSAS: Work and Social Adjustment Scale [61].

    jBDI-II: Beck Depression Inventory Version II [48].

    kBAI: Beck Anxiety Inventory [55].

    lBHS: Beck Hopelessness Scale [65].

    mSASS: Social Adaptation Self-evaluation Scale [66].

    Duffy et al [41] examined the outcomes of using iCBT as a prequel for patients requiring high-intensity treatment. The results indicated that 58% exhibited reliable improvement from baseline to iCBT exit and around 20% of the sample achieved reliable recovery in advance of starting face-to-face therapy. This was demonstrated by the decrease in depression (within-group effect size: Cohen d=0.61) and anxiety (within-group effect size: Cohen d=0.69). In addition, the decrease in the WSAS score, which estimates the severity of work and social adjustment impairment, followed a similar pattern with a substantial reduction (mean 2.4, SD 8.7 points) in the severity score at iCBT exit (within-group effect size: Cohen d=0.31).

    Finally, Whitfield et al [42] investigated the effectiveness of the compact disc CBT self-help intervention. They adopted an ITT analysis where the last observation for the participant was carried forward if data were missing due to questionnaire nonresponse. The ITT analysis could possibly reduce any potential bias by accounting for the situation of those who had dropped out of the study. Although they did not calculate the effect sizes, the mean BDI-II scores for depression symptoms significantly decreased (t19=4.91; P<.001), and the mean BAI score for anxiety symptoms significantly decreased as well (t19=2.51; P=.02; Table 5). Furthermore, Whitfield et al [42] found the effectiveness of the intervention persisted at the 3-month follow-up.

    Summary

    To summarize, there is some evidence of the effectiveness of digital interventions to improve depression and anxiety symptoms in individuals waiting for psychotherapy. Seven of the eight studies reported digital interventions that led to a reduction in depressive and anxiety symptoms, with, on average, a moderate within-group effect size. However, only one study found that the waiting list intervention was effective in comparison with the control group. Moreover, due to the small between-group effect sizes in four of the five RCTs and the small sample sizes in the non-RCTs, it is difficult to conclude that the waiting list interventions were effective.

    The secondary outcomes are shown in Table S3 in Multimedia Appendix 2. In brief, the reviewed studies showed that the user engagement of the self-guided digital interventions was low, but the guided digital interventions appeared more engaging compared to the unguided ones. The low engagement and high dropout rates may have contributed to the interventions being less effective. The interventions were generally perceived as credible and useful, with moderate to high user satisfaction, suggesting that digital interventions have the potential to be adapted as a pretreatment for patients on waiting lists.


    Principal Findings

    This systematic review identified 8 studies of digital interventions aimed at reducing the depression and anxiety symptoms of patients waiting for face-to-face psychotherapy. We found that waiting list interventions were effective in reducing symptoms of depression and anxiety, but not more effective compared to simply waiting for treatment or control groups who used a self-help book. Although the digital interventions were perceived as credible and useful, low user engagement was a major concern for treatment adherence and effectiveness.

    Effectiveness and Features of the Waiting List Digital Interventions

    We found that the DMHIs implemented with patients on waiting lists reduced depression and anxiety symptoms with moderate effect sizes. These effect sizes were similar to those of prior meta-analyses on the effectiveness of DMHIs in general by Firth et al [22] and Moshe et al [28]. Nevertheless, this review did not find significant differences between the intervention group and the control group. Several factors might have reduced symptoms in the control groups. First, in the RCTs conducted by Kenter et al [11] and Kolovos et al [36], the control group was provided with a self-help book instead of merely waiting; the aim was to increase the participation rate. Even with small effect sizes, previous studies have shown that self-help books were effective in reducing depression symptoms [8]. In addition, some spontaneous improvements were found among the waiting control group, as demonstrated in Twomey et al [39]. This is in line with previous studies that show that spontaneous improvement does occur among some patients with depression and those with anxiety [16,67].

    Concerning the features of the DMHIs, there was no difference between DMHIs implemented for patients specifically on waiting lists and general DMHIs. In addition, across the reviewed 8 studies, we did not find evidence of a specific technological platform (ie, using phone vs computer) or a specific psychological therapy (ie, CBT vs PST) having better effectiveness in DMHIs, which is consistent with previous studies [8,28].

    While DMHIs have the potential to provide more scalable, more accessible, and more cost-efficient treatments compared to face-to-face therapies [23,24], it is important to acknowledge that DMHIs may not be suitable to address some patients’ need for face-to-face psychotherapy, especially for those with severe symptoms [11,36]. Due to these concerns, Twomey et al [39] and Krämer et al [37] recommended that digital interventions should not be provided as a substitute for face-to-face psychotherapy but were better suited to being an additional or complementary treatment option that could bridge the waiting time. This is consistent with Cornish’s [68] stepped-care model, where patients are able to receive timely digital interventions on the same day that they seek care, balancing out the treatment intensity and resources available.

    User Engagement

    As identified by earlier studies, guided interventions are more engaging compared to unguided ones [31,32], and general DMHIs implemented during the waiting time exhibited lower user engagement and treatment adherence compared to face-to-face psychotherapies [28-30]. In this review, Krämer et al [37] show that there was a lower dropout rate and higher treatment adherence compared to the other self-guided studies. This might be explained by the participants’ demographics. For example, Twomey et al [39] reported that although 85% of the males dropped out of the study, the number of females that dropped out was only 8%. In contrast, Kenter et al [11] and Kolovos et al [36] reported that 64.7% of the participants who did not complete the study were female. They also found that participants who were younger, less educated, and had lower incomes were more likely to drop out [11,36]. In addition, as Farrington et al [69] suggest, technical difficulties experienced by certain age groups (ie, older people) and incompatibility between different digital device systems may also have been barriers to user engagement.

    Another factor for low user engagement might be related to how the digital intervention was promoted. In all the studies reviewed, the interventions were presented to the participants as an additional or temporary treatment rather than a replacement for the face-to-face psychotherapy they were waiting for. Promoting the interventions in this way might have lowered the expectancy of participants as regards the effect of the treatment and they may also have perceived it as less important. For instance, Hentati et al [40] reported that the participants were not convinced that the digital intervention would result in any major reduction in their symptoms although they perceived the intervention as credible. Kenter et al [11] found that the completion rate was positively associated with participants’ expectancy and perceived credibility of the intervention. Therefore, changing the way digital interventions are promoted may increase the engagement rate.

    Limitations and Future Studies

    Due to the understudied nature of the topic, the first limitation of this systematic review was the restricted number of randomized control studies (n=5), which reduces the possibility of drawing a robust conclusion concerning the effectiveness of waiting list interventions. In addition, according to the quality assessment, the reviewed studies had some shared concerns about using self-report data and missing values due to dropouts. In future studies, once a sufficient number of high-quality studies with adequate sample sizes are available, conducting a meta-analysis will be crucial to provide a more comprehensive synthesis of the evidence in this area. Second, various instruments were adopted to measure depression and anxiety symptoms, posing challenges when controlling for baseline differences and when comparing the effectiveness of the studies. Moreover, the psychological outcomes were self-reported in most studies, and these might be less sensitive to symptom changes compared to a clinician’s assessment [37]. Third, this study was limited to patients with depression and those with anxiety with relatively stable conditions. Patients with severe suicidal ideation and psychosis were excluded, even though these groups are at a higher risk for symptom deterioration while waiting for psychotherapy. Ketner et al [11] reported one case of suicide in the control group, suggesting that future waiting list interventions could target individuals with less stable conditions. Additionally, when considering future studies, we recommend the use of multiple research protocols for RCTs in the field of digital interventions for patients on waiting lists that target a wider range of symptoms (ie, phobic disorders and eating disorders) [70-72]. Moreover, we recommend that researchers adopt more therapies (ie, exposure therapy) and include larger samples [73,74]. Finally, it is also important for future studies to consider the use of digital interventions in different populations. We acknowledge that this review only included English studies, which may have missed valuable studies conducted in non-English speaking countries. The reviewed studies in this study were also conducted in high-income Western countries. It is unclear whether resource-limited areas may benefit more from low-cost, web-based interventions for mental health issues.

    Conclusions

    As a potential solution to the problem of prolonged waiting lists for psychotherapy, this systematic review examined the effectiveness of DMHIs in reducing anxiety and depression symptoms of patients on waiting lists for psychotherapy. Our results showed that among the RCTs, DMHIs were overall not more effective when compared with simply waiting or the control groups who used a self-help book. Among the non-RCTs, although the intervention reduced depression and anxiety symptoms, the study design with no control group made it difficult to conclude the effectiveness of the intervention. Waiting list DMHIs may prove to be an adequate treatment for some patients, but low user engagement remains a concern for treatment adherence and effectiveness. Despite patients rating DMHIs as credible and useful, it is nevertheless evident that as yet DMHIs might not fully replace face-to-face psychotherapies for all patients.

    Authors' Contributions

    All authors designed the study protocol together, contributed to writing, as well as reviewed and revised the final manuscript. SH, YW, GL, and TJN conducted the literature search, data extraction, and synthesized the data. SH and YW made Figure 1. SH and TJN made all the tables. TJN supervised the research project.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

    PDF File (Adobe PDF File), 1076 KB
    Multimedia Appendix 2

    Quality assessments and secondary outcomes.

    DOCX File , 22 KB
    1. Patel V, Saxena S, Frankish H, Boyce N. Sustainable development and global mental health—a Lancet commission. Lancet. 2016;387(10024):1143-1145. [CrossRef] [Medline]
    2. GBD 2019 DiseasesInjuries Collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: a systematic analysis for the global burden of disease study 2019. Lancet. 2020;396(10258):1204-1222. [FREE Full text] [CrossRef] [Medline]
    3. Kola L, Kohrt BA, Hanlon C, Naslund JA, Sikander S, Balaji M, et al. COVID-19 mental health impact and responses in low-income and middle-income countries: reimagining global mental health. Lancet Psychiatry. 2021;8(6):535-550. [FREE Full text] [CrossRef] [Medline]
    4. COVID-19 Mental Disorders Collaborators. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet. 2021;398(10312):1700-1712. [FREE Full text] [CrossRef] [Medline]
    5. Torales J, O'Higgins M, Castaldelli-Maia JM, Ventriglio A. The outbreak of COVID-19 coronavirus and its impact on global mental health. Int J Soc Psychiatry. 2020;66(4):317-320. [CrossRef] [Medline]
    6. DeRubeis RJ, Siegle GJ, Hollon SD. Cognitive therapy versus medication for depression: treatment outcomes and neural mechanisms. Nat Rev Neurosci. 2008;9(10):788-796. [FREE Full text] [CrossRef] [Medline]
    7. Bai Z, Luo S, Zhang L, Wu S, Chi I. Acceptance and commitment therapy (ACT) to reduce depression: a systematic review and meta-analysis. J Affect Disord. 2020;260:728-737. [CrossRef] [Medline]
    8. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry. 2013;58(7):376-385. [FREE Full text] [CrossRef] [Medline]
    9. Mental Health Gap Action Programme (mhGAP) Guideline for Mental, Neurological and Substance Use Disorders [Internet]. Geneva. World Health Organization; 2023.
    10. Ruesch M, Helmes A, Bengel J. Cognitive behavioral group therapy for patients with physical diseases and comorbid depressive or adjustment disorders on a waiting list for individual therapy: results from a randomized controlled trial. BMC Psychiatry. 2017;17(1):340. [FREE Full text] [CrossRef] [Medline]
    11. Kenter RMF, Cuijpers P, Beekman A, van Straten A. Effectiveness of a web-based guided self-help intervention for outpatients with a depressive disorder: short-term results from a randomized controlled trial. J Med Internet Res. 2016;18(3):e80. [FREE Full text] [CrossRef] [Medline]
    12. Hongkong Hospital Authority. URL: https://www.ha.org.hk/visitor/sopc_waiting_time.asp?id=7&lang=ENG [accessed 2023-12-13]
    13. Gagliardi AR, Yip CYY, Irish J, Wright FC, Rubin B, Ross H, et al. The psychological burden of waiting for procedures and patient-centred strategies that could support the mental health of wait-listed patients and caregivers during the COVID-19 pandemic: a scoping review. Health Expect. 2021;24(3):978-990. [FREE Full text] [CrossRef] [Medline]
    14. Reichert A, Jacobs R. The impact of waiting time on patient outcomes: evidence from early intervention in psychosis services in England. Health Econ. 2018;27(11):1772-1787. [FREE Full text] [CrossRef] [Medline]
    15. Furukawa TA, Noma H, Caldwell DM, Honyashiki M, Shinohara K, Imai H, et al. Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta Psychiatr Scand. 2014;130(3):181-192. [CrossRef] [Medline]
    16. Whiteford HA, Harris MG, McKeon G, Baxter A, Pennell C, Barendregt JJ, et al. Estimating remission from untreated major depression: a systematic review and meta-analysis. Psychol Med. 2013;43(8):1569-1585. [CrossRef] [Medline]
    17. Grünzig SD, Bengel J, Göhner W, Violetta Krämer L. Low-intensity interventions to reduce depressive symptoms before outpatient psychotherapy—a systematic literature review. Psychother Psychosom Med Psychol. 2019;69(6):212-223. [CrossRef] [Medline]
    18. Dao KP, De Cocker K, Tong HL, Kocaballi AB, Chow C, Laranjo L. Smartphone-delivered ecological momentary interventions based on ecological momentary assessments to promote health behaviors: systematic review and adapted checklist for reporting ecological momentary assessment and intervention studies. JMIR Mhealth Uhealth. 2021;9(11):e22890. [FREE Full text] [CrossRef] [Medline]
    19. Saramago P, Gega L, Marshall D, Nikolaidis GF, Jankovic D, Melton H, et al. Digital interventions for generalized anxiety disorder (GAD): systematic review and network meta-analysis. Front Psychiatry. 2021;12:726222. [FREE Full text] [CrossRef] [Medline]
    20. Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res. 2019;21(7):e12869. [FREE Full text] [CrossRef] [Medline]
    21. Pauley D, Cuijpers P, Papola D, Miguel C, Karyotaki E. Two decades of digital interventions for anxiety disorders: a systematic review and meta-analysis of treatment effectiveness. Psychol Med. 2023;53(2):567-579. [FREE Full text] [CrossRef] [Medline]
    22. Firth J, Torous J, Carney R, Newby J, Cosco TD, Christensen H, et al. Digital technologies in the treatment of anxiety: recent innovations and future directions. Curr Psychiatry Rep. 2018;20(6):44. [FREE Full text] [CrossRef] [Medline]
    23. Hollis C, Morriss R, Martin J, Amani S, Cotton R, Denis M, et al. Technological innovations in mental healthcare: harnessing the digital revolution. Br J Psychiatry. 2015;206(4):263-265. [FREE Full text] [CrossRef] [Medline]
    24. Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. 2016;51(5):843-851. [FREE Full text] [CrossRef] [Medline]
    25. Patel S, Akhtar A, Malins S, Wright N, Rowley E, Young E, et al. The acceptability and usability of digital health interventions for adults with depression, anxiety, and somatoform disorders: qualitative systematic review and meta-synthesis. J Med Internet Res. 2020;22(7):e16228. [FREE Full text] [CrossRef] [Medline]
    26. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual research review: digital health interventions for children and young people with mental health problems—a systematic and meta-review. J Child Psychol Psychiatry. 2017;58(4):474-503. [CrossRef] [Medline]
    27. Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, Murray E. The effectiveness of technology-based strategies to promote engagement with digital interventions: a systematic review protocol. JMIR Res Protoc. 2015;4(2):e47. [FREE Full text] [CrossRef] [Medline]
    28. Moshe I, Terhorst Y, Philippi P, Domhardt M, Cuijpers P, Cristea I, et al. Digital interventions for the treatment of depression: a meta-analytic review. Psychol Bull. 2021;147(8):749-786. [CrossRef] [Medline]
    29. Weightman M. Digital psychotherapy as an effective and timely treatment option for depression and anxiety disorders: implications for rural and remote practice. J Int Med Res. 2020;48(6):300060520928686. [FREE Full text] [CrossRef] [Medline]
    30. Fuhr K, Schröder J, Berger T, Moritz S, Meyer B, Lutz W, et al. The association between adherence and outcome in an internet intervention for depression. J Affect Disord. 2018;229:443-449. [CrossRef] [Medline]
    31. Musiat P, Johnson C, Atkinson M, Wilksch S, Wade T. Impact of guidance on intervention adherence in computerised interventions for mental health problems: a meta-analysis. Psychol Med. 2022;52(2):229-240. [CrossRef] [Medline]
    32. van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, et al. Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One. 2014;9(7):e100674. [FREE Full text] [CrossRef] [Medline]
    33. Smeets RJEM, Beelen S, Goossens MEJB, Schouten EGW, Knottnerus JA, Vlaeyen JWS. Treatment expectancy and credibility are associated with the outcome of both physical and cognitive-behavioral treatment in chronic low back pain. Clin J Pain. 2008;24(4):305-315. [CrossRef] [Medline]
    34. GBD 2015 DiseaseInjury IncidencePrevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet. 2016;388(10053):1545-1602. [FREE Full text] [CrossRef] [Medline]
    35. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. [FREE Full text] [CrossRef] [Medline]
    36. Kolovos S, Kenter R, Bosmans J, Beekman A, Cuijpers P, Kok R, et al. Economic evaluation of internet-based problem-solving guided self-help treatment in comparison with enhanced usual care for depressed outpatients waiting for face-to-face treatment: a randomized controlled trial. J Affect Disord. 2016;200:284-292. [CrossRef] [Medline]
    37. Krämer LV, Grünzig SD, Baumeister H, Ebert DD, Bengel J. Effectiveness of a guided web-based intervention to reduce depressive symptoms before outpatient psychotherapy: a pragmatic randomized controlled trial. Psychother Psychosom. 2021;90(4):233-242. [CrossRef] [Medline]
    38. Villemaire-Krajden R, Myhr G. Evaluating the use of a computerized CBT program for outpatients on a waitlist in a university CBT unit. J Psychiatr Pract. 2019;25(4):268-278. [CrossRef] [Medline]
    39. Twomey C, O'Reilly G, Byrne M, Bury M, White A, Kissane S, et al. A randomized controlled trial of the computerized CBT programme, MoodGYM, for public mental health service users waiting for interventions. Br J Clin Psychol. 2014;53(4):433-450. [CrossRef] [Medline]
    40. Hentati A, Forsell E, Ljótsson B, Lindefors N, Kraepelien M. A self-guided and monitored digital problem-solving intervention for patients with symptoms of depression or anxiety on the waiting list for treatment in routine psychiatric care: feasibility study. BJPsych Open. 2022;8(2):e43. [FREE Full text] [CrossRef] [Medline]
    41. Duffy D, Enrique A, Connell S, Connolly C, Richards D. Internet-delivered cognitive behavior therapy as a prequel to face-to-face therapy for depression and anxiety: a naturalistic observation. Front Psychiatry. 2019;10:902. [FREE Full text] [CrossRef] [Medline]
    42. Whitfield G, Hinshelwood R, Pashely A, Campsie L, Williams C. The impact of a novel computerized CBT CD rom (overcoming depression) offered to patients referred to clinical psychology. Behav Cogn Psychother. Nov 22, 2005;34(1):1-11. [CrossRef]
    43. Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 2016;1(3):385-401. [CrossRef]
    44. Franke GH, Derogatis LR. Brief Symptom Inventory von L.R. Derogatis ; BSI ; (Kurzform der SCL-90-R) ; deutsche Version. Germany. Beltz Test; 2000.
    45. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606-613. [FREE Full text] [CrossRef] [Medline]
    46. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097. [CrossRef] [Medline]
    47. Svanborg P, Asberg M. A new self-rating scale for depression and anxiety states based on the comprehensive psychopathological rating scale. Acta Psychiatr Scand. 1994;89(1):21-28. [CrossRef] [Medline]
    48. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, TX. Psychological Corporation; 1996.
    49. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. [FREE Full text] [CrossRef] [Medline]
    50. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. [FREE Full text] [CrossRef] [Medline]
    51. EuroQol Group. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199-208. [CrossRef] [Medline]
    52. Ware J, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233. [CrossRef] [Medline]
    53. Fischer EH, Farina A. Attitudes toward seeking professional psychologial help: a shortened form and considerations for research. J Coll Stud Dev. 1995;36(4):368-373. [CrossRef]
    54. Ladwig I, Rief W, Nestoriuc Y. What are the risks and side effects of psychotherapy?—Development of an inventory for the assessment of negative effects of psychotherapy (INEP). Verhaltenstherapie. 2014;24(4):252-263. [FREE Full text] [CrossRef]
    55. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56(6):893-897. [CrossRef] [Medline]
    56. Evans C, Mellor-Clark J, Margison F, Barkham M, Audin K, Connell J, et al. CORE: clinical outcomes in routine evaluation. J Ment Health. 2000;9(3):247-255. [CrossRef]
    57. Spinhoven P, Ormel J, Sloekers PP, Kempen GI, Speckens AE, Van Hemert AM. A validation study of the hospital anxiety and depression scale (HADS) in different groups of dutch subjects. Psychol Med. 1997;27(2):363-370. [CrossRef] [Medline]
    58. Morin C. Insomnia: Psychological Assessment And Management: Psychological Assessment and Management (Treatment Manuals for Practitioners). New York. Guilford press; 1993.
    59. Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav. 1978;19(1):2-21. [CrossRef] [Medline]
    60. Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories. Behav Res Ther. 1995;33(3):335-343. [CrossRef] [Medline]
    61. Mundt JC, Marks IM, Shear MK, Greist JM. The work and social adjustment scale: a simple measure of impairment in functioning. Br J Psychiatry. 2002;180:461-464. [CrossRef] [Medline]
    62. Borkovec TD, Nau SD. Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry. 1972;3(4):257-260. [CrossRef]
    63. Lewis JR. The system usability scale: past, present, and future. Int J Hum Comput Interact. 2018;34(7):577-590. [CrossRef]
    64. Rozental A, Kottorp A, Forsström D, Månsson K, Boettcher J, Andersson G, et al. The negative effects questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatments. Behav Cogn Psychother. 2019;47(5):559-572. [CrossRef] [Medline]
    65. Beck AT, Weissman A, Lester D, Trexler L. The measurement of pessimism: the hopelessness scale. J Consult Clin Psychol. 1974;42(6):861-865. [CrossRef] [Medline]
    66. Bosc M, Dubini A, Polin V. Development and validation of a social functioning scale, the social adaptation self-evaluation scale. Eur Neuropsychopharmacol. 1997;7 Suppl 1:S57-70; discussion S71. [CrossRef] [Medline]
    67. Bandelow B, Sagebiel A, Belz M, Görlich Y, Michaelis S, Wedekind D. Enduring effects of psychological treatments for anxiety disorders: meta-analysis of follow-up studies. Br J Psychiatry. 2018;212(6):333-338. [CrossRef] [Medline]
    68. Cornish P. Stepped Care 2.0: A Paradigm Shift in Mental Health. Switzerland. Springer; 2020.
    69. Farrington C, Aristidou A, Ruggeri K. mHealth and global mental health: still waiting for the mH2 wedding? Global Health. 2014;10:17. [FREE Full text] [CrossRef] [Medline]
    70. Kok RN, van Straten A, Beekman ATF, Cuijpers P. Short-term effectiveness of web-based guided self-help for phobic outpatients: randomized controlled trial. J Med Internet Res. 2014;16(9):e226. [FREE Full text] [CrossRef] [Medline]
    71. Vollert B, von Bloh P, Eiterich N, Beintner I, Hütter K, Taylor CB, et al. Recruiting participants to an internet-based eating disorder prevention trial: impact of the recruitment strategy on symptom severity and program utilization. Int J Eat Disord. 2020;53(5):476-484. [CrossRef] [Medline]
    72. Kolar DR, Hammerle F, Jenetzky E, Huss M. Smartphone-enhanced low-threshold intervention for adolescents with anorexia nervosa (SELTIAN) waiting for outpatient psychotherapy: study protocol of a randomised controlled trial. BMJ Open. 2017;7(10):e018049. [FREE Full text] [CrossRef] [Medline]
    73. Levinson CA, Ralph-Nearman C, Brown ML, Gardner T, Cardi V, Treasure J, et al. A pilot randomized control trial of online exposure for eating disorders and mechanisms of change delivered after discharge from intensive eating disorder care: a registered report. Int J Eat Disord. 2021;54(11):2066-2074. [FREE Full text] [CrossRef] [Medline]
    74. Reins JA, Boß L, Lehr D, Berking M, Ebert DD. The more I got, the less I need? Efficacy of internet-based guided self-help compared to online psychoeducation for major depressive disorder. J Affect Disord. 2019;246:695-705. [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    DMHI: digital mental health intervention
    ITT: intention-to-treat
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    PST: problem-solving therapy
    RCT: randomized controlled trial
    ROBINS-I: Risk of Bias in Nonrandomized Studies-of Interventions

    Edited by J Torous; submitted 22.01.24; peer-reviewed by MM Hossain, D Turnbull, C Navas-Ferrer; comments to author 25.05.24; revised version received 14.07.24; accepted 02.08.24; published 10.09.24.

    Copyright

    ©Sijia Huang, Yiyue Wang, Gen Li, Brian J Hall, Thomas J Nyman. Originally published in JMIR Mental Health (https://mental.jmir.org), 10.09.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.