This systematic review adhered to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [20]; a completed PRISMA checklist can be found in Multimedia Appendix 1. The detailed methodology is available in the published study protocol [21], and the study was registered in PROSPERO (CRD420203811). The SPIDER framework was used to develop the review question, which is based on describing the Sample (S), Phenomenon of Interest (PI), Design (D), Evaluation (E), and Research type (R) [22] (see Textbox 1).

We searched the MEDLINE (Ovid), EBM Reviews (Ovid), PsycINFO (Ovid), EMBASE (Elsevier), SCOPUS, CINAHL Complete (EBSCOhost), and Web of Science databases, including Science Citation Index Expanded, Social Sciences Citation Index, and Conference Proceedings Citation Index (Clarivate Analytics). Articles published in the last 5 years (January 1, 2015, to April 30, 2020) were included with no language restrictions. The search of the databases was performed on April 30, 2020.

The search formula was created using thesaurus and entry terms for the following syntaxis: “telemedicine” AND “mental health, anxiety, depression or stress” AND “systematic reviews.” The full search strategy for each database is available in Multimedia Appendix 2.

An extraction form was created for the included systematic reviews. We collected the following information: first author and publication date of the study, characteristics of the participants, main objective, research questions, inclusion criteria for the systematic review, search date, study selection process, quality assessment (if any), main findings, and limitations. The full text of the included articles, tables, and supplementary material were also gathered to perform the qualitative analysis of the text.

The aim of our study was to perform a realist review of systematic reviews using a qualitative strategy to synthesize the information and answer our research question. Therefore, we did not look for a specific result such as effectiveness, cost-effectiveness, or similar. Instead, we were interested in identifying the full text of all studies that answered our research question to perform a grounded-theory analysis with an emergent approach [25]. Priority was given in the analysis to studies with the lowest risk of bias assessed.

To assess the quality of the included systematic reviews, we used A Measurement Tool to Assess Systematic Reviews-2 (AMSTAR-2), which has 16 domains. Seven of these domains are considered critical: (1) protocol registered before the start of the review, (2) adequacy of the literature search, (3) justification for the exclusion of individual studies, (4) risk of bias of individual studies included in the review, (5) adequacy of meta-analytic methods, (6) consideration of the risk of bias in interpreting the results of the review, and (7) assessment of the presence and likely impact of publication bias [26].

AMSTAR-2 classifies the quality of systematic reviews into four categories: high (none or one noncritical weakness), moderate (more than one noncritical weakness), low (one critical weakness with or without noncritical weaknesses), and very low (more than one critical weakness with or without noncritical weaknesses). The quality assessment was rated by two trained researchers independently. In case of difference in the overall quality assessment of the systematic reviews, the AMSTAR-2 criteria were discussed between the two researchers to reach a consensus.

We developed a framework informed by a realist analysis of synchronous digital mental health interventions using a grounded-theory approach with an emergent approach [27]. The realist synthesis was based on interpreting, integrating, and inferring the evaluation elements to better understand the implementation of synchronous digital mental health interventions from all of the included studies [28]. To answer the question “what makes the implementation of these interventions work?”, hypotheses supported by the included studies’ results were developed and generated through discussion and consensus among the researchers [28]. Since our study was designed to perform a realist synthesis of the evidence, we focused on different outcomes to use them as input for assessing the implementation of synchronous digital mental health interventions. Therefore, we did not perform a quantitative synthesis in any case (ie, a meta-analysis of effectiveness).

Three researchers followed the three steps established by Thomas and Harden [28] for qualitative syntheses [29]. First, the extracted data were freely coded. The researchers read the full texts of the included articles and coded each text fragment that provided information to answer the research question. Second, the codified data were organized and then grouped based on descriptive aspects using a context-linked causality approach represented as “context+mechanism=outcome” [25]. Finally, the analytical concepts generated in the previous step were grouped so that they were related to each other. The elements that were related to each other were assumed to be part of a hypothesis that would help to answer the research aim.

The selection of the studies for the realist review was based on the AMSTAR-2 score, with the highest-quality studies being assessed first. We assessed all included studies, down to the criterion of theoretical saturation [30]. All qualitative analyses were performed with NVivo software (version 12, QSR International).

The Confidence in Evidence from Reviews of Qualitative Research (CERQual) approach, which has four components (Methodological Limitations, Relevance, Coherence, and Appropriateness Data), was assessed by a researcher and then reviewed by another independent researcher. The CERQual was evaluated to contribute to an overall assessment of each hypothesis resulting from the realist synthesis to determine the level of confidence (high, moderate, low, or very low) and to present the overall assessment in a Summary of Qualitative Findings table [31,32].

The search strategy retrieved 30,228 records, and after duplicated cleaning, we obtained 14,536 unique records. The evaluation by title and abstract identified 374 results that were evaluated at the full-text level. Among those, 353 were excluded. The reasons for exclusion are listed in Multimedia Appendix 3. Finally, 21 systematic reviews were included in this study (see Figure 1).

The included systematic reviews analyzed a median of 27 studies (range 9-155). Eleven studies reported some form of synchronous digital mental health intervention based on internet, telephone, or online cognitive behavioral therapy (CBT) as the primary intervention [19,33-42]. The remaining studies reported a mix of digital mental health interventions based on synchronous components (ie, telephone, videoconferencing) and asynchronous components (ie, text messages, email, chats, instructional videos, podcasts). Most of the systematic reviews included exclusively randomized controlled trials (RCTs) as primary studies, two included only non-RCTs, and five studies included both. Only six studies did not include a meta-analysis. With respect to the type of therapy, nine reviews stated CBT as the target therapy, one review used the transdiagnosis method, and one included mindfulness-based interventions. The individual characteristics of the included studies are presented in Multimedia Appendix 4. It is important to mention that despite having no language restrictions, all of the included articles were published in English and the systematic reviews did not include qualitative studies.

Most of the studies (19/21, 90%) of the included systematic reviews performed a risk of bias assessment. The most commonly used instrument was the Risk of Bias Cochrane Collaboration tool (12/24, 57%) [33-35,37,38,40,43-48]. Seven studies used other tools to assess the risk of bias such as the Effective Public Health Practice Project Quality Assessment Tool (2/21, 10%); Grading of Recommendations Assessment, Development and Evaluation (1/21, 5%); and others. Only two studies did not report using any risk of bias tool [41,42]. Ten studies did not appropriately account for the risk of bias of the individual studies included when interpreting the results of their review.

Olthuis et al [47] presented a medium level of confidence and Lewis et al [38] presented low confidence. The rest of the included systematic reviews presented a critically low level of confidence (see Figure 2). On average, the included reviews only met 40% of the AMSTAR-2 risk of bias items. The reviews included in Rees et al [41] failed to accomplish any of the AMSTAR-2 items and those in the study by Turgoose et al [49] only passed one AMSTAR-2 item.

The AMSTAR-2 items that were the most fulfilled (if applicable) were item 15 (critical) assessing the presence and likely impact of publication bias (14/21, 93%) and item 12 (noncritical) assessing the potential impact of risk of bias in individual studies (11/21, 73%) in the case of meta-analysis. The AMSTAR-2 items that were the least fulfilled were item 10 (noncritical) on whether the review reported the funding sources of the included studies. Only the study by Irvine et al [36] achieved compliance. Two other items that had a low compliance rate (3/21, 14%) were item 4 (critical) on the adequate literature search and item 3 (noncritical) on the justification for the decision on the study designs to be included in the review, and only one study met each of these criteria [46].

An overall analysis of the CERQual assessment showed that the hypotheses presented have low or very low confidence in the evidence (see Multimedia Appendix 5). The main methodological limitations are that the studies come from research with a low or very low confidence level. In terms of coherence, the baseline assumption and hypothesis 1 showed adequate coherence between the different findings, whereas hypotheses 2 and 3 showed moderate concern, since some reviews have heterogeneous results. Finally, all hypotheses showed the adequacy of the data and relevance of the results.

Our study developed a framework based on three hypotheses and a baseline assumption to understand/explain the implementation of synchronous digital mental health interventions. From the 21 systematic reviews included, studies showed that synchronous digital mental health interventions provide effective clinical outcomes and are as effective as face-to-face therapies that address mental health conditions [33-36,42,47,49,50]. These digital interventions reach populations such as children, veterans, refugees, and people living in rural areas [50,52], thereby reducing geographical barriers to access. Moreover, since patients are treated from their own homes and are no longer required to visit a psychiatric facility, this can reduce the fear of mental health stigma [39]. Nevertheless, there are few considerations to achieve successful therapy, such as a quiet environment for the patient to receive the session, a stable platform [49], the development or improvement of a software specifically designed for that purpose [42], and the presence of technical support when needed [43]. Some limitations should be noted due to the critically low level of confidence presented in the studies and the fact that most of the available evidence comes from high-resource countries with integrated health systems and larger research budgets [42]. Hence, some results may not be extrapolated to low- or middle-income countries.

Implementation science is an emerging and rapidly growing field that has established frameworks, methods, and strategies to improve the adoption and sustainability of interventions within the real world [53]; it has also identified different barriers and facilitators to the implementation of digital mental health interventions [53]. However, strategies specifically designed for implementing digital mental health interventions within the health care system are still limited [53-55].

The implementation of digital mental health interventions allows for overcoming many barriers in health access, such as geographic, human resources, and stigma barriers. These types of interventions allow patients and therapists to remain in their usual, more comfortable, or safer locations. Another advantage is that our framework supports that other mental health providers with lower degrees could deliver digital mental health interventions after appropriate training, which would increase the available human resources pool of therapists [41,43]. In addition, digital mental health interventions could be more attractive than face-to-face therapies, as they present the opportunity to increase privacy and minimize the risk of stigmatization since they can take place outside of mental health institutions, which is especially relevant for populations in which the presence of potential social stigma interferes with the decision to attend mental health facilities [42].

Our study provides hypotheses based on systematic reviews, which allow for obtaining a better understanding for the implementation of synchronous interventions in digital mental health. However, our framework does not provide specific steps or strategies to carry out the implementation process. Therefore, to fill this gap, other researchers could use the ERIC project framework, which presents four general phases for implementing digital interventions in the health system: an implementation strategy exploration phase, preparation phase, implementation phase, and sustainability phase [53,56]. It should be noted that other frameworks that systematize the implementation steps could be used to perform the implementation task, as long as they are adapted to the particularities of the context, health system, resources, and willingness of the actors involved. An alternative that has proven to be useful in favoring the implementation of interventions from heterogeneous contexts is a formative study design that allows for the contextualization of these interventions while evaluating their acceptability, efficiency, and safety within the health system or community [57]. However, this requires greater investment in research by low- and medium-resource countries.

There are currently no frameworks to explain the implementation of digital interventions as the main component in mental health care. Although we have not identified any studies directly comparable to ours, there are related studies. For example, a systematic review of barriers and facilitators to the implementation of electronic mental health interventions identified that the acceptability of electronic interventions depends on (1) patients’ and professionals’ expectations, (2) preferences about what they would receive and what they provide during care, and (3) the appropriateness of the electronic intervention to address patients’ mental health conditions [18]. One study proposed an ethical framework for the development, use, and implementation of digital mental health interventions such as chatbots, based on the principles of beneficence, nonmaleficence, autonomy, justice, and explicability [58]. Although chatbots are not synchronous interventions, they can be used as additional components in synchronous interventions. In the absence of an integrative framework, our study proposes a technical underpinning of available evidence to enable decision-makers to implement electronic interventions to address mental health. We identified different reviews supported by electronic interventions for anxiety, depression, and PTSD, which are equivalent to face-to-face interventions [33-36,42,47,49,50] and are cost-effective in the long term [19].

Despite evidence in favor of digital mental health interventions, there is a considerable difference between the reports from high-income and low-income countries. Some high-income countries had sufficient evidence to conduct country-focused effectiveness evaluations. For example, a systematic review from the United Kingdom identified 7 out of 48 digital interventions promoted by their health system for depression and anxiety as having a small but consistent effect and recommended their use [59]. In addition, the disparity in the amount of evidence remains in economic research, where a systematic review of economic studies identified that internet-based digital interventions for anxiety and depression are cost-effective and recommended their use; however, only studies from high-income countries were identified [60].

In contrast, no reviews of effectiveness, cost-effectiveness, or acceptability of electronic interventions were identified for low- and middle-income countries. The limited evidence from low- and middle-income countries suggests that their health systems made decisions based on minimal local evidence, low-quality evidence (ie, expert review), and evidence from only high-income countries (ie, different contexts). Additionally, material and economic resources and internet access are limited in low- and middle-income countries. Thus, sufficient internet access for health care providers and users should be assured for implementing these technologies. Other problems that could generate inequity, such as limited access to smartphones in rural and low-income areas, low internet speed, and network instability, could generate gaps for adequate implementation of these technologies.

An additional element to highlight, apart from the effectiveness or cost-effectiveness of electronic interventions, is the positive effects they could have on patients’ quality of life. Although quality of life was not an outcome in our study, we found evidence that electronic interventions to treat mental health positively affect quality of life [38,40,46]. These results are consistent with other systematic reviews showing that CBT-based interventions (eg, face-to-face, internet, or group) improve participants’ quality of life [61,62]. Furthermore, this secondary benefit of electronic mental health interventions on quality of life appeared to affect years of life lost due to disability [63]. This explains why this outcome is key for understanding the cost-effectiveness of this type of intervention since its long-term effect is to reduce costs within the health system [19].

Decision-makers and researchers could use this relevant information to support the implementation of electronic mental health interventions within their health systems (ie, teleconsultation network). There is evidence to support digital interventions due to their effectiveness in depression, anxiety, and PTSD; their feasibility and acceptability; their safety; and the additional effect on the quality of life of patients [35,37-40,44-46,48,49,51]. The treatment models that have the most empirical support are those based on CBT, which could be the first type of interventions to be implemented. In addition, evidence supports that models of CBT electronic interventions are cost-effective, making their implementation within health systems feasible in the long term.

Health systems must develop legislation and basic technological conditions to achieve the implementation of synchronous digital mental health interventions. First, legislation such as privacy policies, terms of use, and technological requirements of teleconsultation platforms should be established [4]. All of these issues should be covered and regulated by national policies and there should be an entity to enable their regulation. Consequently, health care systems should develop an integrated digital health/digital mental health system that is user-friendly for all literacy levels.

Second, there is a need for quality internet and cell phone services to increase the likelihood of adherence [4,39,42]. Collaboration among public and private sectors is needed in this regard. Technical support and access to therapies should be flexible in terms of schedules, since participants would adjust the delivery to their own timetables. Hence, night schedules should be considered. In addition, training for personnel with minor degrees must be guaranteed in a standardized and systematic way [41,43,52].

Third, for the implementation and use of electronic interventions, it is necessary to identify the barriers within each health system to achieve the acceptance of the different actors. Lack of access to technology (especially in low-resource countries), limited training in teleconsultation or reluctance of health personnel to use the technology, problems related to patient safety or privacy, and limited legislation on teleconsultation at the country level are necessary elements to evaluate during the planning of electronic interventions in mental health [64].

Fourth, the context of the COVID-19 pandemic has enhanced the use of technologies to provide health care and reduce health care access gaps, and decision-makers need to take advantage of this context to enhance the implementation and adoption of these types of interventions [3-7]. It should be noted that digital interventions are not only a short-term solution, as the trend is to incorporate them as a key part of cost-effective health care systems [19,34,42].

One of the strengths of our study is that we collected information from systematic reviews in a large number of databases, assuring the comprehensiveness of the evidence included. However, our study has limitations. First, the quality of the systematic reviews included was critically low for the most part, which could limit the confidence in the conclusions of the study. Other studies have already reported the low quality of systematic reviews and clinical practice guidelines in mental health [65-67]. Second, the electronic interventions evaluated are very heterogeneous both in the form of delivery (ie, telephone, internet-based, videoconferencing, online) and in the theoretical models used (classical CBT, mindfulness-based CBT, transdiagnostic CBT, nonspecific). Therefore, there may be variations in effect, safety, and acceptability in the way of delivery and the theoretical model used. Third, most of the research has been conducted in high-income countries, and therefore the results may not be comparable in low- and-middle income countries. Fourth, although a realist review analysis was rigorously carried out, the evidence evaluated has methodological limitations, resulting in overall low certainty of the evidence.

Our study assessed all available evidence for the implementation of synchronous digital mental health interventions and developed a framework for the implementation of synchronous digital mental health based on three hypotheses. Since it is known that digital mental health interventions are clinically effective, we hypothesized that those interventions reach otherwise inaccessible populations since they abolish the need of physical presence and mobilization (hypothesis 1) or because a nonspecialist could deliver it with the additional advantage of reducing expenses (hypothesis 2), and that digital interventions are acceptable for those receiving them and maintain the establishment of rapport (hypothesis 3). Each hypothesis represents important outcomes in the implementation process. In addition, we analyzed the barriers and facilitators for these outcomes and identified gaps in the body of evidence that require attention from future researchers.

Our study provides a framework to understand the implementation of synchronous digital mental health interventions, suggests elements to consider at the time of implementation, and establishes gaps. This information will guide decision-makers, researchers, health system managers, and implementation teams.