Efficacy of eHealth Versus In-Person Cognitive Behavioral Therapy for Insomnia: Systematic Review and Meta-Analysis of Equivalence

Introduction

Background

Insomnia, characterized by difficulties initiating or maintaining sleep, which are perceived as distressing and result in significant impairment of daytime functioning, is a common concern in the general population [1]. It is estimated that approximately 20% of the population experience episodic symptoms of insomnia, resulting in negative consequences for daytime functioning, for example, fatigue, with approximately 10% fulfilling the diagnostic criteria for an insomnia disorder [1]. The association between insomnia and adverse physical and mental health outcomes has been thoroughly documented, with numerous prospective studies showing increased risk of developing cardiovascular disease [2,3], infectious diseases such as the common cold or pneumonia [4,5], all-cause dementia [6,7], mental disorders such as depression and anxiety [8], and social withdrawal and loneliness [9,10]. In addition, not only short but also long sleep duration, both possible indicators of sleep disturbances, have been associated with increased mortality [11,12]. Beyond the personal health implications, insomnia is associated with societal costs through increased health care use, higher levels of work absenteeism, diminished work-related productivity, reduced learning capacity, and poorer academic performance [13-15]. This underlines the extensive societal and economic burdens posed by untreated sleep disturbances.

While hypnotic medications are commonly used to treat insomnia, they are not recommended for long-term use due to the risk of developing tolerance and dependence [16] as well as a wide range of adverse consequences, including daytime drowsiness, impaired cognitive function, increased risk of accidents or falls, and rebound insomnia upon discontinuation [17,18]. Instead, the major sleep medicine and research organizations recommend cognitive behavioral therapy for insomnia (CBTI) as the first-line treatment for insomnia [19-21]. CBTI usually involves a combination of two or more of the following five components [17]: (1) sleep restriction therapy, which aims at promoting more efficient and consolidated sleep patterns by first reducing the time spent awake in bed and then gradually allowing the person to increase time in bed [22]; (2) stimulus-control therapy, which aims to strengthen the connection between the bed and sleep by associating the bed and bedroom with sleep rather than wakefulness [23]; (3) relaxation techniques, which aim to reduce stress, anxiety, and tension that may interfere with falling asleep or staying asleep [24]; (4) cognitive therapy, which targets negative thought patterns and maladaptive beliefs about sleep [25]; and (5) sleep hygiene education, which focuses on establishing healthy habits and optimizing the sleep environment to promote better sleep [26]. Several meta-analyses have supported the efficacy of CBTI, demonstrating both short-term [16,27] and long-term effects [28] on insomnia, not only as the primary problem but also as a comorbid condition, for example, in patients with chronic pain [29] and survivors of cancer [30]. Compared with pharmacotherapy, CBTI has been found to be at least as effective in reducing insomnia symptoms and generally demonstrates more durable effects [31].

Nonetheless, substantial challenges remain in extending assistance to those affected. Individuals with insomnia rarely receive guideline-compliant treatment, hindered by various obstacles. These include insufficient numbers of trained CBTI providers, low rates of referral by primary care physicians, and geographical and physical barriers that deter patients from receiving help [32-34]. To address these challenges, several alternative eHealth delivery formats of CBTI have been developed and evaluated [32]. These alternatives include telephone- and videoconference-delivered CBTI and fully automated web-based programs and mobile apps.

Recently published meta-analyses have revealed statistically significant and clinically meaningful effects of eHealth CBTI (eCBTI) on various measures, including insomnia severity; self-reported sleep quality; and sleep diary–based outcomes such as sleep onset latency (SOL), wake after sleep onset (WASO), total sleep time (TST), and sleep efficiency [35,36]. This efficacy extends not only to individuals with insomnia as their primary concern [37,38] but also to those with comorbid insomnia, for example, survivors of cancer [39]. However, the results of recent systematic reviews and network meta-analyses comparing various delivery formats of CBTI suggest that in-person–delivered CBTI (ipCBTI) is generally superior to eCBTI, more so for insomnia severity than for sleep diary outcomes [40,41]. In contrast, a network meta-analysis investigating a Food and Drug Administration–authorized prescription eCBTI compared to traditional ipCBTI found that eCBTI was the most efficacious regarding insomnia severity [42].

The inconclusive results of the existing meta-analyses could be due to their reliance on both direct and indirect comparisons and variations in treatment length, dosage, content, and control group types across studies of both formats, which may compromise comparability [41]. To date, no meta-analysis has focused exclusively on randomized controlled trials, conducting direct head-to-head comparisons of eCBTI and ipCBTI, and it thus remains unclear how well the 2 delivery formats compare in terms of efficacy.

In addition, when examining the equivalence or nonequivalence of 2 interventions with meta-analysis, the conventional nonsuperiority null hypothesis test procedure is insufficient. Here, a nonsignificant result merely indicates a failure to reject the null hypothesis of no difference, which cannot conclusively determine nonequivalence or equivalence [43]. To truly test whether treatments are equivalent, we must reject the null hypothesis of nonequivalence, which means that differences in effect sizes (ESs) are as large as or larger than a predetermined equivalence margin [44]. One will usually choose equivalence margins based on previously determined minimal important differences (MIDs), referring to the minimal difference in an outcome of interest that can be viewed as clinically meaningful [45].

Objectives

Given the prevalence of insomnia and the need for diverse treatment approaches, establishing the equivalence or nonequivalence of digital and traditional CBTI delivery formats is crucial. The possible equivalence or nonequivalence of eCBTI and ipCBTI has not yet been subjected to meta-analysis. The aim of this study was, therefore, to test the comparability of eCBTI and ipCBTI with a systematic review and meta-analysis of equivalence based on randomized controlled trials directly comparing the 2 delivery formats.

Methods

This study was registered with PROSPERO (registration CRD42023390811) and conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) statement (Multimedia Appendix 1) [46].

Search Strategy

The electronic databases of CINAHL, Cochrane, Embase, PsycINFO, and PubMed were searched for publications from the earliest time available until January 5, 2024. Keywords related to insomnia (eg, sleep disturbance OR sleep disorder) were combined with keywords related to CBTI (eg, cognitive behavioral OR CBT) and keywords pertaining to eHealth (eg, telehealth OR digital). The search strings were constructed in collaboration with a skilled librarian. Table S1 in Multimedia Appendix 2 comprises detailed search strings for each database. The electronic database searches were supplemented with backward searches of reference lists of included studies. No separate protocol in addition to the one registered with PROSPERO was prepared. The main methodological changes to the original registered protocol were as follows: (1) the search date was changed from 1991 to the earliest time available due to the inclusion of additional electronic delivery formats, for example, telephone-based interventions, and (2) we also extracted data on secondary nonsleep outcomes of fatigue, anxiety, and depression.

Inclusion and Exclusion Criteria

On the basis of the Population, Intervention, Comparison, and Outcome (PICO) approach [47], the inclusion criteria in Textbox 1 were used.

Study Selection and Data Extraction

Identified references were imported into the web-based software program Covidence (Veritas Health Innovation) [50]. After duplicate removal, title and abstract screening was performed, followed by full-text screening. One author (SMK) conducted the final search, with 3 authors (SMK, DSC, and PC) conducting the screening process independently. Discrepancies were resolved through discussions and, in case of disagreement, by including a fourth author (RZ). The primary outcome was total sleep disturbance calculated as the combined, that is, averaged, results for insomnia severity and sleep quality assessed with validated scales, for example, the Insomnia Severity Index (ISI), the Pittsburgh Sleep Quality Index (PSQI), or similar scales. Secondary sleep outcomes were insomnia severity measured with the ISI; sleep quality measured with the PSQI; and the sleep diary or actigraphy-based outcomes of SOL, WASO, TST, and sleep efficiency calculated as TST relative to time in bed. In addition, we extracted data on the secondary nonsleep outcomes of fatigue, anxiety, and depression, as well as for study characteristics that could potentially explain (moderate) any variations in the differences between ipCBTI and eCBTI, including mean sample age, the proportion of women in the sample, study dropout rates, the type and degree of therapist contact, the number of treatment sessions, treatment length, and the type and number of CBTI components in each condition. A total of 3 authors (SMK, DSC, and PC) extracted data from the included studies independently, and discrepancies were resolved through discussion and by including a fourth author (RZ).

Risk of Bias Assessment

The revised Cochrane Risk of Bias tool [51] was used to evaluate the risk of bias in the included studies. Five sources of bias were assessed: (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in the measurement of the outcome, and (5) bias in the selection of the reported result. All studies were evaluated for each of the 5 potential sources of bias and rated as either low risk, high risk, or some concerns on the primary outcome of sleep disturbance. In addition, an overall assessment of the risk of bias was conducted for each study. As the number of dropouts in studies investigating eCBTI is generally high, with mean attrition rates ranging from 22% to 25% [36,52], it was decided to use a less conservative criterion in domain 3. We thus considered the availability of data from ≥90% of the participants at the postintervention assessment time-point sufficient. The assessments were conducted independently by 3 authors (SMK, DSC, and PC). Disagreements were solved by negotiation.

Data Analysis

Hedges' g, a variation of Cohen d correcting for a possible bias due to a small sample size [53], was used as the standardized ES. All ES calculations were based on differences between the ipCBTI and eCBTI intervention groups in changes (means and SDs) from preintervention to postintervention time points and from preintervention to follow-up time points, standardized by change score SDs. If the relevant data were not reported, we contacted the authors, requesting them to provide this information. We also analyzed the mean differences across the different sleep-related outcomes, that is, mean differences in ISI and PSQI scores; percentages for sleep efficiency; and minutes for SOL, WASO, and TST. ESs were pooled using the inverse variance method, taking the precision of each study into account. A random-effects model was used in all analyses, with positive ESs indicating ipCBTI being more efficacious than eCBTI. If studies reported results for >1 measure per outcome, for example, insomnia severity or sleep quality, we chose the most commonly used outcome measure, that is, the ISI for insomnia severity and the PSQI for sleep quality, so that only 1 result per study was used in each data synthesis, ensuring the independence of results.

Differences between ipCBTI and eCBTI were first analyzed using a conventional random-effects test of superiority for results at both postintervention and follow-up. The pooled ESs were then subjected to analyses of equivalence [44], testing whether the CI fell within an equivalence interval based on the clinical significance thresholds (or MIDs) previously suggested for the various sleep outcome measures. Thus, the MIDs were 0.5 SD for the standardized mean differences of sleep disturbance, insomnia severity, sleep quality, sleep efficiency, SOL, WASO, and TST, as suggested in a previous study [54]. The MIDs for the mean differences were 4.4 points on the PSQI [55], 5% for sleep efficiency, 10 minutes for SOL, and 15 minutes for WASO and TST [54]. The 6-point MID previously suggested for the ISI [56] was based on an analysis of within-subject improvement, that is, minimal important change. We therefore used 0.5 × the average SD of 4.2 (2.6 points) for ISI at baseline in patients with insomnia reported in the original validation paper [48]. This SD corresponds well with the average SD of 4.1 found for ISI scores across studies at baseline in this review. The equivalence interval of SD 0.25 for depression was chosen based on the MID previously suggested [57]. As no specific MIDs were available for the measures of fatigue and anxiety, 0.5 SD was chosen as the equivalence interval for these measures [58]. The equivalence test is based on two 1-sided tests, with the 2 interventions considered to be statistically significantly equivalent if the largest of the 2 P values is <.05 [44].

Heterogeneity was explored by calculating the I² statistic [59,60]. In addition, we calculated the 95% prediction interval, that is, the interval in which 95% of future observations from the same family of studies are expected to fall [61]. Possible reasons for heterogeneity of the differences between ipCBTI and eCBTI were explored with moderator analyses comparing the ESs of studies according to the following study characteristics: mean sample age, the proportion of women in the sample (%), overall study dropout (%), the difference in dropout between eCBTI and ipCBTI (%), therapist contact in the eCBTI condition (reference: fully automated, ie, no direct or indirect therapist contact), the number of treatment sessions, and treatment duration (weeks). In addition, we explored the possible role of the number of CBTI components used in ipCBTI and eCBTI, respectively. Both categorical and continuous moderators were analyzed with meta-regression when K (the number of studies in the analysis) was ≥10.

When K was ≥10, the possibility of publication bias was evaluated with funnel plots and the method developed by Egger et al [62]. If the results were suggestive of publication bias, we planned to calculate an adjusted ES using the Duval and Tweedie trim and fill method [63]. The calculations were conducted with Comprehensive Meta-Analysis (version 4; Biostat, Inc) [64] and various formulas in Microsoft Excel.

Finally, to assess the potential efficacy of each condition, we calculated the pooled within-group differences for each condition at postintervention and follow-up for all outcomes.

Supplementary Bayesian Analyses

To aid the interpretation of the results, we conducted a supplementary Bayesian model-averaged meta-analysis [65] of the overall comparisons of ipCBTI and eCBTI at postintervention and follow-up. The procedure examined the results of four models: (1) the fixed-effect null hypothesis, that is, the difference between ipCBTI and eCBTI is nonzero (fH₀), (2) the fixed-effect alternative hypothesis, that is, the difference is zero (fH₁), (3) the random-effects null hypothesis (rH₀), and (4) the random-effects alternative hypothesis (rH₁). The Bayesian model-averaged analysis thus avoids selecting either a fixed- or random-effects model and addresses 2 questions in light of the observed data: What is the plausibility that the overall effect is 0, that is, equivalent? and Is there a between-study variability in the ESs? Bayesian methods enable direct probability statements about the hypotheses themselves and avoid other issues associated with null hypothesis significance testing, such as the overreliance on relatively arbitrary P value thresholds and the dichotomization of results into “significant” and “nonsignificant” [66]. We chose an uninformed prior probability, that is, 25%, of the 4 models and 2000 iterations. Concerning parameter distributions, we chose previously recommended defaults [65], using a 0-centered Cauchy prior with a scale of 0.707 for the ES and an empirically informed prior distribution of nonzero between-study deviation estimates from 705 meta-analyses [67]. This distribution has been approximated by an inverse-gamma (1, 0.15) prior on the SD (Tau) [65]. The Bayesian analyses were conducted with the computer software JASP (version 16; University of Amsterdam) [68]. All data included in this review are available in tables and figures in the manuscript or Multimedia Appendix 2.

Results

Study Selection

A total of 5180 records were identified via databases, and 2 additional records were identified via reference lists. After 2039 (39.36%) duplicates were removed, 3141 (60.64%) references were screened by title and abstract. Full-text screening was carried out for 1.66% (52/3141) records, and after assessing eligibility, we identified 32% (17/52) full-text reports of 15 unique randomized head-to-head comparisons of ipCBTI and eCBTI. The results of the study selection process are shown in Figure 1. A list of studies excluded after full text screening with reasons for exclusion is provided in Table S2 in Multimedia Appendix 2.

‎

Study Characteristics

The characteristics of the included studies are summarized in Table 1. Of the 15 studies, most studies were conducted in the United States (K=5, 33%), followed by Canada (K=3, 20%) and the Netherlands (K=2, 13%). The rest of the studies (K=5, 33%) were all conducted in different countries. A total of 1083 participants were included in the 15 studies and randomized to eCBTI and ipCBTI treatment conditions. A little more than half, 666/1083 (61.5%), were women. The mean age of the total sample was 40.6 (SD 11.3) years, with mean sample ages ranging from 15.5 to 55.1 years. Of the 15 studies, most focused on insomnia as the primary problem (K=11, 73%), and 4 studies (27%) focused on patients with comorbid insomnia, that is, people maintaining sobriety from alcohol (K=1, 25%), survivors of breast cancer (K=1, 25%), and patients with posttraumatic stress disorder (K=2, 50%). All studies had insomnia as an inclusion criterion, based on diagnostic criteria; validated questionnaires; or quantitative criteria such as SOL, WASO, or early morning awakenings of ≥30 minutes occurring at least 3 nights a week, or sleep efficiency <85%.

Sleep outcomes reported in the 15 included studies were insomnia severity (K=13, 87%), with the ISI being the most frequently used instrument (K=11, 85%), and sleep quality (K=5, 33%), with the PSQI being used by all studies reporting this outcome. Sleep diaries and actigraphy were used in 11 (73%) and 3 (20%) studies, respectively, assessing sleep parameters such as SOL, WASO, TST, time in bed, and sleep efficiency. A total of 11 (73%) studies assessed depression, with the Hospital Anxiety and Depression Scale (HADS) [86] (K=3, 27%) being the most frequently used, followed by the Beck Depression Inventory (BDI) [87] (K=2, 18%), the Patient Health Questionnaire (PHQ) [88] (K=2, 18%), and the Center for Epidemiologic Studies Depression Scale (CES-D) [89] (K=2, 18%). Of the 15 studies, 9 (60%) studies assessed anxiety, with most (K=4, 44%) using the HADS, followed by the General Anxiety Disorder-7 (GAD-7) [90] (K=2, 22%), and 7 (47%) studies assessed fatigue, with most frequently using the Multidimensional Fatigue Inventory (MFI) [91] (K=4, 57%). A total of 14 studies reported follow-up data, with time to follow-up ranging from 9 to 52 weeks.

In all but 1 (7%) of the 15 studies, eCBTI was delivered individually. In most studies (K=10, 67%), eCBTI involved some degree of interaction with a treatment provider, with real-time, synchronous therapist contact being available in 8 (53%) studies and asynchronous support, for example, via email, being offered in 2 (13%) studies. One (7%) study provided both synchronous and asynchronous therapist contact. In 4 (27%) studies, eCBTI was provided completely without interaction with a treatment provider, for example, in a fully automated format. ipCBTI was primarily delivered individually (K=10, 67%), with 5 (33%) studies using a group format.

Risk of Bias

The risk of bias in the individual studies is summarized in Figure S1 in Multimedia Appendix 2. Of the 15 studies, 10 (67%) were characterized as having some concerns regarding the overall risk of bias, and the remaining 5 (33%) were classified as having a high risk of bias overall. No studies were characterized as having a low risk of bias. The reasons for a study being categorized as having a high risk of bias stemmed primarily from “bias due to missing outcome data” due to the combination of high rates of missing outcome data and failure to include analyses correcting for this, for example, sensitivity analyses. Bias raising “some concerns” primarily stemmed from “bias in the measurement of the outcome” due to the combination of using a self-reported outcome and nonblinding. Only few studies had attempted some element of blinding. Of the 15 studies, 3 (20%) [79,81,84] reported that participants were kept blind to study hypotheses, 1 (7%) reported that treatment providers were kept blind to study hypotheses [79], and 2 (13%) reported that the data analyst was blinded to the allocation status of the participants [83,84]. In addition, bias raising “some concerns” stemmed from “bias in the selection of the reported result” due to inadequate preregistration of the analytical strategy.

Comparing Intervention Characteristics of ipCBTI and eCBTI

More participants had dropped out of eCBTI than out of ipCBTI at both postintervention (104/528, 19.7% vs 72/526, 13.7%) and follow-up (176/509, 34.6% vs 145/509, 28.5%). However, the differences did not reach statistical significance (P=.17 and .46). No between-condition differences were found in the mean number of sessions (6.0 vs 6.1; P=.91), the duration of the intervention (6.7 weeks vs 6.6 weeks; P=.83), or the number of CBTI components (4.5 vs 4.6; P=.83).

Within-Group Effects

As presented in Table S3 in Multimedia Appendix 2, statistically significant (P<.05) improvements from preintervention to postintervention time points and from preintervention to follow-up time points were observed for both ipCBTI and eCBTI for all self-reported sleep outcomes. At postintervention time points, the ESs (Hedges' g) ranged from 0.27 (TST) to 1.97 (ISI) for ipCBTI and from 0.23 (TST) to 1.36 (ISI) for eCBTI. Similarly, at follow-up, the ESs ranged from 0.43 (TST) to 1.88 (ISI) for ipCBTI and from 0.39 (TST) to 1.41 (total sleep disturbance) for eCBTI. For the few actigraphy-based sleep outcomes at postintervention (K=3, 20%), only the results for SOL in the ipCBTI condition (Hedges' g=0.53; mean difference=–11.5 minutes) reached statistical significance. At postintervention time points, in the ipCBTI condition, the ISI total score was, on average, improved by 9.0 points; the PSQI global score by 4.4 points; diary-based sleep efficiency by 12.1%; and diary-based SOL, WASO, and TST by –20.9, –23.5, and +21.3 minutes, respectively. The comparable results for eCBTI were 7.1 points, 3.5 points, 10.3%, –19.6 minutes, –19.5 minutes, and +16.3 minutes, respectively. As also seen in Table S3 in Multimedia Appendix 2, the within-participant improvements in fatigue, anxiety, and depression were all statistically significant and similar for both delivery formats at postintervention. Similar results were found for the secondary nonsleep outcomes at follow-up (data not shown).

Conventional Superiority Meta-Analysis

As presented in Table 2, when analyzed with conventional superiority meta-analysis, the pooled differences between ipCBTI and eCBTI reached statistical significance in 11 (34%) out of 32 comparisons. The effects generally favored ipCBTI but were small, for example, corresponding to a mean difference of 1.8 points on the ISI and 1.9% in sleep efficiency. The pooled difference for total sleep disturbance corresponded to a small ES (Hedges' g=0.32). The forest plots are shown in Figures 2-4 [69-85] and Figures S2-S11 in Multimedia Appendix 2. Concerning the secondary nonsleep outcomes of fatigue, anxiety, and depression, no differences in the conventional superiority analyses reached statistical significance (Table S4 in Multimedia Appendix 2).

‎

‎

‎

Equivalence Meta-Analysis

As shown in Table 2, the 95% CI for total sleep disturbance was included in the prespecified equivalence interval for this outcome, and, based on the largest p-value of two one-sided tests, the null hypothesis of nonequivalence was rejected (P=.04). As shown in Table 2, when examining the various sleep outcomes, ipCBTI and eCBTI were statistically significantly equivalent for 25 (78%) out of 32 calculations. Furthermore, ipCBTI and eCBTI emerged as statistically significantly equivalent for all 3 secondary nonsleep outcomes at both time points (Table S4 in Multimedia Appendix 2).

Publication Bias and Outliers

Inspecting the funnel plot and Egger's test for total sleep disturbance, which included data from all included studies (15/15, 100%), did not indicate publication bias (Egger test, P=.56; refer to the funnel plot in Figure S12 in Multimedia Appendix 2). Considering ESs, larger or smaller than 2 SDs beyond the pooled ES, revealed no outliers.

Heterogeneity and Moderator Analyses

As seen in Table 2, heterogeneity analyses suggested that varying proportions of the variance in postintervention effects stem from between-study differences beyond random error. The I² values were highest for the questionnaire-based sleep outcomes (62%-70%) and generally lower for the sleep diary and actigraphy-based outcomes (0%-31.2%). The data also suggested relatively high levels of heterogeneity for outcomes at follow-up. As shown in Table 3, when exploring possible explanations for the heterogeneity with meta-regression, 2 (18%) of the 11 analyzed moderators reached statistical significance. Differences between the proportions of dropouts in the eCBTI and ipCBTI moderated the between-group effects, with higher dropout rates in eCBTI compared with ipCBTI being associated with larger differences in favor of ipCBTI at both postintervention and follow-up, explaining 50% and 74% of the variation, respectively. Longer overall treatment duration was associated with larger differences in favor of ipCBTI compared with eCBTI at both time points. No statistically significant effects were found for the remaining moderators analyzed.

Results of Supplementary Bayesian Analyses

As presented in Table S5 in Multimedia Appendix 2, the Bayesian meta-analyses favored the alternative hypothesis of equivalence, that is, a zero difference between ipCBTI and eCBTI, for 4 (67%) out of 6 sleep outcomes at postintervention time points. The Bayes factors (BFs), that is, the probabilities of the alternative hypotheses relative to the null hypotheses, ranged from 1.7 (PSQI) to 9.9 (TST), indicating that a zero difference between ipCBTI and eCBTI is 1.7 to 9.9 times more likely than a nonzero difference. The level of evidence [93] ranged from anecdotal (BF=1-3) for PSQI to moderate (BF=3-10) for SOL, WASO, and TST. A 0 and nonzero difference for sleep efficiency appeared equally likely (BF=1.2). Insomnia severity assessed with the ISI was the only outcome for which the current evidence clearly favored a nonzero difference, with this result being 10.5 times more likely than the null hypothesis. Concerning heterogeneity, the data provided strong (BF=13.0) and anecdotal evidence (BF=2.3) for heterogeneous ISI and PSQI ESs, respectively. Nonheterogeneity was slightly more likely for the remaining outcomes (BF=1.9-3.5).

Discussion

Sleep Outcomes

When pooling the results of the 15 unique randomized trials directly comparing eCBTIs with ipCBTIs using conventional meta-analysis, the observed differences generally favored ipCBTI. Specifically, the postintervention results revealed statistically significant advantages for ipCBTI across several dimensions, including overall sleep disturbance (encompassing both insomnia severity and sleep quality), insomnia severity assessed independently, and sleep efficiency. While ipCBTI was statistically significantly superior to eCBTI for these outcomes, the magnitudes of these differences were modest, corresponding to small ESs (Hedges' g) and small mean, nonstandardized differences. For example, concerning the latter, the pooled mean differences in favor of ipCBTI for insomnia severity and sleep efficiency were only 1.8 points (on the ISI) and 1.9%, respectively. Furthermore, for total sleep disturbance and sleep efficiency, the CIs fell within the suggested equivalence margins of 0.5 SD and 5%, respectively [54].

Regarding the remaining self-reported sleep outcomes at postintervention, none yielded statistically significantly superior results in favor of either delivery type. Furthermore, with the exception of Hedges' g for the PSQI, all remaining analyses showed the 2 delivery types to be statistically significantly equivalent; that is, the CIs of the pooled effect parameter fell within the suggested equivalence margin for that parameter. The same general pattern was observed for the results obtained at the (on average) 21-week follow-up. On the basis of the available data, eCBTI and ipCBTI were statistically significantly equivalent for almost all self-reported sleep outcomes, except for the ES for sleep quality assessed with the PSQI and the mean difference in minutes for TST.

While equivalence indicates similar efficacy, if one only examines the between-group differences, it cannot be determined whether the equivalence stems from similar small or similar large improvements in both conditions. Therefore, we also calculated the within-group effects for each delivery format. The results revealed that both ipCBTI and eCBTI were associated with statistically significant within-condition improvements in all self-reported outcomes at both postintervention and follow-up. The largest effects were seen in both conditions for total sleep disturbance, insomnia severity, sleep quality, and sleep efficiency. Small-to-medium effects were observed for the remaining self-reported sleep outcomes. Therefore, it may be concluded that both delivery formats appear efficacious, displaying improvements at postintervention corresponding to 9- and 7-point reductions on the ISI, 12% and 10% improvements in sleep efficiency, 21- and 20-minute reductions in SOL, 24- and 20-minute reductions in WASO, and 21- to 16-minute increase in TST for ipCBTI and eCBTI, respectively. These effects are all clinically relevant and well beyond the suggested MIDs and minimal important changes for these outcomes, that is, a 6-point within-person change on the ISI [56] and 10, 15, and 15 minutes for SOL, WASO, and TST, respectively [54]. In addition, these clinically relevant, positive improvements were sustained over time, supporting previous findings that CBTI, regardless of the delivery format, yields robust long-term effects [28].

Only 3 (20%) of the 15 studies assessed sleep objectively, that is, with actigraphy. Despite the small number of studies, the results for sleep efficiency and SOL showed the 2 delivery formats to be statistically significantly equivalent. In contrast, the results for actigraphy-assessed TST appeared to be in favor of eCBTI, with eCBTI resulting in increased TST and ipCBTI in reduced TST. However, neither the conventional nor the equivalence analyses reached statistical significance. When examining the within-group effects, statistically significant improvements were seen for actigraphy-based SOL in the ipCBTI group. The remaining effects for TST and sleep efficiency failed to reach statistical significance. Such discrepancies between effects on self-reported and objectively assessed sleep outcomes, especially concerning estimates of sleep duration, are a well-recognized issue in sleep research and clinical practice [94].

Nonsleep Outcomes

It is well known that insomnia can lead to various physical and mental symptoms, including increased levels of fatigue [95], and that it is a significant predictor of later onset of mental disorders such as depression and anxiety [8]. Therefore, we also explored the effects on the secondary nonsleep outcomes of fatigue, anxiety, and depression. The 2 delivery formats of CBTI were statistically significantly equivalent in their effects on these symptoms, and both yielded statistically significant medium-to-large within-condition improvements of almost identical magnitude in all 3 outcomes. Thus, our results add to the more general findings that CBTI may not only improve insomnia itself but also associated psychological and physical symptoms such as depression, anxiety, and fatigue [96-98], with our findings indicating that both delivery formats appear equally efficacious in reducing these symptoms.

Heterogeneity and Its Sources

When exploring possible heterogeneity of the postintervention effects, the relatively large I² statistics observed for both insomnia severity and sleep quality and the combined outcome of total sleep disturbance indicate that a considerable proportion of the variance is due to systematic differences between the study characteristics. In contrast, the differences in effects on sleep diary outcomes displayed little or no signs of heterogeneity. When we investigated possible sources of heterogeneity for the combined total sleep disturbance outcome, 2 study characteristics emerged as statistically significant moderators at both postintervention and follow-up.

First, higher dropout rates in eCBTI than in ipCBTI were significantly associated with larger differences in favor of ipCBTI in effect on total sleep disturbance. At postintervention time points, on average, 30% more participants in eCBTI had dropped out compared to ipCBTI. While we do not know the reasons for the higher dropout rates in eCBTI, this factor, which explained between 50% and 74% of the variance in between-condition differences in effect, could represent an important, potentially modifiable factor that needs to be addressed if the efficacy of eCBTI is to be further increased. While the research on adherence to electronically adapted interventions for insomnia is still limited, studies in this [99] and other clinical populations suggest that common factors influencing dropout and adherence across such interventions include engagement and motivation; technical issues and usability; and demographic factors such as age, educational level, and digital literacy [100,101]. Second, while there was no difference in the mean duration of the 2 delivery formats, interventions with longer duration favored ipCBTI. The moderating effect of intervention duration persisted when adjusting for study dropout. We have no clear explanation for this finding, but longer treatment duration may allow for increased trust and improved therapeutic alliance in personally delivered CBTI, which, in turn, will increase the effect.

None of the remaining moderators analyzed reached statistical significance, including demographic characteristics such as mean sample age and percentage of women in the sample; study characteristics such as time to follow-up; and treatment characteristics such as therapist contact versus no contact, the number of treatment sessions, and the number of CBTI components. Some of the nonsignificant results could be viewed as surprising. For example, one might have expected larger differences between ipCBTI and fully automated eCBTI than between ipCBTI and eCBTI with some degree of therapist contact. One would also have expected age to play a role, for example, that older sample age would be associated with larger between-condition differences. Possible reasons for nonsignificant findings could be insufficient between-study variation, for example, in sample age, and inadequate statistical power due to the relatively small number of studies. Further research is needed to identify the common and different factors associated with the increased efficacy of the 2 delivery formats.

Limitations

Our findings should be interpreted cautiously for several reasons.

First, the interpretability is challenged by between-study heterogeneity, for example, by considerable between-study differences in eCBTI formats, with some eCBTIs delivered with direct therapist contact via telephone or videoconferencing; some delivered on the web with asynchronous therapist contact, for example, through email; and others offered as fully automated programs. While we attempted to explore the possible moderating role of such variations and found no indication of a moderating effect of the degree of therapist contact involved, the relatively small number of studies may have limited our ability to identify the influence of such characteristics.

Second, as demonstrated by the results of the Bayesian analyses, the small number of studies restricts the strength of the evidence. While the currently available evidence favored equivalent effects for 4 (67%) out of 6 outcomes (sleep quality, SOL, WASO, and TST), the level of evidence was weak (ie, anecdotal) to moderate. The evidence for sleep efficiency was inconclusive, and while the level of evidence for a nonzero difference in favor of ipCBTI was characterized as strong, the BF was only just above the lower limit (ie, ≥10) [93].

Third, interpreting the differences between eCBTI and ipCBTI as equivalent or nonequivalent clearly depends on the chosen equivalence margins. While we chose the MIDs suggested in the literature, for example, 2.6 points, 4.4 points, 5%, 10 minutes, 15 minutes, and 15 minutes for ISI, PSQI, sleep efficiency, SOL, WASO, and TST, respectively [48,54,55], specific MIDs have not been identified for all the corresponding ESs. While we used the 0.5 SD suggested in the literature [54,58], the clinical relevance of this value has yet to be established for several of the sleep outcomes investigated in this review.

Finally, assessed with the revised Cochrane Risk of Bias tool [51,102], one-third of the studies were characterized as having an overall high risk of bias, and the remaining two-thirds were characterized as having some concerns. Among the main reasons for these categorizations were high rates of missing outcome data and the use of self-reported outcomes. These issues cannot easily be amended. For example, it is not too surprising that behavioral interventions, in general, and eHealth interventions, in particular, have higher dropout rates than pharmacological trials. In addition, while sleep characteristics such as SOL, WASO, and TST can be assessed with both self-report and objective measures, insomnia is inherently a subjective outcome, which can be evaluated only with self-report. Furthermore, ensuring blinding is another factor that is difficult to obtain with behavioral interventions and not possible when comparing in-person and electronically delivered interventions. Other reasons for the identified risks of bias can be addressed more easily, including the failure to include sensitivity analyses correcting for missing outcome data and insufficient preregistering of analytical plans.

Conclusions

This, to our knowledge, first systematic review and meta-analysis of randomized head-to-head comparisons of eCBTI and ipCBTI suggests that while the effects tended to be in favor of the latter, the mean differences were generally of small magnitudes, with several approaching 0. Furthermore, the 2 CBTI delivery formats were statistically significantly equivalent for most outcomes examined. Statistically significant equivalence means that the CIs of the differences fell within the prespecified equivalence margins, with the latter being based on the minimal clinically relevant differences suggested in the literature for each of the outcomes in question. Importantly, when examining the within-condition effects, both delivery formats yielded large and clinically relevant effects on most outcomes, including the nonsleep outcomes of fatigue, anxiety, and depression. Although the results should be interpreted cautiously due to the currently limited evidence base, they support eCBTI, including fully automated programs, as clinically relevant alternatives to ipCBTI. These results are promising for people with insomnia, given the challenges of meeting population needs with conventional treatment formats.