Assessing the Short-Term Efficacy of Digital Cognitive Behavioral Therapy for Insomnia With Different Types of Coaching: Randomized Controlled Comparative Trial

Introduction

Background

Insomnia disorder is the most common sleep-wake disorder. It is characterized by difficulty initiating or maintaining sleep despite having adequate opportunities to sleep or having nonrestorative sleep, not explained by other sleep disorders [1,2]. The prevalence of insomnia disorder ranges from 2.3% to 25.5% globally [3] and was heightened during the COVID-19 pandemic, affecting up to one-third of the general population [4-6]. Insomnia is associated with high societal and economic costs resulting from health care utilization, work absenteeism, and lost productivity [7,8]. In the United States, the estimated annual cost of insomnia reaches up to US $100 billion [9]. In addition, insomnia contributes to substantial losses in annual gross domestic product, amounting to US $19.6 billion in Canada, US $41.4 billion in the United Kingdom, and US $19.2 billion in Australia [10]. The cost of untreated insomnia outweighs the cost of treating insomnia, with cognitive behavioral therapy for insomnia (CBTi) achieving greater cost-effectiveness than pharmacological treatments [11]. Technology-aided delivery of CBTi may have even greater cost-effectiveness, given its high scalability and reduced demands for human resources compared to face-to-face CBTi [12].

CBTi is an evidence-based, first-line treatment for insomnia disorder recommended by health organizations around the world [13,14]. It targets the cognitive and behavioral mechanisms perpetuating sleep difficulties, namely compromised sleep drive, disturbed circadian rhythm, and hyperarousal, especially hyperarousal associated with the sleeping environment or sleep per se [15-17]. Integrating multiple treatment techniques, CBTi aims to preserve sleep drive, stabilize circadian rhythm, and reduce hyperarousal; it is typically delivered in 6 to 8 hourly sessions by a trained mental health professional [18]. CBTi effectively improves sleep quality and reduces insomnia symptoms across populations, including populations with medical and psychiatric comorbidities. On average, CBTi leads to 20- to 30-minute reductions in both sleep onset latency and wake after sleep onset and approximately 10% increases in sleep efficiency at the end of treatment [19] with less consistent effects on total sleep time (TST) [20]. CBTi also improves mood, fatigue, and quality of life—indirectly through sleep improvements or directly through changes in behavior and cognition [21,22]. Despite its strong evidence base, CBTi is not consistently delivered to most people due to various implementation obstacles, notably the lack of mental health workers trained in delivering CBTi and limitations in time and resources required for delivering and receiving CBTi [23].

Digital CBTi (dCBTi) is a promising alternative mode of delivery of CBTi, given its high scalability and low demands for human resources. Meta-analyses have found comparable efficacy estimates of dCBTi compared to face-to-face CBTi [24,25]. However, a direct comparative trial found dCBTi less efficacious than face-to-face CBTi [26]. Notably, fully automated dCBTi is the most cost-effective treatment of insomnia, followed by group CBTi and individual CBTi [27]. Of the different types of dCBTi, dCBTi with therapist support is found to have the highest efficacy compared to other types of dCBTi without therapist support [28]. Indeed, CBTi recipients attribute treatment success to the working alliance with the therapist, and they perceive therapist-assisted support, such as the provision of personalized feedback, motivational enhancement, and accountability, to be critical for increasing engagement with dCBTi [23]. Nonetheless, the need for therapist support hinders the scalability and accessibility of dCBTi, which are the core benefits of dCBTi over face-to-face CBTi.

dCBTi with nontherapist guidance is a lower-cost alternative. Although it may be limited in delivering expert advice and addressing challenging motivational or emotional barriers, nontherapist support can provide personalized feedback, motivational enhancement, and accountability. Nontherapist support has been found to improve treatment outcomes of self-help CBTi [29]. Technological advancements have enabled the development of virtual conversational agents designed to mimic patient-therapist interactions and provide personalized content and feedback, also known as chatbot-based coaching or e-coaching [30]. Nontherapist and chatbot-based support require no therapist and preserve the core benefit of dCBTi over face-to-face CBTi. Meta-analyses have found mixed results regarding the effect sizes of dCBTi with different types of coaching support [24,25], but a direct comparative trial has not yet been conducted.

Low treatment adherence and high attrition rates are the major challenges in implementing dCBTi and even face-to-face CBTi [31,32]. On average, half of the dCBTi recipients do not adhere to treatment [33] compared to 14% to 44% in face-to-face CBTi [34]. While dCBTi offers greater potential for scalability, if engagement and treatment adherence are low, its impact on population health will remain minimal. Adding human or chatbot-based guidance to dCBTi is one of the most discussed solutions to improving engagement [31,33]; however, empirical support for its effects on treatment adherence is still being determined.

In sum, dCBTi is undoubtedly a promising intervention for insomnia that could have a major impact at the population level. Nevertheless, more research on how to optimize its benefits is needed. In particular, adding human or chatbot-based coaching has been frequently regarded as a useful strategy to improve treatment efficacy. However, a direct comparative trial of dCBTi with different types of coaching compared to dCBTi without any coaching (dCBTi-unguided) has yet to be conducted. Furthermore, the effects of different types of coaching on treatment adherence have not been evaluated.

Objectives

This study is the first empirical evaluation of the efficacy of dCBTi with different types of coaching. Moreover, we aim to evaluate whether different coaching types improve treatment adherence along with treatment outcomes. Specifically, in a 5-arm randomized controlled comparative trial, we aim to evaluate whether (1) a fully automated mobile phone–based dCBTi has superior efficacy to an active digital sleep hygiene and self-monitoring control (dSH); (2) adding coaching, regardless of type, would enhance treatment adherence and efficacy compared to dCBTi-unguided; (3) dCBTi with human support would enhance treatment adherence and efficacy compared to dCBTi with chatbot-based coaching only (dCBTi-chatbot); and (4) support from a therapist is superior to support from a nontherapist. We hypothesized that (1) dCBTi, regardless of the presence of coaching, would be efficacious for improving insomnia symptoms compared to dSH; (2) dCBTi with coaching would promote greater improvements in insomnia symptoms and greater treatment adherence than dCBTi-unguided; (3) dCBTi with human coaching would promote greater improvements in insomnia symptoms and treatment adherence than dCBTi-chatbot; and (4) dCBTi with chatbot-based coaching and therapist support (dCBTi-therapist) would promote greater improvements in insomnia symptoms and greater treatment adherence than dCBTi with chatbot-based coaching and research assistant support (dCBTi-assistant).

Methods

Study Design

The study was a 5-arm, parallel, participant-blinded, randomized controlled comparative trial. The 5 conditions included dCBTi-therapist, dCBTi-assistant, dCBTi-chatbot, dCBTi-unguided, and dSH. The study protocol was preregistered on ClinicalTrials.gov (NCT05136638).

Ethical Considerations

This study was approved by the University of Hong Kong Human Research Ethics Committee before data collection (EA210458). Electronic informed consent was obtained from each participant before study participation. Each participant was informed that participation was entirely voluntary, and they could withdraw from the study at any point without negative consequences. All data were kept confidential in a password-protected drive. Only the research team had access to the data. All personal identifying information was removed from the research data and will be kept separately from the research data for 3 years after the publication of the main study findings to ensure that there are no problems regarding consent, fabrication, or falsification. Anonymous data will be kept indefinitely. Each participant was compensated HK $60 (approximately US $8) for completing research measures at each time point.

Randomization and Blinding

Simple randomization with equal chances of assigning a participant to 1 of the 5 conditions was conducted using the randomization function implemented in Sleep Sensei, a mobile app developed with the MobileCoach platform [35,36] specifically for this study. Participants were informed that they were assigned to one of the intervention conditions. However, they were not informed about the conditions or the condition to which they were assigned. They were not informed about the study hypotheses either. As all participants were given access to Sleep Sensei, we considered them blinded to the condition assignment and study hypotheses. The therapists and assistants who provided coaching support were not blinded to the assignment. The assessments of treatment outcomes were all self-administered using Qualtrics (Qualtrics International Inc).

Participants

The inclusion criteria were as follows: participants who (1) have an Insomnia Severity Index (ISI) score of ≥10, indicating clinically significant insomnia [37]; (2) are aged 18 to 65 years; (3) have access to a smartphone and a local telephone number; and (4) are able to understand written Chinese and spoken Cantonese, the languages used in Sleep Sensei. The exclusion criteria were as follows: (1) self-reported sleep apnea or high risk of sleep apnea identified using the Berlin Questionnaire [38]; (2) self-reported acute, untreated mental or medical illnesses that would interfere with participation; (3) suicidal ideation suggested by a score of ≥1 on the Patient Health Questionnaire-9 (PHQ-9) and confirmed by a follow-up interview by a clinical psychology trainee; (4) unstable medication use that can affect sleep; (5) currently receiving psychotherapy for insomnia; and (6) other conditions that prevent adherence to CBTi recommendations, such as shift work. Participants who did not complete baseline research assessments were also excluded. Eligible participants showed sufficient digital literacy to be able to complete the web-based screening survey and use the mobile app.

An a priori power analysis was conducted to determine the sample size required for detecting significant group-by-time interaction effects if there were clinically meaningful differences in the primary outcome (a 4-point difference in the ISI total score [39]) in the patterns specified in our 4 hypotheses. A simulation-based power analysis was performed using the R package mixedpower [40]. We simulated a database using the means and SDs of ISI scores from a local sample for another insomnia trial as the baseline values [41] because these data were most likely the closest estimates of baseline ISI values in our study sample. For hypothesis 1, we simulated a database with after-treatment ISI values to be 4 points lower than the baseline ISI values in the experimental condition. For hypotheses 2 to 4, we simulated a database with after-treatment ISI values to be 4 points lower than the baseline's and 4 points lower than the comparison group's. On the basis of these simulations, a sample size of 120 would be required for detecting significant results with statistical power >95%, >80%, >90%, and >80% for hypotheses 1, 2, 3, and 4, respectively.

Procedures

Participants were recruited using mass emails sent to students, staff, and affiliates of the university, as well as social media advertisements with the institutional affiliation displayed. Inclusion and exclusion criteria were evaluated based on potential participants’ responses to the screening survey, followed by telephone interviews as needed by authors SHCL and AKMC, both of whom were clinical psychology trainees under the supervision of lead author WSC, a licensed clinical psychologist. Eligible participants then downloaded Sleep Sensei for free. All conditions were delivered via Sleep Sensei. They were given access to modules and functions according to the condition to which they were assigned (Table 1). They had access to Sleep Sensei from baseline to follow-up. Assessments of outcomes were administered at baseline, after treatment, and at 4-week after-treatment (follow-up) using Qualtrics. Figure 1 presents the CONSORT (Consolidated Standards of Reporting Trials) flow diagram.

‎

Interventions

Sleep Sensei

Sleep Sensei was developed specifically for this study and hence was customized to deliver the 5 conditions with differing combinations of content and functionalities. Sleep Sensei used the talk-and-tools user-interface paradigm, which comprised a talk system enabling text-based chat interactions between the user and a conversational agent (ie, a chatbot-based coach) and a tools system allowing the user to observe and manipulate the objects in the interface [30]. Before launching the app, Sleep Sensei underwent 3 rounds of usability testing with 6 volunteers.

In Sleep Sensei, the core CBTi treatment components were implemented using the tools system, which consisted of (1) 6 video lessons providing psychoeducation and the rationale for each treatment recommendation, delivered chronologically in the order described in the next subsection; (2) a resource library storing and presenting video lesson content in written format as well as additional resources such as relaxation recordings; (3) daily diary entries and visualization of diary data; (4) automatic, individually tailored weekly sleep schedule suggestions; and (5) weekly goal-setting and action-planning entries. Multimedia Appendix 1 presents screenshots of the Sleep Sensei interface.

CBTi Core Treatment Components

CBTi is a multicomponent intervention that combines behavioral techniques and cognitive therapy [14]. The core treatment components of CBTi include sleep restriction, stimulus control, sleep hygiene, psychoeducation about sleep, relaxation, and cognitive therapy. Sleep hygiene is included in CBTi but is not considered CBTi on its own. It is often used as an active control condition in clinical trials of CBTi [42].

Sleep restriction is a technique to improve one’s sleep efficiency by restoring a high sleep drive via limiting time in bed (TIB), specifically to match one’s current sleep needs. It is achieved by tailoring TIB to match one’s average TST. Once adequate sleep efficiency is achieved, TIB can gradually extend until optimal sleep duration is reached. In this study, the rationale and procedures for sleep restriction are presented in video lesson 1. If the participant’s previous-week sleep diary data show that their sleep efficiency is ≥85%, and they indicated that they were not sleep deprived by answering a yes or no question, they would be asked to maintain their TIB with consistent wake time and bedtime. If their sleep efficiency is ≥85%, and they indicated that they were sleep deprived, the app would recommend TIB with a 20-minute increase. If their sleep efficiency is <85%, the app would recommend a range of TIBs, from the minimum TST (equivalent to the previous week’s average TST based on diary data but at least 5 hours) to 20 minutes less than the previous week’s TIB. The participant will be asked to specify a consistent wake time suitable for them and choose a bedtime that would result in a TIB within the suggested range. This implementation of the sleep restriction procedures enables the participant to choose to restrict TIB more aggressively or gradually by 20 minutes each week, allowing for greater flexibility and potentially greater adherence.

Sleep hygiene refers to daily habits that influence sleep drive, circadian rhythm, or arousal associated with the sleeping environment. The daily habits include keeping the sleeping environment dark, quiet, and cool; having a consistent sleep schedule, especially a consistent wake time; maintaining adequate light exposure and activity levels; reducing stimulating activities before bedtime such as eating too much, intense exercise, alcohol or nicotine consumption, and screen time; and setting up a relaxation routine before bedtime. The rationale and procedures for sleep hygiene are presented in video lesson 2.

Stimulus control refers to reconditioning the sleeping environment to be a place only for sleeping and to reduce conditioned arousal. The procedures instruct the participant to enter the sleeping environment only when they feel ready to fall asleep, leave the sleeping environment if they cannot fall asleep or stay asleep for approximately >20 minutes by estimation, and go back to bed only when one feels sleepy. The distinction between feeling fatigued and sleepy is also discussed. The rationale and procedures for stimulus control are presented in video lesson 3.

Relaxation includes practices of progressive muscle relaxation and visual imagery exercises that can be used to reduce hyperarousal. The rationale and procedures for relaxation are presented in video lesson 4. Participants are guided to practice relaxation during the video lesson and told that recordings of relaxation guidance could be accessed in the resource library. In addition, another clinical technique targeting excessive worry is also introduced in this lesson—setting up worry time (referring to setting aside a 30-minute period each day [not close to bedtime] for worrying) and limiting worrying only to this time to restrict the impact of worrying on mental health and sleep.

Cognitive therapy refers to a set of techniques to identify and reframe thoughts and beliefs that may promote and sustain sleep difficulties, such as thoughts that lead to heightened worries and frustration about the consequences of not having good sleep or thoughts that reduce motivation and treatment adherence. The rationale and procedures for cognitive therapy are presented in video lessons 5 and 6.

Chatbot-Based Coaching

The chatbot-based coaching element was delivered using the talk system via a series of logic-based preprogrammed conversational turns created by WSC, SHCL, and AKMC with the goal to simulate therapeutic interactions. These text-based conversations covered the following areas: (1) after-video summary and quiz (the chatbot-based coach guided the participants to reflect on what they learned from the video lessons and facilitated them to apply the learned strategies to their own situations), (2) weekly goal setting and action planning (the participants were guided to set up goals and action plans to implement the treatment recommendations during the week), (3) positive feedback and reflection (encouraging messages were sent to participants when they completed daily sleep diaries and achieved their weekly goals), and (4) problem-solving (when the participants did not complete daily diaries or achieve weekly goals, the chatbot-based coach guided them to think about different solutions to remove the barriers to implement the treatment recommendations).

Research Assistant Support

Research assistant support was provided to participants in dCBTi-assistant at the end of sessions 1, 3, and 6 by 2 undergraduate research interns who had no prior experience in counseling or CBTi. They were trained by WSC to provide supportive contact, including expressing appreciation for the participants’ time and effort in using the app, encouraging them to continue using the app and complete daily diaries, and asking whether they had encountered any technical issues that needed support. If the participant asked sleep- or CBTi-related questions, the assistants would direct the participant to review the CBTi materials on the app. The contact time was restricted to between 20 and 30 minutes. The percentages of telephone calls completed were 74% (20/27), 33% (9/27), and 30% (8/27) for sessions 1, 3, and 6, respectively.

Therapist Support

Therapist support was provided to participants in dCBTi-therapist at the end of sessions 1, 3, and 6 by 2 postgraduate clinical psychology trainees (SHCL and AKMC) who had received at least 1 year of clinical psychology training and training in a CBTi protocol. The support included reinforcing the understanding of the intervention materials, providing sleep- and CBTi-related expert advice, reviewing treatment progress, identifying and resolving barriers to implementing CBTi treatment strategies, and addressing any motivational issues. Similar to the dCBTi-assistant condition, the contact time was restricted to between 20 and 30 minutes. The percentages of telephone calls completed were 84% (21/25), 76% (19/25), and 68% (17/25) for sessions 1, 3, and 6, respectively.

Measures

Overview

The assessments were all electronic questionnaires administered using Qualtrics. These questionnaires had been tested by research assistants before being distributed to the participants. The questionnaires were distributed through email, and participants were instructed to complete them within 1 week. The order of the questions was the same for each participant. All questions were mandatory.

Primary Outcome (ISI)

The ISI is a widely used self-report questionnaire assessing insomnia symptoms in the previous 2 weeks [43]. It consists of 7 items asking the participants to rate the severity of their insomnia symptoms on a 5-point Likert scale ranging from 0 to 4, with higher scores indicating greater insomnia severity. The composite score ranges from 0 to 28, with a score of ≥10 indicating clinical insomnia. The validated Chinese version was used in this study [44]. The ISI showed acceptable to excellent internal consistencies across the 3 time points, with Cronbach α values in the range of 0.77 to 0.90.

Secondary Outcomes

Sleep Condition Indicator

The Sleep Condition Indicator (SCI) is a newer measure of insomnia symptoms developed based on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), diagnostic criteria [1]; research diagnostic criteria [45]; and recommended quantitative parameters [46]. It consists of 8 items assessing the severity of sleep difficulties and daytime impairment during the past month. Four items assess insomnia symptoms on 5-point scales with quantitative anchors (ie, frequency or duration); for instance, item 1 asks, “How long does it take for you to fall asleep?” and the participants respond on a scale ranging from 0 to 4 (0=0-15 min, 1=16-30 min, 2=31-45 min, 3=46-60 min, and 4=>61 min). The other 4 items assess insomnia symptoms on 5-point scales with qualitative anchors; for instance, item 4 asks, “How do you rate your sleep quality?” and the participants respond on a scale ranging from 0 (very good) to 4 (very bad). The SCI has been validated against the ISI, and it has shown good reliability and convergent validity. The validated Chinese version of the SCI was used in this study [47]. The total score ranges from 0 to 32; higher scores indicate lower levels of insomnia, and a score of ≤16 indicates insomnia disorder. In this study, the SCI had acceptable to good internal consistency, with Cronbach α values in the range of 0.70 to 0.88.

PHQ-9 for Depressive Symptoms

The PHQ-9 was used to assess depressive symptoms [48]. The PHQ-9 consists of 9 items asking participants to rate the frequency of their depressive symptoms in the past 2 weeks on a 4-point Likert scale ranging from 0 (not at all) to 3 (nearly every day). The total score ranges from 0 to 27; higher scores indicate higher levels of depressive symptoms, with a PHQ-9 cutoff score of ≥10 indicating clinically significant depressive symptoms. The validated Chinese version was used in this study [49]. The PHQ-9 showed good internal consistencies across the 3 time points, with Cronbach α values in the range of 0.81 to 0.88.

Generalized Anxiety Disorder-7

The Generalized Anxiety Disorder-7 (GAD-7) was used to measure anxiety symptoms [50]. The GAD-7 consists of 7 items asking participants to rate the frequency of their anxiety symptoms in the past 2 weeks on a 4-point Likert scale ranging from 0 (not at all) to 3 (nearly every day). The composite score ranges from 0 to 21; higher scores indicate higher levels of anxiety symptoms, with a GAD-7 cutoff score of ≥8 indicating clinically significant anxiety symptoms. The validated Chinese version was used in this study [51]. The GAD-7 showed excellent internal consistencies across the 3 time points, with Cronbach α values in the range of 0.92 to 0.95.

Fatigue Assessment Scale

The Fatigue Assessment Scale (FAS) was used to assess fatigue [52]. The FAS consists of 10 items asking participants to rate the frequency of their fatigue symptoms on a 5-point Likert scale ranging from 1 (never) to 5 (always). The total score ranges from 10 to 50, with higher scores indicating higher levels of fatigue. The validated Chinese version was used in this study [53]. In this study, the FAS had good internal consistencies across the 3 time points, with Cronbach α values in the range of 0.88 to 0.90.

Epworth Sleepiness Scale

The validated Chinese version of the Epworth Sleepiness Scale (ESS) was used to assess daytime sleepiness [54]. The ESS consists of 8 items asking participants to rate their chances of dozing in different situations on a 4-point Likert scale, ranging from 0 (never) to 3 (high chance). The composite score ranges from 0 to 24, and higher scores indicate greater daytime sleepiness, with an ESS score of ≥11 indicating excessive or clinically significant daytime sleepiness. The ESS showed good internal consistencies across the 3 time points, with Cronbach α values in the range of 0.82 to 0.83.

Satisfaction With Life Scale

The Satisfaction With Life Scale (SWLS) was used to measure general psychological well-being [55]. The SWLS asks participants to rate their agreement with 5 statements of life satisfaction on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The total score ranges from 5 to 35; higher scores indicate greater psychological well-being. The validated Chinese version was used in this study [56]. The SWLS showed good internal consistencies across the 3 time points, with Cronbach α values in the range of 0.90 to 0.92.

Mechanistic Outcomes

Dysfunctional Beliefs and Attitudes About Sleep-16

The Dysfunctional Beliefs and Attitudes About Sleep-16 (DBAS-16) measures one’s endorsement of dysfunctional thoughts and beliefs that could elevate anxiety and frustration about sleep difficulties, leading to the perpetuation of insomnia [57]; for example, 1 of the items is “I need 8 hours of sleep to feel refreshed and function well during the day.” The DBAS-16 asks participants to rate how much they believe the 16 statements about sleep on an 11-point Likert scale ranging from 0 (strongly disagree) to 10 (strongly agree). The total score ranges from 0 to 160; higher scores indicate stronger dysfunctional beliefs about sleep. The validated Chinese version was used in this study [58]. The DBAS-16 showed good to excellent internal consistencies across the 3 time points, with Cronbach α values in the range of 0.85 to 0.95.

Sleep-Related Behaviors Questionnaire

The Sleep-Related Behaviors Questionnaire (SRBQ) measures one’s engagement in sleep-related safety behaviors, which are behaviors that aim to alleviate the distress and consequences of insomnia but inadvertently perpetuate insomnia [59] (eg, preoccupation with sleep, such as “I spend time considering ways to improve sleep”; and reduced daytime engagement to preserve energy, such as “I take on fewer social commitments”). The validated Chinese version of the SRBQ, which consists of 20 statements, was used [41]. Participants rated these statements on a 5-point Likert scale ranging from 0 (almost never) to 4 (almost always). The composite score ranges from 0 to 80; higher scores indicate greater engagement in sleep-related safety behaviors. The SRBQ had good to excellent internal consistencies across the 3 time points, with Cronbach α values in the range of 0.85 to 0.93.

Treatment Adherence

In this study, video completion and sleep diary completion for each dCBTi session were used as the indicators of treatment adherence, which are common global indicators of treatment adherence that have been used in other CBTi and dCBTi trials [60].

Statistical Analysis

Analyses were performed with R (version 4.0.2; R Foundation for Statistical Computing). All tests for significance were 2-tailed, and P<.05 was considered statistically significant. Intention-to-treat analyses were conducted using linear mixed models with the restricted maximum likelihood method for handling missing data. The restricted maximum likelihood method incorporates the observed data and model covariance structure to estimate the variance parameters in the model with missing data [61]. The models included the treatment groups (dCBTi-therapist, dCBTi-assistant, dCBTi-chatbot, dCBTi-unguided, and dSH), time (baseline, after treatment, and follow-up), and the group-by-time interaction as fixed effects. Participant IDs were included as the random effect in the model. Planned contrasts were specified in the models to test our 4 hypotheses: hypothesis 1, all dCBTi conditions compared to dSH; hypothesis 2, all guided dCBTi conditions compared to unguided dCBTi; hypothesis 3, dCBTi with human support compared to dCBTi-chatbot; and hypothesis 4, dCBTi-therapist compared to dCBTi-assistant.

Cohen d was calculated from the mean differences between the conditions after treatment to indicate the effect size of each significant effect. Fisher exact tests were conducted to evaluate whether the percentages of participants who completed the video and sleep diary for each session were different across the conditions. In addition, we conducted chi-square tests on remission rates to evaluate the differences in remission rates across the conditions after treatment. Remission was defined as having an ISI score of <10 or an SCI score of >21. We also conducted an analysis on the percentages of participants achieving a clinically meaningful reduction in depressive symptoms (a 5-point reduction in the PHQ-9 score), anxiety symptoms (a 4-point reduction in the GAD-7 score), fatigue (a 4-point reduction in the FAS score), and daytime sleepiness (a 2-point reduction in the ESS score).

Results

Descriptives

Of the 819 individuals who completed the screening survey, 690 (84.2%) were not eligible, declined to participate, or did not complete the baseline measures; thus the final sample consisted of 129 (15.8%) participants (age: mean 34.09, SD 12.05 y). Most of the participants were female (98/129, 76%), had never married (87/129, 67.4%), had completed tertiary education (104/129, 80.6%), and were employed full time (77/129, 59.7%). There were no significant differences in age, marital status, education level, employment status, or monthly household income across treatment conditions (Table 2). Table 3 presents the mean values and SDs of each outcome at 3 time points. No significant differences were observed in all outcomes at baseline. Significant differences in the ISI, SCI, PHQ-9, GAD-7, DBAS-16, and SRBQ scores across the conditions were observed after treatment, favoring the treatment conditions over the control.

Treatment Efficacy

Hypothesis 1: dCBTi, Regardless of the Presence of Coaching, Would Be Efficacious for Improving Insomnia Symptoms Compared to dSH

As shown in Table 4, the condition-by-time interaction effects on the SCI, PHQ-9, GAD-7, FAS, DBAS-16, and SRBQ scores were significant, indicating that participants who received dCBTi had greater improvements in insomnia symptoms measured by the SCI and greater reductions in fatigue, depressive symptoms, anxiety symptoms, dysfunctional thoughts about sleep, and safety behaviors related to sleep than participants who received dSH (Figure 2). At follow-up, significant condition-by-time interaction effects were observed on the SCI, FAS, PHQ-9, SWLS, and SRBQ scores (Table 4), indicating greater improvements in these outcomes experienced by dCBTi recipients compared to dSH recipients (Figure 2). In addition, as shown in Table 5, the remission rate based on the ISI scores was 58% (45/77) in the dCBTi conditions, which was significantly greater than that in the dSH condition (6/27, 22%). The remission rate based on the SCI score was 36% (28/77) in the dCBTi conditions, which was significantly greater than that in the dSH condition (2/27, 7%). No significant differences were observed for the rates of achieving clinically meaningful differences in the PHQ-9, GAD-7, FAS, and ESS scores.

‎

Hypothesis 2: dCBTi With Coaching Would Promote Greater Improvements in Insomnia Symptoms and Greater Treatment Adherence Than dCBTi-Unguided

No significant interaction effects were found on all outcomes after treatment and at follow-up when comparing guided dCBTi and dCBTi-unguided (Table 4), suggesting that adding chatbot-based coaching and human support did not improve treatment efficacy. Similarly, the rates of remission of insomnia and the rates of achieving clinically meaningful changes in the secondary outcomes did not differ significantly between guided dCBTi and dCBTi-unguided (Table 5).

Hypothesis 3: dCBTi With Human Coaching Would Promote Greater Improvements in Insomnia Symptoms and Treatment Adherence Than dCBTi-Chatbot

Significant condition-by-time interaction effects were observed on the FAS and SRBQ scores at follow-up (Table 4), indicating that participants who received dCBTi-therapist or dCBTi-assistant experienced greater reductions in fatigue and sleep-related safety behaviors than those who received dCBTi-chatbot (Figure 2). The rates of remission of insomnia and the rates of achieving clinically meaningful changes in the secondary outcomes did not differ significantly between dCBTi with human support and dCBTi-chatbot (Table 5).

Hypothesis 4: dCBTi-Therapist Would Promote Greater Improvements in Insomnia Symptoms and Greater Treatment Adherence Than dCBTi-Assistant

A significant condition-by-time interaction effect was observed on the FAS scores at follow-up (Table 4), indicating that participants who received dCBTi-therapist experienced greater reductions in fatigue than those who received dCBTi-assistant (Figure 2). In addition, the rate of achieving clinically meaningful changes in the GAD-7 scores was significantly greater in dCBTi-therapist than in dCBTi-assistant (Table 5).

Treatment Adherence

Table 6 presents the results of treatment adherence across the conditions. As expected, participants in dCBTi-therapist and dCBTi-assistant completed significantly more video sessions than participants in dCBTi-unguided. They also completed more weeks of sleep diaries than participants in dSH. Significant differences in treatment adherence were observed especially in later sessions, with more participants in dCBTi-therapist completing sleep diaries during sessions 4 to 6 compared to those in dCBTi-unguided and dSH.

Discussion

Principal Findings

This work presents the first randomized controlled comparative trial that evaluates the effects of chatbot-based coaching and human support on the treatment efficacy of, and adherence to, dCBTi. We found that participants who received dCBTi had greater improvements in insomnia symptoms (measured using the SCI), mood disturbances, fatigue, and life satisfaction as well as greater reductions in dysfunctional beliefs and safety behaviors related to insomnia than those who received dSH, with medium effect sizes comparable to those in previous studies of dCBTi [62-65]. Most of the improvements in the dCBTi conditions were sustained at 4-week follow-up. Surprisingly, adding chatbot-based coaching and human support did not significantly improve treatment effects on insomnia. Nonetheless, adding human support, especially therapist support, promoted greater improvements in fatigue as well as greater reduction in safety behaviors related to sleep. Adding chatbot-based coaching and human support also improved some indicators of treatment adherence.

Does Fully Automated dCBTi-Unguided Work?

Supporting hypothesis 1, dCBTi delivered by a fully automated mobile app is efficacious for improving insomnia, mood disturbances, fatigue, and quality of life in adults with insomnia, with effect sizes comparable to those of other tested versions of dCBTi [62-65]. Recipients of dCBTi, regardless of having coaching, achieved an average increase of 12% (SD 13.43%) in sleep efficiency after treatment, from 75.4% to 87.4%; noting that ≥85% sleep efficiency is considered remission of insomnia [45]. The remission rate reached 58% (45/77) in dCBTi conditions compared to 22% (6/27) in dSH. This study was one of the few randomized controlled trials of dCBTi conducted in non-Western populations, and this was the only dCBTi mobile app implemented in Cantonese with published efficacy. This study also extended previous findings by showing that dCBTi was also efficacious for reducing dysfunctional beliefs about sleep and maladaptive behaviors related to sleep—the mechanisms theorized to bring about the treatment effects in CBTi. This finding provided even stronger support for dCBTi by showing that it worked in a way that was consistent with the theory.

Unexpectedly, a greater reduction in insomnia symptoms in the dCBTi group was only reflected by the SCI scores but not the ISI scores. The SCI differs from the ISI in that its ratings on sleep difficulties are based on the recommended quantitative criteria from the DSM-5 as opposed to qualitative impressions of insomnia symptom severity. The inconsistent results reflected by the scores on the two scales might suggest that quantitative anchors are more sensitive in detecting changes in insomnia symptoms. Nevertheless, the absence of a treatment effect of dCBTi on the ISI scores in this study differed from the findings of previous studies [62,65,66]. These previous studies used more stringent participant inclusion criteria; for instance, in addition to scoring ≥10 on the ISI, participants had to meet the duration (≥3 mo) and frequency (≥3 d/wk) diagnostic criteria of insomnia disorder. Participants in these studies might have had more chronic and severe insomnia to begin with and hence experienced greater improvements. Indeed, the ISI scores were 17 [65] and 19 [62] in prior studies and 15 in our sample. Our sample might have also included individuals with acute insomnia. As acute and subclinical insomnia could predict chronic insomnia and depressive episodes [67,68], the evidence for the efficacy of dCBTi for this group of participants with a potentially wider range of symptom duration and severity adds confidence for the impact of dCBTi at the population level where people with differing symptom duration and severity could benefit from dCBTi.

Does Adding Chatbot-Based Coaching and Human Support Improve dCBTi?

Partially consistent with hypothesis 2, adding coaching support improved treatment adherence to dCBTi but not efficacy. Our findings suggested that both human-assisted guidance and chatbot-based coaching were useful strategies to enhance engagement in the middle and late stages of dCBTi. The treatment adherence rates in the guided dCBTi conditions were double those of dCBTi-unguided. Among all types of guided dCBTi, dCBTi-therapist had the highest adherence rate, followed by dCBTi-assistant and dCBTi-chatbot. However, increased adherence rates were not associated with greater efficacy. Consistent with previous meta-analyses [24,69,70] showing that dCBTi was not inferior to face-to-face CBTi for alleviating insomnia, this study did not find any meaningful differences (a change of >4 points in the ISI total score) between guided CBTi conditions and dCBTi-unguided. This study was the first direct comparison of guided CBTi and dCBTi-unguided with different types of guidance and provided primary evidence indicating that adding either therapist or research assistant support does not promote meaningfully greater treatment efficacy.

It is possible that the high degrees of personalization offered by dCBTi might have minimized the benefit of coaching support on treatment efficacy; for instance, in the dCBTi-unguided condition, participants still received a tailored sleep schedule suggestion based on their diary-reported sleep data in the previous week. They were also prompted to set up individualized weekly goals and action plans. With mobile technology, even dCBTi-unguided could deliver tailored treatment recommendations, which is one of the promising benefits of dCBTi. Furthermore, our sample might have included participants with acute insomnia, and the insomnia symptoms experienced by these individuals might not necessitate coaching support. In addition, our sample is overrepresented by highly educated young adults (104/129, 80.6%) with few psychiatric comorbidities. Coaching support may not be most needed for this population; adding therapist support may be beneficial specifically for patients with psychiatric comorbidities [71]. Nevertheless, the differences in adherence between guided dCBTi and dCBTi-unguided reflected the utility of coaching support for enhancing engagement and potentially reducing early dropouts and motivational barriers.

Similarly, partially consistent with hypothesis 3, adding human support did not promote greater improvements in insomnia and most outcomes compared to dCBTi-chatbot. However, greater improvements in fatigue and greater reductions in safety behaviors related to sleep were observed in dCBTi with human support compared to dCBTi-chatbot. These incremental benefits might promote greater or more sustained improvements in sleep and well-being in the long term because reduced fatigue and safety behaviors related to sleep could potentially enhance the maintenance of positive changes resulting from dCBTi, such as maintaining adequate daytime activities and inhibiting anxiety and frustration about sleep. Indeed, as shown in a previous study, adding human support to a self-help CBTi did not lead to greater improvements in insomnia symptoms after treatment and at 4-week follow-up; however, the incremental improvements appeared later at the 3-month follow-up [29].

Is Support From a Therapist Better Than That From a Research Assistant?

Inconsistent with hypothesis 4, support from a therapist did not promote superior treatment efficacy for most outcomes or superior treatment adherence compared to support from a research assistant. While this study was the first to directly compare human therapist support and research assistant support in dCBTi, our results converged with a prior study on digital intervention for depression to suggest that treatment efficacy was comparable between therapist-guided and nontherapist-guided digital interventions [72]. However, it should be noted that the dCBTi-assistant telephone call completion rate was much lower than the dCBTi-therapist telephone call rate, suggesting that therapist support was much more welcomed by the participants in comparison. The lack of differences between these two conditions could be explained by the lack of statistical power for detecting smaller effects. There were no prior data on the expected effect size for the difference between dCBTi-therapist and dCBTi-assistant. This study could have missed smaller effects between these two conditions. Indeed, as shown in Table 4, differences between dCBTi-therapist and dCBTi-assistant amounting to small effect sizes were observed in the SCI, PHQ-9, GAD-7, FAS, and SRBQ scores. Future studies with larger sample sizes statistically powered to detect small effects are needed to further elucidate whether therapist support promotes incremental treatment efficacy.

Limitations

Our findings need to be interpreted in light of the following limitations. First, our sample consisted of mostly highly educated young adults (104/129, 80.6%); therefore, the findings may not be generalizable to other populations. While dCBTi was found to be effective across demographic groups [67], all studies were conducted in samples of working-age adults [68]. It remains unclear whether older adults also respond as well to dCBTi. Second, although this study was adequately powered for detecting meaningful differences in the ISI scores, it could not detect smaller effects that might have existed in the comparison between dCBTi conditions with different coaching types. Nonetheless, we argue that such differences would have limited practical and clinical implications. Third, the follow-up assessment was conducted 4 weeks after treatment, thereby limiting any conclusions that could be drawn about the long-term efficacy of the dCBTi intervention. In particular, although we did not find significant meaningful differences in the primary outcomes between dCBTi conditions with different types of coaching at short-term follow-up, dCBTi with coaching, especially dCBTi-therapist, performed better than dCBTi without coaching on the mechanism of action, that is, sleep-related safety behaviors. This greater improvement in the mechanism of action may promote incremental benefits on the primary outcome that appear at a longer follow-up. Future studies with longer follow-up are necessary to fully evaluate the potential benefits of adding coaching to dCBTi. Finally, we did not collect data on participants’ use of strategies learned in dCBTi. Although adding therapist support improved video session and sleep diary completions, it remains unclear whether the addition of such support increased participants’ use of the learned strategies in their daily lives. More detailed assessments of adherence would provide greater insights into the relationship between treatment adherence and efficacy or the lack thereof.

Conclusions

Our findings supported the efficacy of a fully automated dCBTi intervention, Sleep Sensei, compared to an active control for treating insomnia, reducing mood disturbances and fatigue, and improving quality of life. Adding chatbot-based coaching and human support did not significantly improve the efficacy of Sleep Sensei for treating insomnia, but doing so may improve long-term efficacy, given their effects on increasing treatment adherence and additional benefits on reducing fatigue and behaviors that could perpetuate insomnia. In sum, Sleep Sensei can be used as a stand-alone intervention for treating insomnia and is the only Cantonese mobile app for CBTi published with demonstrated efficacy.