Following an uncontrolled feasibility pilot study with 189 participants (S. Buntzel, unpublished data, 2018), we conducted a randomized controlled trial comparing an intervention group with access to a mobile CBT-I intervention with a waiting list control group (Multimedia Appendix 1). Outcomes were assessed through online questionnaires at baseline (pretreatment), postintervention (8 weeks after randomization), and 6-month follow-up. The primary outcome was the change in sleep quality (insomnia symptoms and Regensburg Insomnia Scale [RIS] total score) at postassessment compared with the baseline assessment.

Participants were randomized in a 1:1 ratio. The randomization was block stratified by gender (female vs male); severity of insomnia symptoms, poor (RIS total score ≥13 [72]) versus good sleep (RIS total score <13); and the consumption of sleep-inducing drugs (yes vs no). The random allocation sequence was implemented on the data collection platform.

Participants received information about their group assignment by email. Participants in the intervention group had immediate access to the intervention. By contrast, participants in the waiting list group received access to Refresh after the 6-month follow-up had been completed if they were still interested in using the intervention. Participants in both groups were invited to the post- and follow-up assessments via emails. They received up to 5 reminders at intervals of 3 days.

The study was approved by the Ethics Committee (Institutional Review Board) at Technische Universität Dresden (reference number EK 111032919) and registered prospectively in the ISRCTN database (registration number ISRCTN53553517).

We included participants over 18 years of age who were fluent in German and had access to the internet during the intervention period. Exclusion criteria were (1) current treatment for depression, (2) a history of psychotic or bipolar disorders, and (3) a suicidal ideation according to the answer in the last item of the 9-item Patient Health Questionnaire (PHQ-9). Poor sleep was not necessary to participate in the study. All adults showing interest in improving their sleep were welcome if none of the exclusion criteria were present.

Participants were recruited through flyers, postcards, and posters in several sleep laboratories in Germany as well as in practices of general practitioners, pharmacies, and medical supply stores in Dresden. In addition, recruitment activities took place at Technische Universität Dresden and included a press release, student email newsletters as well as postcards and posters distributed on the campus. A newspaper article was published in a regional newspaper. Social media channels (eg, Facebook groups dealing with sleep issues) were used to address further potential participants. The recruitment material included a link and a QR code leading to the study website. Interested participants received written information about the study and informed consent was obtained online. Study data were collected and managed using REDCap (Research Electronic Data Capture; Vanderbilt University) electronic data capture tools hosted at the Centre for Clinical Studies at Technische Universität Dresden. REDCap [73] is a secure, web-based application designed to support data capture for research studies, which provides (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources.

Refresh is an unguided, fully automated app-based intervention adapted and translated from an e-mail-delivered CBT for sleep health program for college students, which was originally developed and evaluated in the United States [74], and subsequently translated and evaluated in a Japanese college student population [75]. The intervention was adapted for the general adult population and designed as an app-based self-help intervention. Refresh was implemented using a commercially available eHealth platform [76]. Some screenshots of Refresh are provided in Multimedia Appendix 2.

The intervention consists of 8 consecutive chapters that can be completed in about 10 minutes each and cover all CBT-I components recommended by the German Sleep Society (DGSM): psychoeducation, sleep hygiene, stimulus control, sleep restriction, cognitive restructuring, and relaxation [30]. Table 1 summarizes the content of the individual chapters.

The chapters are multimedia based with short text passages, audio and video content, vignettes, and questions (multiple choice or free text) to foster an active engagement with the intervention content as well as to tailor subsequent content based on the participants’ preference (eg, whether they preferred information to be presented as text or video). The number of pages per chapter and the amount of plain text per page were kept to a minimum as factors such as extensive text content and text content complexity are likely to increase the risk of nonadherence [78]. The intervention is supplemented by a “30-second sleep diary” to be filled in every morning to monitor insomnia symptoms. For a duration of 8 weeks, an automated reminder to fill in the sleep diary was sent every morning at 7 AM. There were no reminders for progressing through the intervention. At the beginning, participants received a short automated feedback about their sleep based on the baseline RIS score. Participants reporting poor sleep were especially encouraged to use parts of the intervention that were marked as for “people with poor sleep” (eg, sleep restriction).

The intervention is self-paced, but the recommended duration was 8 weeks with 1 chapter per week.

Based on results from the pilot study (S. Buntzel, unpublished data, 2018), we assumed a pre-post between-group effect size (Cohen d) of 0.30 and a dropout rate of 60% at postintervention. To detect the anticipated effect with an 80% probability at a significance level of 5%, a sample size of 586 participants was required.

All statistical analyses were performed with SPSS Statistics version 27 (IBM Corp). Statistical significance was set at α=.05. Differences in baseline sociodemographic and clinical scores between the intervention group and the control group as well as between participants with poor and good sleep were analyzed using Pearson chi-square tests for dichotomous variables and independent sample t tests (2-sided, unpaired) for metric variables. Differences between completers and noncompleters were analyzed in the same way.

Adherence and acceptance data were analyzed descriptively and compared between participants with poor and good sleep by applying independent sample t tests. To identify potential predictors of adherence, Pearson correlations were calculated between adherence markers (number of chapters completed and number of sleep diary entries) and baseline variables.

The primary (RIS) and secondary (perceived insomnia-related impairment and PHQ-9 score) outcomes were analyzed using linear repeated-measures mixed-effect models with restricted maximum likelihood estimation and an unstructured covariance matrix. This method follows the intention-to-treat (ITT) approach and is recommended for randomized controlled trials with missing data [85]. Group, time, and interaction of group × time were entered as fixed variables with group as a between-group variable and time as a within-group variable. The outcomes RIS, perceived insomnia-related impairment, and PHQ-9 scores were entered as dependent variables in separate analyses. Within-group effect sizes (Cohen d) were calculated by dividing the estimated mean change from baseline to postassessment (or follow-up) by the pooled SD [86]. Between-group effect sizes (Cohen d) were computed based on the difference of the estimated mean change from baseline to postassessment (or follow-up) in the intervention group compared with the control group divided by the pooled SD at baseline as recommended by Morris [87].

The analyses for the whole sample were repeated including only participants with poor sleep at baseline as indicated by an RIS score of ≥13 at baseline. Improvements in insomnia symptoms were tested for clinical relevance by comparing the frequency of participants with poor sleep (RIS ≥13) at baseline in both study arms who had improved at postintervention and at follow-up using Pearson chi-square tests for assessment completers. In addition, the odds of suffering from poor sleep (RIS ≥13) at postintervention were compared between the 2 groups using a more conservative ITT approach including all randomized participants with poor sleep. We used a logistic regression model and adjusted for sex, age, and RIS score at baseline. Data on participants who did not complete postassessment questionnaires were imputed by assuming poor sleep at postintervention.

The recruitment period and baseline assessments ran from April 2019 to May 2020. Follow-up data collection was completed in September 2020. Given that the pre-post dropout rate was much smaller than expected, we included fewer participants than originally planned. A total of 393 completed the baseline assessment, of which 371 were randomized (Figure 1). About two-thirds of participants (n=245, 66.0%) reported poor sleep according to an RIS total score of ≥13.

A total of 250 participants (67.4%) provided the primary outcome (RIS total score) at postassessment and 214 (57.7%) completed the RIS at the follow-up assessment. Dropout was higher in the intervention group than in the control group at both assessment points (χ²_1post=24.477; P<.001 and χ²_1FU6=28.247; P<.001). Participants with poor initial sleep did not differ from those with good sleep in dropout rates. There also were no differences in any of the baseline measures between study dropouts and completers at postassessment. At follow-up, women (χ²₁=5.813; P=.02) and participants with a higher level of education (χ²₁=7.832; P=.005) were more likely to complete the assessment. In the subgroup of participants with poor sleep, education also predicted completion at postassessment (χ²₁=5.163; P=.02).

The majority of participants in the full sample were women (223/371, 62.8%), in a relationship (271/371, 73.0%), and reported a higher level of education (324/371, 87.3%; Table 2). About 1 in 4 had children and about 1 in 3 were university students. The mean age of participants was 37.3 (SD 14.24) years. Sleep was poor on average (indicated by an RIS score ≥13) and the average level of depression severity was mild.

The subgroups of participants with poor or good sleep differed significantly in most of the sociodemographic variables and in the clinical scores (Table 2). For example, participants with poor sleep were significantly older (P<.001), were more likely to have children (P<.001), and more likely to be self-employed (P=.04). They also reported more severe depressive symptoms (P<.001) than those with good sleep and were more likely to receive treatment (P<.001) or use medication for their sleep problems (P<.001).

There were no significant differences between the intervention group and the control group in sociodemographic characteristics, clinical scores (see Multimedia Appendix 3), or the proportion of participants with poor sleep (117/186, 62.9% vs 128/185, 69.2%, respectively; χ²₁=1.634; P=.23.

A total of 60/186 (32.3%) participants never logged into the platform or opened the first chapter. Participants who logged into the intervention at least once (n=126) completed on average 4 chapters (SD 3.06; range 0-8) and opened 42/83 (51%) intervention pages (SD 38.14). A total of 39/126 (31.0%) participants completed all chapters of the intervention and 55/126 (43.7%) completed at least half of the intervention; 116/126 (92.1%) completed at least chapter 1 (Figure 2). A total of 106/126 participants (84.1%) used the sleep diary at least once and made entries for 25.23 days (SD 22.3; range 0-83).

Male participants were more likely to not log into the intervention at all (30/65, 46.2% vs 30/121, 24.8%; χ²₁=8.829; P=.003). Participants with children made fewer entries in the sleep diary than those without (P=.02). None of the other sociodemographic variables were predictive for the number of chapters completed or the number of sleep diary entries, and also both RIS total score and PHQ-9 total score at baseline did not predict adherence to the intervention.

As many as 87 participants from the intervention group provided WAI-SR data that allowed calculating the score of the Task and Goal subscales. The participants agreed between “sometimes” and “fairly often” with the tasks of the intervention and between “fairly often” and “often” with the goals of the intervention. Acceptance did not differ between participants with poor and good sleep at baseline (Table 3).

Participants who answered the session rating of the intervention chapters rated chapters 1-6 as “good” on average, whereas chapters 7 and 8 were rated as “moderate.” There was no significant difference in session ratings between participants with poor and good sleep, except for session 2 that was rated better and nearly “excellent” by participants with good sleep (P<.001).

Results of the mixed model analyses are presented in Table 4. For the full sample, a significant group × time interaction was found for the RIS at postassessment (F_1,269.31=11.93; P=.001) with a small between-group effect (d=0.26). Participants in the intervention group showed a stronger reduction (mean difference = –2.66; d_within=0.42) in the RIS scores than those in the control group (mean difference = –1.04; d_within=.17). At follow-up, the group × time interaction failed to reach significance (F_1,227.63=3.54; P=.06). Within-group improvements in the perceived insomnia-related impairment due to sleep problems as well as in the PHQ-9 did not differ significantly between the study arms (Table 3).

In the subgroup analyses including only participants with poor sleep at baseline, a significant group × time interaction was found for the RIS at postassessment (F_1,180.66=12.59; P<.001; d=0.47) and for the perceived insomnia-related impairment at postassessment (F_1,184.45=6.69; P=.01; d=0.45) as well as for the RIS at the follow-up assessment (F_1,147.56=4.60; P=.03; d=0.33). Although the intervention group showed somewhat larger reductions in the PHQ-9 than the control group, the difference was not significant (P_post=.13; P_FU6=.15; also see Table 4).

To assess clinically relevant improvements, we calculated how many participants with poor sleep at baseline had reduced their RIS score below 13 points at postintervention and follow-up assessments. At the postintervention assessment, this was true for 25/69 (36%) assessment completers in the intervention group and 16/102 (15.7%) assessment completers in the control group. This difference was statistically significant (χ²₁=9.531; P=.002). At follow-up, there was no significant difference between the groups (21/53 IG completers vs 25/90 CG completers; χ²₁=2.145; P=.14). In the ITT analysis of clinically relevant improvements, the odds of suffering from poor sleep at postintervention were also significantly larger in the control group than in the intervention group (odds ratio 0.462, 95% CI 0.218-0.976; P=.02).

To assess clinically relevant deterioration, we assessed how many participants with good sleep at baseline had increased their RIS score to 13 points or more at postintervention and follow-up assessments. At postassessment, this was true for 1/34 (3%) assessment completers in the intervention group and 10/45 (22%) assessment completers in the control group. This difference was significant (χ²₁=6.007; P=.01). At follow-up, there was no significant difference between the groups (2/29 IG completers vs 2/42 CG completers; χ²₁=0.147; P=.71).

The aim of this study was to evaluate an unguided app-based training to improve sleep quality (insomnia symptoms) in individuals who wish to improve their sleep compared with a waitlist control group. Specifically, we examined participant adherence to the intervention; working alliance and intervention acceptance; and intervention effects on sleep quality (insomnia symptoms), insomnia-related impairment, and depression symptoms.

Nonusage attrition was comparable to other digital interventions [88], with a larger loss of participants during the early intervention phase and about 1 in 3 participants completing the intervention. About 1 in 3 participants who were allocated to the intervention group never accessed the intervention. Failure to commence the intervention was 2 times as likely in men, with 1 in 2 never accessing the intervention. The proportion of participants who never started the intervention was markedly lower in a previous study investigating an unguided mobile CBT-I–based intervention [65]. However, we applied broader inclusion criteria compared with [65], that is, we also included individuals reporting slight insomnia symptoms or symptoms below the cutoff (RIS <13). Because our sample was healthier, it can be assumed that the insomnia-related distress of these participants was lower and may have resulted in a lower motivation to start the intervention after completing the baseline assessment. Furthermore, we did not exclude participants with depression. Evidence suggests that adherence to an internet-based CBT-I program is reduced if participants experience other psychiatric or medical problems next to insomnia [89].

Because of the heterogeneity in adherence markers reported (if any were reported), adherence can be compared in detail only with 2 other recent clinical trials [66,90] that investigated unguided digital CBT-I interventions. Participants, on average, completed 4 out of 8 chapters in our study, which is comparable with adherence in another trial, in which participants completed 3.4 out of 6 modules [90]. However, in Lorenz et al [66], adherence was higher, with participants completing 5.6 out of 6 treatment sessions. In our study participants completed the sleep diary, on average, on 25 out of 56 days (45%), whereas they did so for 86% of days in one of the other trials [66].

Participants in the intervention group were able to form a moderate working alliance with the unguided intervention, with a slightly larger concordance regarding goals than regarding tasks. Working alliance in psychotherapy tends to be stronger, but a similar pattern in concordance regarding goals versus tasks is typically also seen in patients undergoing psychotherapy [91], which could be explained by a lesser degree of ambivalence toward treatment goals compared with the actual behavior changes (tasks) that are necessary to achieve them. Working alliance has not been investigated in recent clinical trials investigating unguided digital CBT-I interventions.

The first 6 chapters of the intervention, which provided psychoeducation and various strategies to improve sleep-related habits and behaviors, were on average rated “good,” whereas the final 2 chapters, which mainly focused on reviewing and reinforcing the sleep restriction method and provided fewer new strategies, were rated “moderate.”

Compared with previous studies on digital CBT-I–based interventions, we were able to recruit a sample of participants that was large enough to detect even the small intervention effects that are to be expected in an unguided intervention. The study had a waitlist control condition and a longer follow-up period than 1 of the 2 previous studies investigating a mobile intervention [65]. Other than free access to the intervention, study participants received no incentives for taking part in the study or engaging with the intervention, thus avoiding potential incentive-caused bias that may impact intervention adherence. Thus, we can assume that people who used the intervention intensively had an intrinsic motivation to do so. Despite the lack of incentives, dropout at postintervention (121/371, 32.6%) was smaller than expected from our pilot study and within the range of dropout rates (0%-44%) reported in self-help CBT-I interventions [94]. However, not completing trial assessments (ie, dropout attrition [88]) does not necessarily mean that participants do not use an intervention (ie, nonusage attrition). Some people may be interested in using the intervention, but may not wish to complete assessments. Adherence to the intervention was comparable to other trials on digital interventions, and 1 in 3 participants completed the treatment. However, a substantial number of participants never started the intervention. Prerequisites and time requirements to participate in our study were very low. Although many clinical studies require participants to undergo a longer clinical interview during the screening process, participants in our study only had to fill in a relatively short anonymous online survey. This may have resulted in a larger proportion of participants who were ambivalent about the intervention. Although people could participate in the study regardless of whether they had insomnia symptoms, the proportion of individuals with poor sleep at baseline was high, indicating that the intervention was mostly reaching the users it was designed for.

Limitations of our study were the lack of more specific measures for insomnia-related impairment and socioeconomic consequences, including absenteeism or loss of productivity, as well as an objective measure of sleep quality. However, subjective measures of sleep quality and insomnia symptoms are widely used in CBT-I research and the added value of objective measures is a matter of debate [95]. The participants in this trial might have also suffered from sleep disorders other than insomnia (eg, obstructive sleep apnea, circadian rhythm sleep-wake disorders), in which an intervention covering the principles of CBT-I is likely to have limited or no effect. Future studies may thus benefit from including insomnia diagnosis confirmed by a clinician. Adherence to the intervention was approximated by examining usage data. We did not collect information on whether and how participants in the intervention group implemented the suggested behavior changes. Besides, the study design was not suitable to determine the extent to which the components of the interventions contributed to the effects. Secondary analyses are therefore needed to investigate possible moderators or mediators, which are not the subject of this publication. In accordance with other trials evaluating fully automated internet-based CBT-I, the majority of our sample were highly educated participants who are not representative of the general population. However, even if low education is associated with insomnia-related symptoms [96,97], sleep problems are also frequent in people with a higher level of education. Nevertheless, future research should include a more heterogeneous sample with regard to the socioeconomic level and increase efforts to also reach a lower educated population that might be especially underserved.

In our study, the unguided app-based intervention was associated with a short-term improvement of insomnia symptoms that remained stable over 6 months. Participants in the waitlist condition also reported an improvement in their insomnia symptoms, but this improvement took longer. Effects were more pronounced in individuals with low sleep at baseline. We detected no effect on depression symptoms in our sample, which may largely be due to the low average symptom load at baseline. Although the inclusion criteria for our study were broad, our recruitment strategy mostly reached the individuals the intervention was designed for.

Overall, an anonymous unguided intervention is a feasible option to deliver CBT-I–based treatment to a large number of people, either through self-referral or by primary care providers. Even if the individual effects are smaller than in guided self-help or face-to-face CBT-I, the public health impact could still be profound given the scalability of unguided approaches. Given that very few people have access to face-to-face CBT-I, making it available digitally can increase its availability [98] and public health impact [99]. As in face-to-face CBT-I, some people will not respond to the intervention and precautions must be taken to prevent negative expectations regarding future psychological interventions in those individuals. Symptoms can be continuously measured within the intervention and if participants do not respond, a more intensive treatment can be recommended. Thus, a digital self-help program can serve as a first step of treatment within a stepped care approach [100]. Moderator analyses can contribute to identifying those who will likely benefit from unguided app-based CBT-I and those who need more guidance or face-to-face treatment.

It is noteworthy, however, that 1 in 4 women and 1 in 2 men who were assigned to the intervention group did not start the intervention. In clinical practice, the motivation and ability to adhere to a self-guided intervention should thus be carefully assessed.

In participants who did start the intervention, adherence was comparable to other digital interventions. In future adaptations of the interventions, adherence may be improved by involving a multidisciplinary team of psychologists, user experience specialists, and user experience designers in the design of the user interface, which unfortunately was not possible in our study due to budget constraints. In this context, adherence to the intervention may benefit from persuasive elements such as reminders, automated positive reinforcement, graphical reports, or tailored recommendations. However, such elements need to be implemented in a way that allows users some level of control (eg, options to determine the frequency of reminders [101]). However, adding several persuasive technology features does not necessarily result in better outcomes [102]. Adding some form of human interaction to Refresh could also improve adherence, but this in turn would result in a reduced scalability [103]. Automated feedback that is given when users show low engagement with the intervention may be an alternative [101]. Increased adherence by implementing design features to encourage engagement with the intervention might contribute to more pronounced improvements in insomnia symptoms.