This cross-sectional comparative study sought to assess the effectiveness of community implementation of a supportive SMS text message intervention program focused on reducing symptoms of stress, anxiety, and depression during the COVID-19 pandemic. The study protocol [28] was approved by the Research and Ethics Board of the University of Alberta (Pro00086163).

In the Text4Hope program [29], individuals self-subscribe to receive daily supportive SMS text messages for three months by texting the word “COVID19HOPE” to a short code number. The messages are aligned with a cognitive behavioral framework, with content written by mental health professionals and coauthors of this paper (VIOA, MH). The messages were uploaded to a web-based platform, which delivered messages at 9 AM each day. The first message welcomed subscribers to the service and invited them to complete a web-based baseline survey that captured demographic and clinical information. At 6 weeks, subscribers were invited again via a text message link to complete a web-based follow-up survey. At baseline and at 6 weeks, we collected clinical information on stress, anxiety, and depression about each subject based on the 10-Item Perceived Stress Scale (PSS-10) [30], Generalized Anxiety Disorder-7 (GAD-7) scale [31], and the Patient Health Questionnaire-9 (PHQ-9) [32], respectively.

We were able to cross-reference clinical and demographic responses from individuals by asking clients to enter the mobile number they used for Text4Hope at the baseline and 6-week time points. No incentives were offered to respondents. Participation in the program was voluntary, and completing the survey was not required to receive the supportive SMS text messages. Subscribers could opt out at any time by texting “STOP” to the same sort code number used to enroll in the program. Baseline data collection occurred between March 23 and 30, 2020, and the sixth week follow-up data were collected between May 3 and 11, 2020. Figure 1 presents the subscriber flowchart, which indicates the number of subscribers who completed the web-based surveys at each time point. Furthermore, on May 11, 2020, there were 45,775 subscribers to the Text4Hope program, of which 6178 subscribers opted out of the program, giving a dropout rate of 13.5%.

Primary outcomes included the mean differences in scores on the PSS-10, GAD-7, and PHQ-9 scales at the sixth week versus baseline and the changes in the prevalence rates of self-reported moderate or high stress, likely generalized anxiety disorder (GAD), and likely major depressive disorder (MDD) at the sixth week from baseline.

In a sixth-week evaluation report, 77% of subscribers to the related Text4Mood program indicated that the daily supportive text messages helped them to manage their depression and anxiety [24], which informed our decision to evaluate the Text4Hope program at the sixth week and to determine if subscribers generally had reduced anxiety and depression. Furthermore, a randomized controlled trial of daily supportive text messaging resulted in close to 25% additional improvement in mood (measured by the Beck Depression Inventory [BDI]) in the intervention group compared to the control group [23]. On this basis, we hypothesized that the Text4Hope intervention would result in >25% reduction in mean scores and prevalence rates in all 3 factors, the PSS-10, GAD-7, and PHQ-9 scales, at the sixth week versus baseline.

With a projection that daily supportive text messages would result in a 25% reduction in mean PSS-10, GAD-7, and PHQ-9 scores at the sixth week from baseline, a population variance of 5.0 for each scale mean score, a one-sided significance level α=.05, and an acceptable difference between sample mean and population mean score for each scale of zero (μ – μ0 = 0), we estimated that a sample size of 686 would be sufficient to detect mean differences between the baseline and 6-week PSS-10, GAD-7, and PHQ-9 scores with a power of 80% (β=.2).

Data analysis was undertaken using SPSS for Windows version 26 (IBM Corporation) [33]. To assess the primary outcome measures for our intervention, we used the paired t test to assess the mean difference between the mean PSS-10, GAD-7, and PHQ-9 scale scores at baseline and the sixth week for subscribers who completed the instruments at both time points. In addition, we used the chi-square test to compare prevalence rates for perceived stress, likely GAD, and likely MDD at baseline and the sixth week. Moderate or high perceived stress, likely GAD, and likely MDD were assessed using cutoff scores of ≥14 on the PSS-10 [30], ≥10 on the GAD-7 [31], and ≥10 on the PHQ-9 [32], respectively. There was no imputation for missing data, and the total numbers reported represent the total responses recorded for each variable.

Of the 766 individuals who completed both the baseline and 6-week surveys, 73 (9.6%) identified as male, 678 (88.7%) identified as female, and 13 (1.7%) identified as other gender. Table 1 provides the distribution of the demographic characteristics by gender of subscribers who completed both the baseline and sixth week surveys. Table 1 summarizes the demographic characteristics of the respondents who completed both the baseline and 6-week surveys as n (%). The data presented in Table 1 suggest that the majority of the respondents were aged between 26 and 60 years (601/758, 79.3%), White (656/766, 85.9%), had a postsecondary education (678/764, 88.7%), were employed (555/764, 72.6%), were married, cohabiting, or partnered (507/763, 66.4%), and owned homes (521/760, 68.6%).

Table 2 presents the changes in primary outcome measures after 6 weeks from baseline for subscribers who completed both the baseline and 6-week surveys. The data displayed in Table 2 indicate that for subscribers who completed both the baseline and 6-week surveys, the mean scores on the PSS-10 and GAD-7 scales were significantly lower at 6 weeks compared to the mean scores at baseline, suggesting improvement in stress and anxiety symptoms. The effect size as measured by Cohen d was small (<0.5) for both the stress and anxiety scales.

There was a reduction in the mean score on the GAD-7 scale of 19.0% at the sixth week compared to the baseline scores. The reduction in the PSS-10 scores at six weeks compared to the baseline scores, although statistically significant, was much smaller (4.1%). There was no statistically significant within-subjects difference between the baseline and sixth week PHQ-9 mean scores (P>.05).

Table 3 indicates that there were statistically significant reductions in the prevalence rates of moderate or high stress and likely GAD but not of likely MDD when comparing the baseline and 6-week assessments. The largest reduction in prevalence rates was for anxiety (13.5%).

To assess the generalizability of our data, based on the mental health burden in our baseline samples, we examined the clinical parameters between people who only responded to the baseline survey versus those who responded to both surveys (baseline and sixth week) (Table 4 and Table 5). No statistical difference was elicited between the two groups (all P>.05), suggesting that at baseline, the mental health burden was similar between our study sample and subscribers who did not complete the 6-week survey.

Similarly, we examined the clinical parameters between subscribers who responded to the 6-week survey only and subscribers who responded to both surveys (Table 6 and Table 7). No statistical difference was elicited in prevalence of stress, anxiety, or depression symptoms between the two groups (P>.05), suggesting that after receiving the intervention for 6 weeks, the mental health burden was similar between our study sample and subscribers who only completed the 6-week survey.

The Text4Hope program was provided as an intervention tool for the general population to support the mental well-being of individuals living in the Canadian province of Alberta during the global COVID-19 pandemic. Other technology-based interfaces have been deployed during the COVID-19 pandemic to track the disease spread in populations [34], to gather data related to the general knowledge, attitudes, and behavior of the public related to the pandemic [35,36], and to offer mental health support to the public during the pandemic [37-40]. To the best of our knowledge, this is the first study to assess the impact of a text-messaging intervention on self-reported symptoms of stress, anxiety, and depression experienced during the COVID-19 pandemic. Our study yielded interesting results regarding temporal changes in the self-reported severity and rates of symptomatology related to the three psychiatric health conditions under study. After receiving daily messages for 6 weeks, we observed significant reductions in the respondents’ mean scores on the GAD-7 (18.7%) and the PSS-10 (4.0%), suggesting that the program was effective in reducing anxiety and stress symptomatology in the respondents. There was no significant reduction in the mean PHQ-9 score at 6 weeks from baseline. In terms of prevalence rates, the largest significant reduction in prevalence rate was for likely GAD (13.5%), followed by moderate or high stress (4.1%). Again, there was no significant reduction in the prevalence rate of likely MDD at 6 weeks from baseline.

The self-reported rates of anxiety symptoms in our study at baseline were higher than those reported in other studies [41,42]. However, the greatest improvement recorded after the provision of the Text4Hope program was for anxiety, with a 13.5% reduction in symptom prevalence rate and 19% improvement in GAD-7 score. Text4Hope achieved a small Cohen d effect size (0.4) in mitigating anxiety symptoms, which is comparable to the effect sizes found for an internet CBT program aimed at reducing anxiety symptoms [43]. Typically, interventions that do not include therapist support demonstrate lower effect size outcomes compared to those including therapists [44,45]. By contrast, our intervention reached thousands of individuals during the pandemic, and it aimed to provide a general population intervention rather than individual psychotherapy. The overall change in GAD-7 scores in our study (–1.8) appears to be consistent with the magnitude of score changes recorded after providing other remote health services. For example, adding a telephone service to computerized CBT in combating anxiety yielded a reduction of 1.18 in GAD-7 scores [46]. Again, the percentage of change in GAD-7 scores in our study after providing a daily text message for 6 weeks (19.0%) is consistent with the effect of medications on anxiety symptoms; in a very large randomized controlled trial in the United Kingdom, sertraline was evidently effective in reducing GAD-7 scores by 21% after 6 weeks, and this result was described as clinically important [47].

Our findings indicate that there was a modest effect of the program on improving stress symptoms, with greater benefit than other internet-based cognitive behavioral theory (iCBT) platforms [48]. In a Japanese study, iCBT was used to alleviate anxiety, stress, and depressive symptoms in university students [48]. The most significant effect was mainly reported for anxiety, while stress symptoms did not show a difference between case and control group members after the intervention period [48].

There was no significant change between baseline and 6-week mean scores for likely MDD. Comparing our results with other remotely delivered health services yielded variable results. Two meta-analyses found that the effectiveness of iCBT programs, including MoodGYM, in mitigating depressive symptoms showed small effect sizes, especially in short-term assessments [43,44,49]. On the other hand, a significant improvement in BDI-II score with moderate effect size was observed at 3 months in patients with depression and comorbid alcohol use disorder who received supportive SMS text messages twice daily compared to the control group, who only received a thank-you SMS text message every fortnight [23]. The Text4Hope program was primarily designed as a health promotion tool to support the general population in Alberta during the COVID-19 pandemic and to combat potential stress and anxiety symptoms that are usually associated with epidemics or global crises. Our study participants were members of the general population rather than a patient sample, which may account for the observed differences in results. Furthermore, in a previous randomized trial [23], participants received the intervention for 12 weeks compared to 6 weeks in our study, which may account for the observed differences in the effects. Another study examined the effect of sertraline, an antidepressant of the selective serotonin reuptake inhibitor class, in ameliorating depressive symptoms; this study reported only a 5% relative reduction (95% CI 7%-15%; P=.41) in the mean PHQ-9 score at week 6 [47], which is not vastly different from the apparent 2% improvement observed with our Text4Hope intervention.

Three months after the launch of the Text4Hope program, the dropout rate was 13.5%. A high withdrawal rate is not uncommon for a texting service provided via SMS. When Bendsten and Bendsten [50] compared an SMS texting service to services provided via email, they found that people in the SMS group opted out at significantly higher rates than those in the email group (20.1% versus 5.2%, respectively). Additionally, in a review study on behavioral changing interventions provided via SMS text message, authors reported a wide range of withdrawal rates (0%-57%) among participants. [51]. They justified this result as being due to the untailored and unilateral nature of these texting programs, which may be less engaging and therefore may result in low retention rates [51].

Our study has several limitations. For ethical reasons, we lacked a comparative control group that did not receive the Text4Hope intervention during the same phase of the pandemic against which the recorded changes in stress, anxiety, and depression levels could be compared. It is therefore possible that the reductions in stress, anxiety, and depression levels are not all attributable to the Text4Hope intervention. Second, we relied on self-rated scales to assess stress, anxiety, and depression symptomatology, which could potentially overestimate the levels of these mental disorders when compared with prevalence rates that would have been obtained using structured clinical interviews with the Diagnostic and Statistical Manual of Mental Disorders 5th Edition. Third, our results may not be generalizable to the general population and are at risk of participation bias, where individuals with pre-existing mental health conditions are characteristically more inclined to enroll in the Text4Hope program compared with individuals with no pre-existing mental health disorders. We did not ask subscribers about pre-existing mental disorders, which would have helped to distinguish new symptoms from pre-existing ones but may have resulted in limited enrolment; in our experience, subtle changes in signup processes for subscribers can result in marked decreases in participation. Finally, the sample size of subscribers who completed both the baseline and 6-week assessments was rather small, and it is possible that other subscribers had variable changes in their mental health parameters from baseline to the sixth week. This limitation notwithstanding, our sample size was larger than the projected 686 subscribers needed to detect mean differences between the baseline and 6-week PSS-10, GAD-7, and PHQ-9 scores with a power of 80%. In addition, the mental health burden in our sample at baseline and at the sixth week were not significantly different from those of subscribers who only completed the baseline survey and the subscribers who completed the 6-week survey, respectively (Tables 4 and 5). Furthermore, the demographic characteristics of subscribers who completed both the baseline and 6-week surveys mirror those of all 8267 subscribers who completed the baseline survey by July 12, 2020 [52]. Specifically, the proportions of the various demographic characteristics in our sample compared to the proportions of the same demographics in the larger sample of 8267 subscribers was as follows: female gender, 88.7% versus 87.1%, respectively; White, 85.9% versus 82.3%, respectively; postsecondary education, 88.7% versus 85.2%, respectively; employed, 72.6% versus 73.3%, respectively; married, cohabiting, or partnered, 66.4% versus 71.1%, respectively; and homeowner, 68.6% versus 65.9%, respectively [52]. These proportions support the generalizability of our results to all subscribers. Future studies using this style of intervention could attempt to minimize attrition by offering incentives for participation or by sending messages encouraging people to continue subscribing.

Finally, the effect sizes in our study were relatively small, which may minimize the strength of the produced results. However, interventions that do not include therapists often report low effect sizes compared to those including therapists [44,45].

The Text4Hope program resulted in statistically significant reductions in mean scores on the PSS-10 and GAD-7 scales but not the PHQ-9 scale at the sixth week from baseline. The program also resulted in statistically significant reductions in subscribers’ prevalence rates of moderate or high stress and likely GAD but not of likely MDD. The largest reductions in the mean scores and prevalence rates were observed for anxiety symptoms. It should be noted this paper reports data from the midpoint of the Text4Hope program implementation, and the rates of change for outcomes for stress, anxiety, and depression may differ after the program ends at 3 months.

The relatively large improvements in anxiety symptoms achieved in our sample after 6 weeks of receiving the intervention during the COVID-19 pandemic suggest that the Text4Hope program is a useful intervention that can be deployed during natural and humanitarian disasters to support individuals at the population level. Over half of Canadians have reported that their mental health needs are not fully met [53]. A commonly reported reason is the cost of services [53]. As such, free mobile-based services such as Text4Hope can help address financial barriers. In the Canadian context and in other global contexts, these services can also be delivered remotely, which helps maintain essential physical distancing requirements during pandemics but also provides a means of access for those at rural or remote locations with little or no capacity for accessing mental health support services. We did not differentiate urban from rural or remote subscribers, as the program was offered to everyone in the province; however, regardless of geographical region, the pattern of urban, rural, and remote subscribers’ locations is of interest in relation to understanding the full value of SMS text messaging in this context. This is an additional step that we are considering in evaluating this program approach.