Young people experience a high burden of disease attributable to mental ill-health, with approximately one in four young people (aged 15-24 years) experiencing a mental health problem every year [1]. Unfortunately, the prevalence of psychological distress in young people in Australia appears to be increasing [2], and suicide remains the leading cause of death for Australian youth [3]. Many mental health problems have their onset in adolescence or early adulthood, and if left untreated, can result in more frequent and severe episodes of ill-health throughout the lifespan [4-7]. There is limited capacity within the current mental health service system to meet the demands associated with youth mental health problems in a timely manner, and many young people are going without vital support [8].

Although there have been steady improvements in the provision and uptake of treatment services among young people in the past decade, rates of service utilization among adolescents and young adults remain suboptimal; in the latest Australian Child and Adolescent Survey of Mental Health and Wellbeing, 21.4% of young people aged between 12 and 17 years had accessed a service for emotional or behavioral problems in the past 12 months [8,9]. Furthermore, despite increased provisions and extensive public destigmatization efforts, a significant proportion of young people are reluctant to access community services for their mental health problems [10,11]. Common barriers to help-seeking for young people include limited service availability (particularly in rural areas), accessibility issues (in terms of location and time), financial constraints, concerns about confidentiality, and fear of judgment or stigma [10,12,13]. A desire for autonomy in the help-seeking process and a preference for self-reliance are also salient barriers for young people [10,14,15]. In addition to these common barriers, there are a number of young people who report having unfavorable experiences with mental health professionals and are therefore hesitant to either return or seek further professional help [16]. In the light of these barriers, innovative approaches to reaching and engaging, or reengaging, young people with mental health support are required.

Digital mental health services have the potential to overcome common barriers to help-seeking experienced by young people [17]. Recent research indicates that young people perceive web-based mental health support to have many benefits over traditional face-to-face services, including greater accessibility, immediacy, interactivity, lower financial costs, and reduced embarrassment associated with the experience [18]. The use of internet among young people is prolific [19] and there is growing evidence that young people are turning to digital platforms to access mental health information and support [2]. Australian research indicates that approximately one in five 12 to 17-year-old adolescents, and one in three 18 to 25-year-old adults who have experienced a mental health problem have used internet search engines to access mental health information [20]. A national survey of young Australians conducted in 2018 [2] also found that young people were amenable to using the internet for mental health support. For example, 37% of those with psychological distress said that they would use the internet to source information about specific issues and 22% reported that they would use the internet to access information about available services [2]. Another recent study found that the majority of youth (72%) would access a web-based resource if they were experiencing a mental health problem, and a third (32%) would prefer a web-based resource over face-to-face therapy [18].

The flexible, freely accessible, and multicomponent nature of unstructured digital services may be appealing to young people experiencing issues more broadly; they may also be particularly attractive to young people with mental ill-health who are reluctant to engage with clinical services for several reasons. First, they can be accessed anonymously, whereas most structured programs require users to provide personal information, thereby circumventing concerns about confidentiality [2,10]. Second, they are self-directed and therefore responsive to young people’s preference for autonomy by providing a range of resources to select from [10]. Third, they are able to deliver a service experience that is tailored to the needs and preferences of each individual young person through personalization [21]. Finally, there are many evidence-based digital mental health services that can be accessed for free or at a low cost, with no waiting period [22,23]. Digital services might also act as an alternative or adjunct intervention when young people have had previous negative experiences with mental health professionals. In such cases, digital services may be well-positioned to provide interim support and rebuild service readiness in young people who are in need of more intensive intervention but are dubious about returning to professional services [24].

The growth in the accessibility and uptake of digital technologies offers a cost-efficient and highly scalable alternative to reach young people needing mental health support when compared with traditional models of clinical care [25]. Digital services have the potential to reach and deliver positive outcomes across the mental health spectrum. Moreover, they can be appropriate for young people who experience subthreshold mental health problems and who may be ineligible for other supports, or those who are reluctant to seek professional help owing to minimization and/or self-stigma. Currently, several digital support services exist to support young people in managing their mental health, including web-based counseling services (eg, eheadspace), structured web-based therapies (eg, Moodgym), psychoeducational websites, and web-based blogs and forums [26]. There is also increasing evidence of the effectiveness of web-based counseling, therapies, and mobile self-monitoring tools in improving mental health outcomes and help-seeking [27-33]. Web-based positive psychology interventions have also been linked to increased well-being and decreased stress in nonclinical youth populations [34]. Not only is there evidence of the effectiveness of digital services in the prevention and early intervention space but there is also evidence that mobile apps designed to prevent suicide can bring about reductions in psychological distress among vulnerable populations (eg, Ibobbly, [35]). Moreover, web-based interventions may also prevent relapse in mental illness and support a process of recovery [36,37]. Given the flexibility and versatility of digital media, digital services are well placed to support the broad spectrum of unique mental health issues that young people, both well and unwell, experience.

Although there is substantial evidence for structured and clinician-led digital interventions, limited research has been conducted to investigate the effectiveness of unstructured digital mental health interventions, and there is a lack of ecologically valid research for digital interventions. Although randomized control trials (RCTs) represent the gold standard in research and provide important insights for many treatment outcomes, a reliance on these methods means that much evaluative research of digital programs is conducted under highly controlled and prescriptive conditions that rarely mimic the experience of users in the real world [38]. To adequately evaluate the effectiveness of digital interventions, it is imperative to examine these digital services in the environments in which they are intended to be used [38]. This study aimed to bridge this gap by exploring the impact of one unstructured digital mental health intervention, ReachOut.com (hereafter ReachOut), on mental health outcomes of young people who used it over a 3-month period, in a naturalistic study.

Preliminary findings from previous cross-sectional evaluations of ReachOut have suggested that the service has a positive impact on well-being [39,40]; however, further research is needed to investigate its effectiveness in improving the mental health and well-being outcomes of young people. This research aims to add to the extant evidence base for digital mental health interventions by critically evaluating the impact of ReachOut on symptoms of depression, anxiety and stress, and suicidal ideation in a large cohort of users.

A longitudinal study was conducted over a 12-week period to explore whether participating in a brief, unstructured digital intervention would be associated with changes in symptoms of depression, anxiety, stress, and suicide risk. Outcome measures of mental health status and suicide risk were assessed at 3 time points across the 12-week study period. We also briefly explored the user experience ratings of the service. This study was approved by the University of Melbourne Human Research Ethics Committee.

ReachOut is an unstructured, self-administered digital mental health service for young people aged 14 to 25 years. ReachOut resources are developed with the involvement of young people and are reviewed by a Clinical Advisory Group, composed of a diverse team of clinicians. ReachOut is accessed by more than 2 million people in Australia annually (based on visitation statistics from January 2020), which indicates that this unstructured digital service modality holds appeal for young people. ReachOut is a multi-layered service with components that vary in intensity. At the base level, it provides psychoeducational information, personal stories, quizzes, videos, and audio recordings. It also offers apps and tools, peer-support from both peers and trained moderators, pathways to clinical support, and customized recommendations to users through NextStep [41]. ReachOut is designed to support well-being affected by everyday stressors to more complex mental health issues, by providing evidence-informed self-help strategies, sharing stories of recovery, building knowledge of professional help, and increasing self-efficacy to seek help. We expected using ReachOut as part of this intervention to increase professional help-seeking behavior and reduce the impact of mental health symptoms.

Participants were recruited through a pop-up notification on the ReachOut website. On clicking the notification, users were asked to complete an eligibility questionnaire. Participants were required to be residing in Australia, aged between 16 and 25 years, and to have used the ReachOut website previously, either to access information or support for themselves or for someone they knew. If eligible, participants were presented with an information statement for providing informed consent. Rolling recruitment occurred across two phases. Phase one recruitment commenced in November 2014 and ended in August 2015. Following this, phase two recruitment commenced in February 2016 and was completed in June 2016.

Participants completed 4 surveys across a 3-month period. A baseline survey (T1) captured demographic characteristics, mental health, and suicide risk. Further surveys were administered 1 week post baseline (T2), 5 weeks post baseline (T3), and 12 weeks post baseline (T4). User experience, including satisfaction with and impressions of the ReachOut website, were assessed at T2, using bespoke measures taken from past ReachOut user studies. Mental health and suicide risk were not assessed at T2. T3 and T4 repeated all measures from the baseline survey, excluding demographics. Each survey took 15-40 min to complete: Baseline (30-40 min), T2 (10-15 min), T3 and T4 (20-30 min).

Frequency and descriptive analyses were conducted to explore the characteristics of the participants. To explore whether participants experienced a reduction in depression, anxiety, and/or stress across the 12-week unstructured intervention study, a series of 3-way mixed analysis of variance (ANOVA) was conducted. These analyses were also used to determine whether the trends across time differed based on gender, sexual orientation, and age group. The assumption of sphericity was violated for all 3 analyses; therefore, the Greenhouse-Geisser correction was applied. The 3-way interactions were examined to explore whether reductions in depression, anxiety, and stress significantly differed between gender, sexual orientation, and age groups. We used Cochran’s Q analysis to explore whether there was a reduction in the proportion of ReachOut users at high risk of suicide across the 12-week study period. Finally, to explore changes in the proportion of people at risk of suicide by gender, sexual orientation, and age group, data files were split by group, and analyses were run separately as above.

A final sample of 1982 young people aged between 16 and 25 years (mean 19.40, SD 2.98) was recruited. The study retention rate was 81.18% (1609/1982), with 1609 participants completing the 3-month follow-up survey. Characteristics of all participants compared with those included in the analysis are shown in Table 1. No major differences were observed between those included and those not included in the analysis. However, those participants who self-selected for this study may not be representative of all people who use ReachOut more broadly. The sample was predominantly female, and many had previously sought help from a mental health professional. Of the participants who had sought help, only 54.53% (656/1203) said they found the help they received from the professional to be helpful. A relatively large number also had a history of mental health-related hospital admissions. See Table 1 for a full description of participant characteristics.

ANOVA revealed a significant main effect of time, showing that there was a significant decrease in overall DASS-21 scores from baseline to follow-up. There was also a significant main effect of DASS-21, highlighting differences in scores between depression, anxiety, and stress. However, these main effects were superseded by significant 2-way interactions between DASS-21 and time. Although scores on depression, anxiety, and stress decreased over time, depression showed the largest decrease from baseline to the 3-month follow-up, when compared with anxiety and stress. See Table 2 for means and standard deviations for interaction effects of DASS and time, gender, sexual orientation, and age group from the ANOVA models described below.

The 3-way interaction between gender, DASS-21, and time was not statistically significant (Table 3), nor was the 3-way interaction between sexual orientation, DASS-21, and time (Table 4). The 3-way interaction between age group, DASS-21, and time was also not statistically significant (Table 5). This demonstrates that the changes in depression, anxiety, and stress scores across time did not significantly differ with respect to gender, sexual orientation, or age group.

Although there were no statistically significant 3-way interactions, there were several statistically significant 2-way interactions. A statistically significant 2-way interaction was observed between the DASS-21 and gender. The differences between depression and anxiety (F_2,1565=6.51, P=.002, η_p²=0.008) and between depression and stress (F_2,1565=7.94, P<.001, η²=0.010) differed based on gender. Participants who identified with the other gender identity consistently scored the highest on depression, anxiety, and stress. However, while female participants scored higher on anxiety and stress compared with male participants, male participants scored higher on depression when compared with female participants (Table 2 for means and standard deviations).

Furthermore, although there was a statistically significant 2-way interaction between DASS-21 and sexual orientation, upon further inspection of the effect, there did not appear to be a meaningful interaction and it did not demonstrate clear patterns between sexual orientation and the DASS-21. That is, heterosexual young people consistently scored lower for depression, anxiety, and stress, while lesbian or gay and bisexual young people consistently scored lower than those with the other sexual orientation. However, there were no other consistent trends between participants who were identified as gay or lesbian, bisexual, or unsure or questioning in depression, anxiety, or stress scores (Table 2 for means and standard deviations). This lack of interaction could have been caused by low participant numbers in the various groups.

Moreover, there was a statistically significant 2-way interaction between age group and time, from baseline to follow-up (F_2,1558=3.06, P=.05, η²=0.004). That is, although all age groups showed a decrease in overall DASS-21 scores from baseline to the 3-month follow-up, this decrease was most pronounced for participants aged between 19 and 21 years (baseline mean 20.17, SD 9.94; follow-up mean 15.94, SD 10.44) when compared with those aged between 16 and 18 years (baseline mean 20.46, SD 9.92; follow up mean 17.47, SD 10.44) and participants aged between 22 and 25 years (baseline mean 20.01, SD 9.93; follow-up mean 16.77, SD 10.44).

There was also a statistically significant 2-way interaction between the DASS-21 and age groups. Contrasts revealed that there were differences in scores between depression and anxiety across age groups (F_2,1558=5.70, P=.003, η_p²=0.007) and between anxiety and stress across age groups (F_2,1558=18.44, P<.001, η_p²=0.023). Although the trends across age groups were similar for depression and stress, they differed for anxiety. Participants aged 16 to 18 years scored the highest for anxiety, followed by those aged 19 to 21 years, while those aged 22 to 25 years scored the lowest on anxiety (Table 2 for means and standard deviations).

A total of 1577 participants completed the SIQ at baseline, 5 weeks postbaseline, and at the 3-month follow-up. There was a statistically significant difference between the proportions of participants at high risk of suicide across the different time points (χ²₂=23.1, P<.001). About 12.30% (194/1577) were at high risk of suicide at baseline, 10.53% (166/1577) at 5 weeks postbaseline, and 8.50% (134/1577) at the 3-month follow-up. Pairwise comparisons were performed using Dunn’s procedures with a Bonferroni correction for multiple comparisons, presenting adjusted P values. Compared with the baseline percentage of participants at high risk of suicide, there was a statistically significant decrease in the percentage of those at high risk at the 3-month follow-up (P<.001). There was no statistically significant difference between the baseline and 5-week postbaseline values (P=.08) but there was a statistically significant reduction from the 5-weeks postbaseline value to the 3-month follow-up (P=.01).

Participants favorably rated their user experience of ReachOut, agreeing that it was relevant and accessible and that it helped them to understand their experiences and gave them practical help strategies and tools. See Table 1 for the percentages of endorsement for each user experience goal.

Overall, the study provided valuable insights into the potential of an unstructured digital intervention; however, this study is not without its limitations. First, participants self-selected into the study, and there were minimal eligibility criteria other than age, location, and current use of ReachOut. Although all participants were ReachOut users, there was no minimum requirement of usage for the duration of the study. Therefore, it is difficult to specifically attribute dosage patterns to improvement in mental health and suicide risk. Furthermore, one goal of the intervention was to facilitate access to professional help, and as such, the use of professional mental health services alongside ReachOut may have resulted in greater reductions observed in mental health scores over time. The possible impact of other help-seeking avenues reduces our confidence in the attribution of causality to the intervention. In addition, this sample had complex mental health needs that may not be representative of young people across the mental health spectrum but rather of those in high distress. Self-report measures were used to capture mental health status and no observer-rated assessments were used. The majority of participants in the study were female, thus limiting generalizability; however, there is also a skew toward young female users within the wider ReachOut user population and the profile of young people who seek help for psychological distress in Australia [13].

Furthermore, while the results indicate significant improvements in mental health and suicide risk among young people, the lack of a control group precludes the definitive attribution of these outcomes to their use of ReachOut. Although RCTs represent the gold standard in evaluating some treatment outcomes, there can be challenges in implementing these studies within the context of a brief, unstructured, and self-directed digital intervention that operates within a dynamic service environment and is rapidly responding to the changing needs of young people. Further, the pace at which digital technology is evolving requires a faster delivery of research and evaluation insights to ensure that these are timely and relevant; the significant time investment of RCT designs is often not conducive to the fast-paced innovation cycle. More rapid sources of evidence are therefore required in addition to RCTs to support interventions to adapt to new technologies and evolving user needs. Further, as others have noted, a favorable outcome in an RCT does not always imply that the intervention will have a meaningful impact in the real world, especially in the digital space [38]. There have been a number of web-based and app-based interventions that have demonstrated favorable results through RCTs [56-58]; however, there has been little consideration of the application of these interventions in the natural environment, in which usage patterns and dosages are unknown, and the environments in which these are implemented are uncontrolled [38,59]. Despite these challenges, discrete components of the ReachOut service offering have been subjected to controlled trials [41]. Future evaluation research that includes a control group would be valuable for drawing more definitive conclusions regarding the attribution of any observed changes to the intervention.

It is also important to consider that spontaneous remission (ie, remission of symptoms that are not related to a given treatment) may account for a proportion of the improvements in mental health and suicide risk observed over the 3-month study period. Without a control group, it is difficult to estimate the influence of spontaneous remission on these findings. There is currently limited evidence regarding the rates of spontaneous remission in young people. However, a meta-analysis that examined remission in untreated depression, defined as ‘clinically significant improvement’, found that 23% of adults remitted at 3 months, with a higher remission rate (not mentioned in the paper) observed in children and adolescents [60]. On the basis of this, we might conservatively estimate that between 20% and 30% of adolescents with an affective disorder spontaneously remit with the passage of time, and it is therefore possible that a proportion of the cases that experienced improvements in mental health outcomes would have experienced it anyway, even in the absence of the support they accessed from ReachOut. However, the level of reduction in symptoms that was observed, coupled with the young people’s positive endorsement of user experience goals as well as a self-report of the subjective impact (ie, how ReachOut helped them) in the qualitative data (see [61]), suggests that ReachOut did contribute to the improvements seen in mental health and suicide risk, although different research designs are required to establish more definitively the extent of this contribution. As noted above, a study design that involves a matched control group would allow for a more thorough investigation of the potential impact of spontaneous remission on the observed effects. This would also allow for an exploration of whether there were any possible study effects associated with completing the evaluation surveys, as opposed to engaging with the intervention per se.

A final limitation of the study is the lack of objective data on the young people’s activity on ReachOut, which precludes the examination of engagement variables as a potential mediator of impact. Future research on ReachOut will build on this study by capturing the users’ browsing data to explore the impact of different components of the intervention on mental health outcomes and the dosage effects. It is hoped that this research will further elucidate the critical factors of unstructured interventions that lead to improvements, including any influences of dose (frequency and duration of use), modality of delivery, and content format.

Despite these possible limitations, this study provided preliminary evidence of promise for a brief, unstructured, self-directed digital intervention to support young people’s mental health and well-being. The large potential for digital interventions for youth mental health and well-being is widely acknowledged [62], and given the substantial evidence of the acceptability, effectiveness, and cost-effectiveness of digital interventions, there is a clear rationale to invest in the development and evaluation of evidence-based digital services for youth mental health.

Digital mental health interventions can address barriers to access and provide services to young people in need at a population level, and there is growing evidence that they are effective in improving mental health outcomes among youth. Specifically, these findings demonstrate the promise of an unstructured digital mental health intervention, ReachOut, in reducing depression, anxiety, and stress symptoms, and risk of suicide in young people. Although ReachOut is primarily intended for prevention and early intervention, the findings from this study highlighted that the service is attracting young people in high levels of distress. Despite the considerable mental health needs of this cohort, significant improvements in all outcomes were observed and young people rated their experience of using ReachOut highly. Our findings, coupled with existing research, indicate that unstructured digital mental health services are not only accessible and acceptable but also have the potential to cater to the diverse mental health needs of young people, in a way that other services are unlikely to be able to do because of limited resources and ability to scale. Future research that involves a comparison group and the collection of user data to explore dose-response effects is needed to confirm and expand on these findings.