Trials of DMHIs have used a variety of controls, including minimal comparators such as waitlists and brief psychoeducation, as well as more active alternatives, such as popular DMHIs or human-supported psychotherapy [2]. However, for DMHIs intended for online delivery, these controls do not constitute TAU because they fail to reflect the resources that are typically accessed by online help-seekers [3]. Instead, online help-seekers are more likely to engage with mental health–related popular online content. Such content—defined as online media intended for a public audience—is ubiquitous, often tailored to specific groups, and, in some cases, may offer valuable guidance and emotional support. As such, it may serve as the foundation for more ecologically valid TAU (ie, online help-seeking as usual) controls in DMHI trials.

Some popular online content is created with the explicit aim of supporting consumers’ mental health (eg, TikToks offering self-care advice, attention-deficit/hyperactivity disorder infographics, and blog posts about a writer’s experience with anxiety). Other content is not explicitly focused on mental health but may nevertheless support relevant goals. For example, a philosophy podcast might promote a positive mindset shift, while a favorite music video could help to regulate difficult emotions. However, few content creators are trained mental health professionals, and all operate within an internet ecosystem that rewards engagement rather than user well-being. Consequently, much mental health–related popular online content is not evidence-based, and some may be unhelpful or even harmful [4]. At the same time, its widespread popularity and use suggests that many individuals find it helpful.

Popular online content has been evaluated in several RCTs. One trial sent participants weekly emails with psychoeducational online content related to positive psychology, including videos and blog posts [5]. Another study repurposed popular videos, articles, and infographics as stress management “microinterventions” to help participants develop self-care skills and reduce stress and depression [6]. Together, these studies provide preliminary evidence that popular online content can be feasible and acceptable in RCT contexts. However, further research is needed to determine how popular online content–based controls compare with conventional controls, and which types of popular content best fulfill the functions of a control (ie, accounting for alternative explanations, such as the natural course of the treatment target, expectations of benefit, and engagement) [1-3].

In secondary analyses of 2 RCTs [7], this study examined the feasibility of popular online content as a TAU control condition by evaluating the acceptability and efficacy of popular online content–based programs relative to evidence-based DMHIs and a conventional attention-matched control condition.

In study 1, participants were recruited globally through advertisements on social media and within the United States via the online participant recruitment platform CloudResearch Connect [8]. To be eligible, participants were required to be fluent in English, be aged 16 years or older, and score at least 3 (range 0‐6) on either the depression or anxiety subscale of the 4-item version of the Patient Health Questionnaire, as these scores are suggestive of an elevated depressive or anxiety disorder [9]. In study 2, US participants experiencing elevated loneliness were recruited via CloudResearch Connect. Eligibility criteria for study 2 included fluency in English, being aged 18 years or older, and meeting predefined criteria for “struggling with loneliness,” defined as distress related to loneliness (binary item) and scoring 6 or greater on the 3-item version of the UCLA Loneliness Scale, reflecting elevated loneliness [10]. In both study 1 and study 2, computer and internet literacy was an implicit eligibility criterion.

CloudResearch Connect was not originally planned as a recruitment platform; however, it was used after other recruitment methods were flooded by fraudulent participants (ie, automated bots or individuals falsely claiming to meet eligibility criteria). At the time of this study, CloudResearch Connect appeared to have a low rate of fraudulent research participation.

Study procedures were approved by the University of California, Irvine Human Subjects Review Board (protocol 1253) and conducted in accordance with the World Medical Association Declaration of Helsinki. Participants provided informed consent via an online information sheet after eligibility screening. Participation was anonymous, and all procedures were conducted online without direct contact between participants and the research team. The data collection platform (Qualtrics) securely stored participant information. In both studies, participants received US $10.00 for completing the screener, intervention, 4-week follow-up, and 8-week follow-up.

Outcomes were assessed at baseline, postintervention, 4 weeks, and 8 weeks, consistent with typical assessment timelines in DMHI trials. All measures were self-reported. In study 1, the primary outcome was psychological distress, assessed using the 9-item Depression Anxiety Stress Scale [16]. The scale asks participants to rate statements about their past week (eg, “I found it hard to wind down”) on a scale from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time), with total scores ranging from 0 to 27. In study 2, the primary outcome was loneliness, measured by version 3 of the UCLA Loneliness Scale [17]. The scale assesses the frequency of feelings such as “left out” or “close to people” on a scale from 1 (never) to 4 (often), with total scores ranging from 20 to 80. In both studies, the Credibility/Expectancy Questionnaire (CEQ) total score (range 4‐67) was used to assess participants’ perceptions of program credibility and expected improvements in the primary outcome [18]. Some CEQ items are rated 1 (not at all) to 9 (very much), and others are rated from 0 (0%) to 10 (100%).

All 3 measures demonstrated reliability and validity in adult samples [16-18]. Measures were presented in a random order at each time point for each participant, and all measures have been used extensively in online questionnaires. Dropout was defined as discontinuing participation in the study while completing the program to which a participant was assigned. Additional outcomes are described in the study preregistrations.

Analysis of variance was used to compare outcomes across conditions at a single time point, followed by Tukey tests for pairwise comparisons. Linear mixed-effects regression models were used to assess differences across conditions in change over time. Chi-square tests and odds ratios (ORs) were used to compare program dropout rates. The primary preregistered analyses in each study evaluated efficacy, whereas secondary preregistered analyses examined feasibility and acceptability. Analyses included all available data under a missing-at-random assumption using an intent-to-treat approach. Statistical significance was defined as a P value of .05 or less.

In study 1, based on effect sizes reported in studies of similar single-session interventions, the aim was to detect a Cohen d of 0.20 between conditions. Power analyses indicated that 394 participants per condition would be required to detect a Cohen d of 0.20 with 80% power; however, this estimate did not account for attrition. The same target effect size was specified for study 2, but due to an error in the sample size calculation, the study was subsequently underpowered. No interim analyses were conducted, and no stopping guidelines were established.

In study 1, data were collected from April to August 2023. In study 2, data were collected from July to September 2023. Both studies concluded upon reaching (or just after reaching) the preregistered target sample size. A total of 916 participants were randomized in study 1 and 431 in study 2. CONSORT flow diagrams detailing participant randomization and completion at each time point are presented in Figure 2 [19]. In study 1, the median participant age was 32 years; 52.6% (482/916) identified as female, and 63.0% (577/916) identified as White. In study 2, the median age was 36 years; 56.1% (242/431) identified as female, and 68.7% (296/431) identified as White. Baseline demographic characteristics can be accessed through the link provided in Kaveladze [20] for study 1 and in Kaveladze [21] for study 2.

In study 1, the curated popular online content library’s mean credibility and expected benefit score did not significantly differ from the single-session intervention (Cohen d=0.08, 95% CI −0.06 to 0.21; P=.88) but was higher than that of unguided online help-seeking (Cohen d=0.23, 95% CI 0.10-0.37; P=.01). Unguided online help-seeking had a lower mean credibility and expected benefit score than the single-session intervention (Cohen d=0.16, 95% CI 0.02-0.29; P=.04). Participants were less likely to drop out during the curated popular content library than during unguided online help-seeking (30/312, 9.6% vs 81/313, 25.9%; OR 3.28, 95% CI 2.09-5.17; P<.001), although the dropout rate for the curated popular online content library was higher than the single-session intervention (14/291, 4.8% OR 2.10, 95% CI 1.09-4.05; P=.03).

In study 2, the mean credibility and expected benefit score did not differ between the curated popular online content library and the control (Cohen d=0.00, 95% CI −0.14 to 0.15; P>.99). The single-session intervention had a higher mean credibility and expected benefit score than both the control (Cohen d=0.34, 95% CI 0.20–0.49; P=.02) and the curated popular online content library (Cohen d=0.32, 95% CI 0.17-0.48; P=.02). The dropout rate for the curated popular online content library (7/145, 4.8%) was higher than that of the other 2 conditions (0/136, 0.0% and 0/150, 0.0%; OR 16.30, 95% CI 0.92-288.05; P=.001). Because 2 conditions had zero dropout, a continuity correction of 0.5 was applied to calculate the OR. The credibility and expected benefit score across conditions for study 1 can be accessed through the links provided in Kaveladze [22] and in Kaveladze [23] for study 2.

Psychological distress in study 1 (b=–2.30; Cohen d=−0.40, 95% CI −0.33 to −0.46; P<.001) and loneliness in study 2 (b=–6.06; Cohen d=−0.53, 95% CI −0.46 to −0.61; P<.001) decreased significantly from baseline to 8-week follow-up. However, changes in primary outcomes over this period did not differ across conditions. In study 1, all between-condition effect sizes were less than 0.15 and all corresponding P values were greater than .06. In study 2, all effect sizes were less than 0.12 and all corresponding P values were greater than .25. Model outputs and figures for study 1 can be accessed through the links provided in Kaveladze [24] and Kaveladze [25] for study 2. No participants reported harm or unintended effects, although the questionnaires did not include specific items assessing adverse effects.

Effectiveness trials of DMHIs delivered online require control conditions that reflect TAU within the online help-seeking context [3]. In secondary analyses of 2 RCTs, the utility of mental health–related popular online content as a TAU control was examined. The findings in this study suggest that curated libraries of popular online content (conceptualized as enhanced TAU) can provide a feasible and ecologically valid control condition in trials of online DMHIs. In study 1, the curated popular online content library outperformed unguided online help-seeking in terms of both dropout as well as the credibility and expected benefit score. In study 2, the credibility and expected benefit score for the curated library did not differ from that of a typical control. However, dropout rates were higher for the curated library than for the single-session intervention in study 1 (30/312, 9.6% vs 14/291, 4.8%) and higher for both the single-session intervention and attention-matched control in study 2 (7/145, 4.8% vs 0/136, 0.0% and 0/150, 0.0%). These findings suggest that the popular online content–based programs may require greater effort than single-session interventions of similar duration and may benefit from further streamlining to facilitate their comparison.

Strengths of this study include the preregistered RCT designs and the variety of programs tested. Limitations include the use of paid online worker samples, which may limit generalizability to other populations, reliance on self-reported outcomes, and differences between the samples in the 2 studies. In addition, differential attrition across conditions (particularly in study 1) may have biased the results (ie, if participants who disliked a program were more likely to drop out, their presumably lower CEQ scores would not have been included in that intervention’s mean). Another limitation is that the curated content libraries were culture- and time-specific and were developed through crowdsourcing rather than a more systematic review of popular online content and rigorous evaluation of its effectiveness. Finally, because the studies were underpowered for detecting small effects in brief interventions, small but important differences in efficacy or acceptability across conditions may have been missed.

One important future direction is the development of improved popular online content controls. Although the curated popular online content conditions in this study functioned as TAU, they were imperfect proxies for typical online help-seeking. Future controls should aim to more closely reflect real-world online help-seeking behaviors, ideally through codesign with individuals who have lived experience of online help-seeking behaviors. One approach may be to integrate the online communities and content-selection algorithms through which help-seekers typically find popular online content. In addition, given the higher attrition observed in the popular online content–based programs, more streamlined interventions (ie, those including only a limited number of content pieces) may also be beneficial. Additional research is needed to minimize the risk of harmful misinformation within popular online content–based programs [4]. Finally, as conversational artificial intelligence chatbots become an increasingly common source of emotional support worldwide, their role as a form of typical online help-seeking should be recognized and accounted for in effectiveness trials.

To be considered effective in real-world settings, DMHIs delivered online should demonstrate benefit beyond the supportive resources individuals typically access when seeking support online. Popular online content represents a practical and ecologically valid TAU control that may strengthen causal inference in trials of online DMHIs. Future TAU conditions in online help-seeking contexts should aim to align with the effort and attrition profiles of the DMHIs to which they are compared and account for the increasingly prominent role of conversational artificial intelligence in online mental health seeking.