Large-scale clinical trials have historically been conducted in person. Yet, in-person centralized data collection is limited by geographical restrictions on participant recruitment, high operational costs associated with large study teams and specialized facilities, and the complex logistics of multisite coordination. Although the COVID-19 pandemic significantly disrupted in-person clinical trials, it simultaneously accelerated the adoption of novel digital research methods compatible with COVID-19–related safety concerns and restrictions surrounding in-person clinical research [1,2]. One such example is decentralized clinical trials (DCTs), which apply digital technologies to conduct and monitor remote study activities and are less location-restricted; thus, DCTs offer a novel approach to overcoming challenges associated with in-person clinical trials.

In addition to overcoming the challenges of in-person trials, DCTs are particularly well-suited to testing digital mental health interventions (DMHIs), which are technology-based interventions designed to improve mental health that can be delivered via tools such as smartphones. Like DCTs, DMHIs are not bound by geographic restrictions, making them well-suited for scalable and remote evaluation. DMHIs hold promise for addressing numerous psychiatric conditions, including depression, substance misuse, anxiety, and trauma-related disorders [3-5]. Given that the need for mental health treatment has outpaced human resources, DMHIs offer practical, real-world benefits, including the ability to deliver evidence-based interventions at scale while maintaining anonymity, engagement, and flexibility [5,6]. Yet, even as DMHIs make it possible to reach larger and more geographically dispersed samples, researchers face novel and substantial methodological challenges associated with remotely evaluating these interventions with the same rigor as large-scale, in-person clinical trials. Meeting this challenge requires methods that not only assess clinical outcomes but also incorporate objective behavioral and biological markers to clarify the mechanistic effects of DMHIs on mental health in ecologically valid contexts.

In this regard, DCTs offer both methodological advantages and unique challenges for mechanistic mental health research. In terms of advantages, they may be particularly well-suited for multimodal sampling of potentially relevant biomarkers across interacting psychological and physiological systems in ecologically valid settings [7-9]. For example, the remote collection of cardiovascular biomarkers, measured via wearable sensors (eg, biometric vests and wrist-worn devices), and the ambulatory assessment of psychological states and behavior have been increasingly integrated into DCTs [7,10]. Yet, very few trials have included remotely collected biological samples (eg, blood and microbiome), which are commonly collected in the laboratory but can, in theory, be self-sampled in DCTs. Challenges to blood and microbiome sample collection—2 biomarkers with particular mental health relevance [11,12]—include the burden on participants to collect sufficient samples and the burden on study teams to ensure samples are temperature-controlled, which is required for some sample collection methods. Similarly, although some digital health trials have included objective behavioral markers of mental health [13], these measures are not widely implemented, in part because of the complex methodological challenges involved in collecting and processing video and audio data.

This gap in the multimodal collection of biological and behavioral data is important. Biological measures are necessary to assess key inflammatory and microbial markers implicated in the neuroimmune and brain-gut-microbiota axes [11,12]. These axes may be especially relevant for mental health interventions as inflammatory signals and gut microbiota directly and indirectly modulate the central nervous system, with consequences for stress, resilience, and mental health [11,12,14,15]. Similarly, past research demonstrates that behavioral measures, such as natural language use [16], reward sensitivity [17], and performance on cognitive tasks [18,19], are meaningful markers of well-being.

Multimodal assessment and the inclusion of objective measures of well-being may be particularly relevant for DCTs testing DMHIs for depression. Depression, a leading cause of disability worldwide [20], is influenced by a complex combination of psychological, behavioral, and biological factors. For example, past research demonstrates depression is associated with disruptions in natural language (eg, greater first-person pronoun use), disruptions to reward sensitivity [17], cognitive disruptions [19], and elevated levels of inflammation [21]. Given the public health burden associated with depression [20], scalable and accessible interventions that are rigorously tested within DCTs, and which allow for the collection of psychological, behavioral, and biological measures to understand how DMHIs are, or are not, improving depressive symptoms and objective measures, are urgently needed [22]. Yet, methodological challenges remain for researchers aiming to remotely incorporate a multimodal battery of measures that can rigorously examine the effects of DMHIs across these domains in large-scale DCTs. Furthermore, researchers also face challenges in using methods that are feasible and not overly complicated or burdensome for participants to complete. Although prior studies have examined remote digital measures or biomarkers in mental health research generally, approaches to incorporating multimodal batteries that include such measures within large-scale DCTs of DMHIs are not well established or widely implemented.

Accordingly, through a National Institutes of Health–funded multidisciplinary network, we developed a methodology for a DCT to remotely deliver a smartphone-based DMHI—the Healthy Minds Program (HMP) app (Humin)—to a large, national sample of adults selected for clinically elevated depressive symptoms. As part of the Behavior, Biology, and Well-being (BeWell) Study, we developed a methodology to integrate psychological (eg, structured clinical interviews and self-report questionnaires), behavioral (eg, video ecological momentary assessment [EMA] data and Mnemonic Similarity Task [MST]), and biological (eg, blood and microbiome) measures into this 3-armed randomized DCT. This paper is not intended to report on the efficacy of the HMP app or the trial’s primary clinical outcomes. Rather, this paper presents the methodology used in the BeWell Study to remotely collect and process a multimodal battery of measures for researchers seeking to implement multimodal assessments within DCTs—particularly for testing DMHIs. Secondarily, we aim to assess the feasibility of the developed methodology by examining data completion and quality, as well as predictors (ie, recruitment source, demographics, and depressive symptom severity) of data completion and quality across the study. We hope that these methodological details and the evaluation of the feasibility of this approach will be useful to other researchers seeking to implement multimodal assessment within large-scale DCTs, particularly those testing DMHIs.

This study reports the methodology developed to incorporate a multimodal battery of psychological, behavioral, and biological measures within a large-scale (n=1156), 3-arm, randomized, longitudinal DCT testing a DMHI—the HMP app—in a national sample of adults selected for clinically elevated depressive symptoms. Secondarily, to inform future research in this area, we assessed the feasibility of this approach by examining data completion and quality across modalities, as well as recruitment sources, demographics, and depressive symptoms as predictors of these outcomes. While advancements have enabled mental health interventions to be delivered in an individual’s real-world context via DMHIs, and therefore incorporated into DCTs, methodological challenges remain for researchers aiming to rigorously test them in the field with objective behavioral and biological measures. The methodology developed for the BeWell Study provides one approach to addressing these methodological challenges. Results were largely encouraging; participants completed a multimodal battery of measures entirely remotely (≥68% completion rate across all modalities and time points), remotely collected data were of high quality (≥84% analyzable data across all modalities and time points), and high participant retention (971/1156, 84%) was maintained. To our knowledge, the BeWell Study is the largest DCT to use a fully remote, multimodal battery of both self-report and objective psychological, biological, and behavioral measures in a sample with clinically elevated depressive symptoms testing a DMHI. In addition to describing and contextualizing the methodology’s strengths, we consider its limitations and feasibility.

We successfully recruited a large sample of participants with elevated depressive symptoms, primarily using Craigslist, a US-based advertising website, and a university campus listserv. Both methods were low-cost and together accounted for 69.41% (801/1154) of enrolled participants. Craigslist, in particular, enabled us to recruit from specific regions of the United States, which facilitated intentional outreach to areas with large racial and ethnic minority populations, and the enrollment of participants from all 50 US states. Although we were ultimately unable to match the US Census with racial and ethnic proportions, 31.12% (346/1112) of participants did not identify as non-Hispanic White, which is an identical proportion reported in a recent systematic review of randomized trials of app interventions for depression and/or anxiety [53] and higher than the proportion reported in a systematic review of randomized trials of mindfulness-based interventions [54]. Notably, we did not find evidence that recruitment sources were associated with measure completion or quality. This finding can allay concerns that participants from recruitment sources that are not primarily designed for clinical trial recruitment (eg, Craigslist) may be less likely to remain engaged. Together, results suggest that recruitment from Craigslist and university campus listservs can yield large samples with minimal cost without sacrificing measure completion and quality.

An important question, but perhaps especially in DCTs, is the degree to which participant characteristics influence measure completion and data quality. We found older participants were more likely to complete measures across modalities (ie, questionnaire, behavioral, and biological), although age was not associated with data quality. College graduates were more likely to complete all measures. Individuals with a gender coded as “other” were less likely than the reference group of women-identified participants to complete the questionnaire data. However, this result warrants caution, as only 1.06% of the study sample identified as the gender category “other.” Differences did not emerge by gender for the other modalities or for data quality. Non-Hispanic White individuals were more likely to pass the attention check than other racial and ethnic categories but no more likely to complete measures. Depression severity at screener predicted lower odds of completing all baseline measures—an enrollment requirement. However, among enrolled participants, baseline depression severity did not predict measure completion or data quality.

The findings highlight that some disparity-linked identities (eg, noncollege graduates and other gender) were associated with attrition, whereas others (eg, race and ethnicity) were not. The effect sizes associated with data completion for other genders (OR 0.38, 95% CI 0.17-0.92, for questionnaires) and educational attainment (ORs 1.56, 95% CI 1.08-2.23, to OR 1.67, 95% CI 1.13-2.48, across measurement domains) are both small, based on conventional guidelines [55]. Nonetheless, they indicate room for improvement regarding engaging the full range of participants throughout the course of a DCT. Findings related to prediction of data quality partially replicate prior work, which has found that some racial and ethnic minority groups are more likely to have data flagged for quality reasons, although other associations (lower-quality data from younger participants) were not replicated [56]. Therefore, it will be important for future DCTs to oversample underrepresented groups to ensure that the sample is not further imbalanced if data are omitted due to quality concerns. Though greater depressive symptoms were associated with a lower likelihood of completing baseline measures and eligibility for enrollment, the lack of association between depression severity and both completion and data quality among participants enrolled in the study is encouraging. This suggests, among enrolled participants, study procedures were equally feasible across the range of symptom severity of enrolled participants, which is notable, given that anhedonia and low motivation are cardinal symptoms of depression. Nonetheless, as discussed below, results must be interpreted knowing that only participants who completed baseline measures were randomized. This created a selection bias among participants enrolled, with implications for the external validity of results.

The BeWell protocol had several strengths, most importantly, demonstrating the feasibility of gathering multimodal data from a national sample at scale, within a fully remote, DCT. Objective behavioral and biological measures, in particular, present substantial challenges for collection and processing within a fully remote design. Yet, such measures are crucial in improving the level of rigor at which DMHIs can be assessed. In particular, we note the novelty of including video EMA data and biological samples remotely, which provide important, objective insights into participants’ well-being sampled within their environment. The original development of the Teddy app took approximately 1 month to develop by a senior software engineer. That the Teddy app is open source and available for researchers to use and replicate within their own studies is a substantial contribution and additional strength [33]. Sharing the platform in this way reduces redundant redevelopment and minimizes waste of financial resources. We note that the use of reloadable debit cards, used to pay participants more quickly than with checks (ie, funds immediately available upon processing), our procedure for sending and receiving biological samples through FedEx, and clear participant instructions for biological sample collection were some features that may have contributed to measure completion. Nonetheless, there are several important limitations. First, completion of some measures, especially the video EMA task, proved somewhat challenging: participants were directed to it only after completing REDCap questionnaires. Some participants completed questionnaires on their computer, which required them to switch to their phone to complete video EMA tasks. Future studies should streamline this process (eg, integrating the video EMA into REDCap survey flow).

We were not able to recruit racial and ethnic minority participants in proportion to the US Census. Future studies may need to combine the low-cost recruitment methods (eg, Craigslist) with higher-cost alternatives that offer targeted recruitment and oversampling based on demographics. This could include the use of online advertising that is based on user demographics or the use of commercial recruitment services that guarantee recruitment based on demographic and other variables (eg, symptom severity) or partnering with community-based organizations. Third, we saw evidence that some baseline demographic characteristics were associated with both measure completion and data quality. This highlights the importance of future efforts aimed at ensuring high rates of completion across demographic groups. Future work may also examine methods to increase data quality. The length of the assessment battery in the BeWell Study (30-90 minutes) may have contributed to lower questionnaire data quality for some groups. Thus, future studies may benefit from a more streamlined set of measures to combat inattentive responding.

Additional important limitations are questions regarding the degree to which we can reasonably generalize from our sample of participants to the broader population of US adults with elevated depression symptoms. Generalizability could be limited due to our recruitment methods (eg, Craigslist and UW–Madison campus emails). These recruitment sources may not reflect the broader depressed population, and generalizability may also be constrained by regulatory, ethical, or logistical differences across countries that could limit replication of this protocol elsewhere. In addition, only a very small fraction of individuals who started the screener were ultimately randomized (ie, 1156/52,052, 2.22%). Furthermore, because depression severity significantly predicted a lower likelihood of completing baseline measures—a necessary step for enrollment—the screening-to-enrollment ratio in this study poses a threat to external validity. Even so, selecting participants who were able to complete baseline measures, which were repeated throughout the study, strengthened its internal validity, presumably by minimizing missing data post randomization. Potential bias and decreased generalizability related to the screening-to-enrollment ratio is a long-standing challenge in clinical trials [57] and certainly a crucial limitation of the current methodology and broader trial. Although our study had relatively few exclusion criteria (eg, did not exclude based on most forms of psychiatric comorbidity), participants were very likely excluded for a variety of unmeasured factors (eg, conscientiousness) linked to their ability to complete the steps required to be enrolled (eg, returning biological samples). At a minimum, retention rates were certainly increased, and fraud related to repeated enrollment minimized, by requiring all participants to complete a synchronous diagnostic interview session and to return biological samples prior to randomization. More clearly determining why participants are or are not willing to continue with enrollment in multimodal DCTs will be valuable to investigate in future studies. In addition, using methods explicitly designed to determine the representativeness of the enrolled sample a priori would also be valuable.

These limitations notwithstanding, this study demonstrates that a multimodal assessment approach is feasible within DCTs. Specifically, the study showed that digital intervention trials can include the kinds of behavioral and biological measures that have been historically limited to the laboratory setting. Although behavioral tasks have been used in fully remote digital health trials previously [58], our study demonstrated the feasibility of gathering a broad set of behavioral and biological measures within a sample with elevated symptoms of depression. Multimodal approaches such as this are essential for understanding the effects of digital interventions on key psychological, behavioral, and biological aspects of depression that cannot be measured using self-report or other measures typically used in DCTs. Multimodal assessment approaches have the potential to yield more effective and more personalized digital interventions to reduce the burden of disease associated with depression and other mental illnesses.