Common psychiatric disorders, such as depression and anxiety, are prevalent and costly. Recent nationally representative data suggest that approximately 1 in 3 adults—and 1 in 2 young adults—in the United States struggles with anxiety or depressive symptoms [1]. Although efficacious mental health treatments exist [2], demand for care outpaces its availability, leaving more than half of those with unmet mental health needs unable to access care [3]. Barriers include cost, inadequate insurance coverage, a shortage of clinicians, geographical challenges, stigma, and inadequate health care provider diversity, according to national data [4]. Alongside long-term health care system reforms, there is a need for immediate, scalable solutions to increase access to care.

Digital mental health interventions (DMHIs), delivered via smartphones or the internet, offer one promising solution. DMHIs can disseminate evidence-based interventions at scale and at low cost, and their inherent privacy can mitigate stigma. However, existing DMHIs have not yet fully realized their potential.

DMHIs range from unguided (ie, self-help apps and electronic learning modules) to guided (those that include varying degrees of human support). As unguided DMHIs offer greater scalability at lower cost, they have been touted as ideal solutions to access problems. However, challenges with efficacy, engagement, and trust have undermined their utility in addressing access problems [5,6]. In contrast, guided DMHIs have stronger evidence of efficacy, engagement, and trust.

First, meta-analytic evidence suggests that unguided (vs guided) DMHIs have lower effect sizes and are less effective for those with more severe psychopathology [7,8]. In addition, when given a choice between using a mental health app as stand-alone treatment or to augment in-person therapy, people with moderate and severe (vs mild) psychopathology prefer the latter [9]. Even internet-delivered cognitive behavioral therapy (iCBT) with minimal human support tends to be less acceptable than individual cognitive behavioral therapy (CBT) for people with higher clinical severity [10]. These findings suggest that the lack of human support may lower perceived quality and acceptability for many, especially those with higher clinical severity.

Second, unguided DMHIs suffer from low real-world engagement. Despite demonstrating efficacy in reducing depression and anxiety in randomized controlled trials (RCTs) [11], many stand-alone mental health apps fail to sustain engagement after just 10 days in real-world settings [12]. Similarly, the dropout rates of iCBT are as high as 80%, despite iCBT’s demonstrated efficacy [13]. A recent systematic review found that DMHI engagement facilitators include greater digital literacy, more structured training in DMHI use, perceived DMHI relevance, and DMHI integration into daily life [14]. Because guided DMHIs can directly address such factors, they may be more capable of facilitating engagement [15]. Indeed, studies comparing unguided and guided versions of the same app show that the latter has better engagement and outcomes [16,17].

Third, mistrust of data handling hinders DMHI uptake [18]. Most DMHIs operate outside health care regulation and lack data use protections and transparency [18]. Examples of problematic data sharing practices and opaque privacy policies abound [19] and have, at times, led to high-profile privacy violations [20]. Such incidents undermine trust [21], contributing to low DMHI engagement [18].

Taken together, these findings suggest that sacrificing human support for scalability can lower quality and acceptability. To reach people with mental health problems that range in clinical severity, it is key to retain human support, design trustworthy DMHIs, and use scalable engagement strategies (eg, support that enhances DMHI use, relevance, and digital literacy). However, the challenge remains: how can this all be done without sacrificing scalability?

While guided DMHIs have stronger evidence, they have also been incomplete solutions for addressing barriers to access. Many simply add digital components to the existing face-to-face care [22-26], which neither lowers cost nor impacts the workforce shortage. To address these barriers, some guided DMHIs have replaced clinicians with less costly supporters (eg, coaches and nonprofessionals) [27]. While the effect sizes of those DMHIs are higher than those of unguided DMHIs [28], it remains unclear when and for whom support is best provided by a clinician, coach, or nonspecialist [29]. Until it is known, retaining a trained clinician in care remains an important ethical consideration in DMHIs.

In addition, a key implementation challenge inherent to guided DMHIs remains unresolved: As DMHIs differ from traditional in-person services, they do not readily fit within traditional clinician workflows; as such, the addition of digital tools and data can feel burdensome to clinicians [30]. As a result, when DMHIs transition from RCTs to real-world settings, uptake tends to be low, and these tools are quickly abandoned [31]. Minimizing the added workload for clinicians is essential for the successful implementation of guided DMHIs. However, examples of effective DMHI integration into health care settings remain scarce [30].

A more comprehensive solution is that of the Digital Clinic, an innovative guided DMHI that combines brief clinician-delivered treatment via telehealth with between-session support from an app and a nonspecialist called a digital navigator [32,33], as seen in Figure 1. Each of the model’s components addresses leading access barriers while supporting effectiveness.

First, to ensure both scalability and quality, the model offers brief evidence-based transdiagnostic treatment by a trained clinician (Figure 1A). Brief evidence-based interventions are more efficacious than treatment as usual for depression and anxiety [34-36]. Their brevity addresses the workforce shortage by freeing up clinicians faster; retaining the clinician in care increases the likelihood of reaching people with higher clinical severity.

Second, for greater scalability and impact, the mental health app mindLAMP is integrated into care to facilitate real-world skills practice and enable measurement-based care (Figure 1B). Recent meta-analytic evidence suggests that supplementing standard interventions with an app has additive effects [7], possibly increasing therapeutic dose without taking up additional clinical resources [23]. In addition, measurement-based care is a well-established yet underutilized approach to increasing care quality [37-39]. To facilitate measurement-based care, mindLAMP streamlines questionnaire administration and data visualization; data are discussed in sessions, informing clinical decision-making. The app’s digital phenotyping capacity also enables the collection of behavioral data, offering additional insights for clinical care [33].

Third, to support patient app engagement and clinician data integration into care, a digital navigator is included in this model (Figure 1C). Designed to support innovative technology-enhanced care models, the broad purpose of the digital navigator role is to make technology usable and useful for patients and clinicians [40-42]. In the Digital Clinic, digital navigators support patients during weekly check-ins in several ways: they explain data use and privacy policies to facilitate trust in mindLAMP (which is already designed with the highest privacy and security standards). They also provide technical support and app use training, as needed, share data highlights with patients, and encourage them to use the app in ways their clinician has recommended. Digital navigators also offer data insights to clinicians that can inform patient care so that data enhances care without adding to clinician workloads [43]. As digital navigators do not need clinical expertise, the addition of this feature preserves the model’s scalability and cost-effectiveness.

Finally, remote delivery methods are prioritized to enhance scalability and mitigate stigma, with therapy sessions and digital navigator check-ins conducted via telehealth. Telehealth is acceptable to patients [44], and brief evidence-based treatment via telehealth is noninferior to its in-person equivalent [45]. In addition, some data suggests that digital approaches can address stigma [46].

Prior research has not focused on or evaluated the feasibility and acceptability of a comprehensive care model incorporating all of these components to address leading access barriers. An earlier report on the development of the Digital Clinic showed that this model is promising [32]. In this study, we conducted an open trial to evaluate the Digital Clinic model’s feasibility, acceptability, and potential efficacy in treating patients with common mental health problems ranging in clinical severity.

A total of 10 clinicians were involved in this study (5 master’s and 3 doctoral mental health counseling students from local colleges, a postdoctoral-level clinical psychologist, and a licensed psychiatrist). Clinicians identified as White (n=6), Asian (n=3), and Black (n=1), and had a background in evidence-based treatment. In addition, 16 nonclinician volunteers with an interest in digital interventions were involved as digital navigators in the study [41,43]. Digital navigators were either college students (n=8) or had recently earned bachelor’s degrees (n=8). They had no prior clinical experience, as the digital navigator role does not require clinical expertise but rather active listening skills, which can be taught in a curriculum [42].

A brief therapy manual based on the UP was created for this study. This manual contained session-by-session guides, including guidance for clinicians to integrate the app and its data into care. Clinicians were first given several in-person training sessions in evidence-based treatment. These sessions, led by a licensed clinical psychologist, amounted to approximately 8 hours of training. The focus of the training was on understanding UP principles and on conducting therapy based on the manual. Once clinicians began seeing participants, their adherence and competency was monitored closely in weekly supervisions. Individual supervision (with a psychologist or psychiatrist) focused on conceptualizing patient problems within the UP and conducting treatment in line with evidence-based principles.

Digital navigators were offered 4 training sessions in person that amounted to a total of 10 hours of training. The lead digital navigator of the research team led these training sessions, which included didactic information and experiential learning through role plays. Topics covered included how to conduct the introductory session and the weekly check-ins with participants, how to troubleshoot technical issues that may arise along the way, how to understand digital phenotyping data collected via mindLAMP, and how to support digital literacy by helping participants interpret and understand their own data. Digital navigators were also trained to handle special considerations that could arise in their support meetings with clients: if a participant expressed suicidal thoughts or intent, for example, the digital navigator was trained to escalate the concern to a designated clinical supervisor. Similarly, if a client sought therapeutic advice, digital navigators were trained to encourage the participant to direct those inquiries to their clinician in the next session and to use the app for support between sessions. Upon the completion of this 10-hour training curriculum, digital navigators completed 2 supervised live appointments with participants before being approved to conduct appointments on their own. These training guidelines have been described and are published elsewhere [42,43].

The Digital Clinic is a blended care program that offers brief evidence-based treatment augmented by a mobile app (mindLAMP) [48] and a digital navigator. The purpose of integrating mindLAMP into care is 2-fold: to help patients acquire and generalize new skills and to collect psychosocial insights that inform treatment. The role of the digital navigator is to support app use, helping to resolve any difficulties (technological or motivational) that may interfere with the patient’s ability to benefit from app use. The digital navigator role has been integrated into this model in light of research showing that app engagement often correlates with clinical improvement yet tends to decline when patients are given a stand-alone app without support [18]. The digital navigator was also introduced into the model to avoid overburdening clinicians with app and data management tasks in addition to their therapy-related responsibilities.

The Digital Clinic intervention has 2 phases. Phase 1 involved app use with digital navigator support. Participants begin the 8-week program with 2 weeks of app use supported by brief weekly check-ins with a digital navigator. The goal of this period is to help participants become accustomed to completing daily and weekly self-report measures on the app, and for the app to begin collecting digital phenotyping data. Digital phenotyping data is defined as the “individual-level human phenotype in-situ using data from smartphones and other personal digital devices” [49] and in this study included several behavioral metrics: steps, movement, screentime, and a sleep estimate. Highlights of this and other data from the app were shared with participants during the digital navigator check-ins, where digital navigators also offered support for continued app use. Phase 2 involved therapy sessions and continued app use along with digital navigator support. In this phase, participants met weekly for 6 weeks with a clinician, who provided transdiagnostic treatment based on the UP. Sessions lasted between 45 and 50 minutes, with an hour-long intake. In each session, mindLAMP data were reviewed with the participant, and UP skills were discussed and assigned as home practice through the app. Therapists shared their screen at key points in each session to review mindLAMP data together with the participant, including week-by-week symptom fluctuation graphs and home practice data.

The UP was selected as the basis of care in the Digital Clinic because its transdiagnostic approach aligned with the clinic’s primary aim of increasing access to care. This approach enhances scalability by training clinicians to deliver a single therapy that can be applied to a wide range of presentations. The UP is an emotion-focused CBT that targets reactivity and avoidance, 2 underlying mechanisms that perpetuate distress across various forms of psychopathology. The therapy begins with conceptualization of the patient’s problems within the UP framework. The therapist then offers emotion psychoeducation on the adaptive function of emotions and helps the patient learn to self-monitor their mood and identify the 3 components of emotional experiences (ie, cognitive, physiological, and behavioral). The goal is to help the patient begin to tolerate and understand, rather than habitually react to, unpleasant emotions. Core UP interventions include mindfulness, cognitive flexibility, countering avoidance, and exposure (including interoceptive, emotional, and situational). A termination session consolidates learning and assists patients in creating a plan to independently practice skills and thus continue making gains after termination. The brief treatment manual designed for the Digital Clinic offers guidance on all of these topics, as seen in Table 1.

Analyses were conducted using R (version 4.2.1). In line with, and guided by, guidelines for feasibility studies [67], we computed descriptive statistics to assess feasibility and acceptability, and then conducted paired samples t tests (2-tailed, with Cohen d effect sizes) to examine within-group pre-post differences as a marker of potential efficacy. As commonly reported in the therapy outcomes literature, we also computed clinically significant improvement and remission rates to further assess potential efficacy. Clinically significant improvement was determined by the proportion meeting the minimum clinically important difference (MCID) thresholds established in prior empirical research, which is 4 points for the PHQ-9 and GAD-7 [61] and 6 points for the PHQ-ADS [61]. Per published guidelines, remission was defined as <8 on PHQ-9 [68] and <8 on GAD-7 [69,70], which corresponds to <16 on the PHQ-ADS.

In preparation for conducting the analyses, for participants who did not complete the questionnaire at the end of the intervention (n=36), we obtained PHQ-9 and GAD-7 scores from mindLAMP if they had completed these measures in the app within 10 days of their last therapy session (n=27). Before conducting each type of potential efficacy analysis, we excluded data from participants with baseline subclinical scores on the PHQ-9 (n=74), on the GAD-7 (n=89), or on the PHQ-ADS (n=81). A total of 7 participants experienced significant life events during the 8-week period, including the death of a close loved one (n=5, 71%) and homelessness due to eviction (n=1, 14%) or fire (n=1, 14%). Although we offered these individuals care, nonetheless, we also excluded their data from analyses as these events would have prevented adequate participation in and response to brief treatment. We also excluded data from 1 individual who was wrongly referred to the clinic for a physical rather than a psychological condition.

This study was reviewed before being conducted by the Beth Israel Deaconess Medical Center institutional review board and was approved as a quality improvement project (reference #2022D000016). All participants provided consent as part of the introductory meeting to the Digital Clinic, where a digital navigator informed them on all aspects of the clinic, including the collection of data and its use in the clinic and afterward. As part of the baseline questionnaire, participants signed an informed consent and acknowledgment of services form, which outlined the use of data and limits of confidentiality in treatment and asked participants for their explicit consent to have their deidentified data be used in aggregate with others’ data for research purposes. Data were collected in Health Insurance Portability and Accountability Act (HIPAA)–compliant systems. Participants did not receive compensation for participating in the Digital Clinic.

The total number of participants who completed the Digital Clinic program was 215 (n=136, 63.3% cisgender women, n=73, 34% cisgender men, and n=6, 2.8% nonbinary), with a mean age of 41 (SD 14) years. Approximately 70.2% (151/215) identified as White, 10.7% (23/215) Asian, 9.3% (20/215) Black or African American, 7% (15/215) Hispanic or Latinx, 2.8% (6/215) Middle Eastern or North African, 0.5% (1/215) Native Hawaiian or Pacific Islander, and 0.5% (1/215) biracial.

Of the 401 individuals approached after initial eligibility was confirmed, 289 decided to enroll in the clinic—a 72.1% (289/401) recruitment rate (good, per the 70% benchmark). The proportion of participants who agreed to enroll after understanding all of the components of the Digital Clinic during their first introductory meeting was 87.8% (289/329; excellent, per the 70% benchmark). The recruitment and enrollment flowchart is shown in Figure 2.

After excluding 31 participants still in progress from the 289 enrolled, 83.3% (215/258) completed the 8-week program, an excellent retention rate (given benchmarks of 70%-76%).

This study was an initial evaluation of the Digital Clinic, an innovative model of care designed to mitigate leading mental health care access barriers by offering brief, clinician-delivered evidence-based treatment via telehealth that is augmented by an app and a digital navigator [34,71]. The aim of this study was to assess the feasibility, acceptability, and potential efficacy of this model by conducting a nonrandomized open trial. Results suggest that this model is feasible, acceptable, and has the potential to be an effective solution for increasing access to high-quality, evidence-based mental health intervention.

The open trial showed good to excellent feasibility and acceptability across most metrics. Results indicated that it is feasible to recruit and retain patients. About 87.8% (289/329) of the participants chose to continue after the introductory meeting, which suggests that many patients were not discouraged by the clinic’s all-digital format or its various innovative components (ie, app, digital phenotyping data collection, and integration of a digital navigator into care). Moreover, the Digital Clinic’s completion rate met and exceeded rates seen in other guided DMHIs (eg, blended CBT [52]) and in-person therapy [72]. It also improved on attrition rates of unguided DMHIs, which reach nearly 50% when accounting for publication bias [73].

Therapist, digital navigator, and mindLAMP adherence were also all high. That therapists were not all highly experienced suggests that the training procedures and clinic manual effectively enabled delivery of brief, transdiagnostic, evidence-based treatment by clinicians with a range of experience, supporting the scalability of the model. That digital navigator adherence was also high suggests that checklists helped facilitate smooth check-ins and should be used in future studies. Finally, mindLAMP adherence among participants was high, with app use between sessions potentially extending the impact of treatment to their daily lives. In addition, these 3 components of the clinic (app, clinician, and digital navigator) appeared to be acceptable to patients, based on high satisfaction ratings with each one.

In terms of the potential efficacy of the Digital Clinic, the results yielded promising findings. The average depression, anxiety, and comorbid depression and anxiety symptom scores fell to the subclinical range by the end of the intervention, suggesting the potential of the DMHI in making a meaningful clinical impact in a short time. Moreover, overall rates of clinically significant improvement (67% and 64% for anxiety and depressive symptoms, respectively) and remission (60% and 53% for anxiety and depressive symptoms, respectively) were high, meeting and exceeding those typically observed in traditional treatment (ie, 51% remission for anxiety disorders [74] and 33% for depression [75], according to recent meta-analyses of primarily evidence-based treatments). However, this feasibility study was not designed to establish efficacy and thus did not use a control group. Nevertheless, these outcomes are promising and warrant a future RCT to establish the efficacy.

While the feasibility of quantitative measures was high for baseline and end-of-intervention time points, completion rate of the 3-month follow-up measures fell below our benchmark. This suggests room for improvement, with our current method of simple email outreach 3 months after the intervention being insufficient. Alternative strategies for a future study include offering participants payment or scheduling a study appointment for the 3-month follow-up questionnaire upfront.

Further room for improvement was suggested in terms of the feasibility of aspects of the digital format. Overall, 72.1% (129/179) endorsed that they encountered no significant hurdles, slightly below our 75% benchmark. Most hurdles reported were around finding a quiet place for therapy sessions (19/179, 10.6%) and difficulty using mindLAMP (6/179, 3.4%). To mitigate the first barriers, in future studies, it will help to build in protected time at the introductory meeting to discuss and resolve practical barriers with each patient around finding privacy for therapy sessions. Regarding the second hurdle, it is likely that difficulty reported around mindLAMP use was resolved in digital navigator meetings. Nevertheless, a future study analyzing the qualitative feedback obtained in this study will help shed light on this question and inform refinements.

Finally, a key lesson about DMHI implementation came out of this study: Partnering with primary care substantially helped with the successful implementation of the Digital Clinic. For one, it eased the burden of delivering mental health care on PCPs—a task that increasingly falls on them due to limited access to specialty mental health care [76]. In addition, PCPs reported that strongly endorsing the Digital Clinic directly to their patients rather than referring them to an external resource seemed to increase patients’ willingness to start care at the Digital Clinic. These PCPs noted that having the Digital Clinic embedded into the health care system may have felt like a natural extension of patients’ primary care received at the hospital, rather than being viewed as a separate mental health program. Thus, partnering with primary care can solve provider problems and increase patient engagement and access to mental health care.

Strengths of this study include its evaluation of a program designed to be both highly scalable and effective, offering a promising solution to increase access to care. Several components increase scalability, including brief therapy, the creation of standardized manuals for the clinician and digital navigator roles, and the use of an open-source smartphone app. The focus on transdiagnostic evidence-based treatment enhances the program’s ability to reach individuals with a wide range of clinical presentations. That the program was effective for a majority of this patient sample without the need for highly experienced clinicians supports the potential for its widespread adoption and implementation in various health care settings.

Despite these strengths, several limitations are also worth considering. First, although valid and reliable instruments were used for all outcomes, the PHQ-9 and GAD-7 measures do not adequately capture distress for some clinical presentations (eg, someone with social anxiety disorder, whose distress is only apparent in specific situations they may generally avoid). Future studies should consider a fuller breadth of outcome measures. Second, the low ethonoracial diversity of our sample may limit the applicability of the findings to more diverse populations. In addition, this study focused entirely on English-speaking participants, and the experiences and outcomes discussed in this study may not be fully reflective of those of non-English speakers. These factors should therefore be considered when interpreting the generalizability of this study’s findings. Third, because this was a feasibility study [77] designed to primarily examine whether this model is feasible and acceptable to patients and providers (and secondarily whether it is potentially efficacious), no control group was used. The lack of a control group precludes definitive conclusions about clinical improvement resulting from the intervention itself rather than the passage of time or other factors. Fourth, we did not track treatment fidelity in a traditional manner, relying instead on supervision meetings and the subsequent examination of clinical notes to assess clinician adherence. This method could be strengthened in a future study by using a checklist system as done with digital navigators, or by obtaining session recordings coded for adherence. Fourth, although data integration into care was a frequent topic of discussion in supervision meetings, we did not formally track the extent to which clinicians regularly incorporated data into care. This limitation paves the way toward fruitful avenues for further enhancing clinician training in the future and implementing a process of actively measuring the extent of data incorporation into care. Finally, we did not formally track whether patients were concurrently receiving other therapies or medications. We inquired about these factors at intake, and in the few cases in which participants were in concurrent outpatient therapy, we recommended that they start in the Digital Clinic after their current treatment. However, we still accommodated those on medications. As a result, for some patients, pre-post clinical improvements may not be solely due to the Digital Clinic intervention. This aspect will need to be more rigorously tracked and addressed in a future RCT.

Results of this study suggest that this model of care shows promise as a solution for mitigating barriers to quality mental health intervention access. This model was informed by research that shows that digital interventions offered with (vs without) human support are more effective [7], especially for people with higher clinical severity [31]. The inclusion of support in terms of clinicians and digital navigators appears to be effective in increasing access and quality in care for people with a range of clinical severity. Results suggest that the Digital Clinic warrants a future efficacy RCT.

Another future direction is to conduct a dismantling study to isolate the additive effects on outcomes of the brief clinician-delivered therapy, the digital navigator, and mindLAMP app integration. Research suggests that the digital components of guided DMHIs may independently contribute to outcomes [78] and that mindLAMP enhanced by a digital navigator alone can have its own beneficial effects [79]. The extent of each component’s impact on efficacy can inform future model adaptation and lead to a more cost-effective, stepped-care version of the Digital Clinic, where more clinical resources are offered first to those with higher severity. Recent research supports such models, as they use limited clinical resources in ways that maximize efficacy and are cost-effective [80].

Related, continuing to focus on the implementation and therapeutic effect of incorporating digital phenotyping data versus actively collected data into care is an important future direction. We observed that some clinician-patient dyads embraced both types of data more readily than others, with some patients also benefiting more from digital phenotyping aspects of the data than others. We also observed that some clinicians required further training to feel more confident in integrating all kinds of data into care. This is not surprising, as most training programs do not prioritize the development of skills in conducting measurement-based care. Addressing these implementation and knowledge gaps may enhance both clinician readiness and patient outcomes.

Finally, a still unanswered question in the field is whether gains obtained from participating in brief, guided DMHIs are sustained in the long term. In this study, clinical improvement was sustained for the 41.7% (68/163) of participants with baseline clinical scores who completed the 3-month follow-up questionnaire. This finding once again suggests that this intervention may be potentially efficacious in the short and long term, but it warrants more research to confirm this possibility.

Guided DMHIs that offer brief, evidence-based treatment may reduce the cost of care and alleviate the provider shortage; integrating an app into treatment that provides between-session support and enables measurement-based care can increase the impact of care delivered in a short time. The digital navigator supports app engagement and can lower the burden of data collection and technology use on the clinician. Results from this study suggest that the Digital Clinic model is feasible, acceptable, and potentially efficacious, warranting a future RCT.