This paper is in the following e-collection/theme issue:

Depression and Mood Disorders; Suicide Prevention (1364) Innovations in Mental Health Systems (391) Web-based and Mobile Health Interventions (3304) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1519) Anxiety and Stress Disorders (1055)

Published on in Vol 10 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/46200, first published .
A Novel and Integrated Digitally Supported System of Care for Depression and Anxiety: Findings From an Open Trial

A Novel and Integrated Digitally Supported System of Care for Depression and Anxiety: Findings From an Open Trial

A Novel and Integrated Digitally Supported System of Care for Depression and Anxiety: Findings From an Open Trial

Authors of this article:

Kate Wolitzky-Taylor1 Author Orcid Image ;   Richard LeBeau2 Author Orcid Image ;   Inna Arnaudova1 Author Orcid Image ;   Nora Barnes-Horowitz2 Author Orcid Image ;   Elizabeth Gong-Guy1 Author Orcid Image ;   Scott Fears1, 3 Author Orcid Image ;   Eliza Congdon1 Author Orcid Image ;   Nelson Freimer1 Author Orcid Image ;   Michelle Craske1, 2 Author Orcid Image
Article Authors Cited by (12) Tweetations (12) Metrics

    Original Paper

    1Department of Psychiatry and Biobehavioral Sciences, University of California - Los Angeles, Los Angeles, CA, United States

    2Department of Psychology, University of California - Los Angeles, Los Angeles, CA, United States

    3Greater Los Angeles Veterans Administration, Los Angeles, CA, United States

    Corresponding Author:

    Kate Wolitzky-Taylor, PhD

    Department of Psychiatry and Biobehavioral Sciences

    University of California - Los Angeles

    760 Westwood Blvd.

    Los Angeles, CA, 90025

    United States

    Phone: 1 3102675339

    Email: kbtaylor@mednet.ucla.edu


    Background: The global burden of anxiety and depression has created an urgent need for scalable approaches to increase access to evidence-based mental health care. The Screening and Treatment for Anxiety and Depression (STAND) system of care was developed to meet this need through the use of internet-connected devices for assessment and provision of treatment. STAND triages to level of care (monitoring only, digital therapy with coaches, digital therapy assisted by clinicians in training, and clinical care) and then continuously monitors symptoms to adapt level of care. Triaging and adaptation are based on symptom severity and suicide risk scores obtained from computerized adaptive testing administered remotely.

    Objective: This article discusses how the STAND system of care improves upon current clinical paradigms, and presents preliminary data on feasibility, acceptability, and effectiveness of STAND in a sample of US-based university students.

    Methods: US-based university students were recruited and enrolled in an open trial of the STAND system of care. Participants were triaged based on initial symptom severity derived from a computerized adaptive test and monitored over 40 weeks on anxiety, depression, and suicide risk to inform treatment adaptation and evaluate preliminary effectiveness.

    Results: Nearly 5000 students were screened and 516 received care. Depression and anxiety severity scores improved across all tiers (P<.001 in all cases). Suicide risk severity improved in the highest tier (ie, clinical care; P<.001). Acceptability and feasibility were demonstrated.

    Conclusions: STAND is a feasible and acceptable model of care that can reach large numbers of individuals. STAND showed preliminary effectiveness on all primary outcome measures. Current directions to improve STAND are described.

    JMIR Ment Health 2023;10:e46200

    doi:10.2196/46200

    Keywords

    depression; anxiety; cognitive behavioral therapy; digital mental health; stratified models of care; model of care; care model; depressive; mental health; CBT; psychotherapy; university; postsecondary; student; college; service delivery; care system; system of care; mHealth; eHealth; online support; student mental health 


    Background

    The public health burden of depression and anxiety is enormous and rising, with effects extending to unemployment, risky substance use, and of greatest concern, suicide. In particular, young adults have seen a substantial increase in anxiety, depression, and suicidal thoughts or behaviors over the past decade [1,2]. Mental health care systems, globally, face significant challenges in meeting the staggering need for care created by this rising public health burden. In particular, access to traditional forms of mental health care is limited by cost, time, effort, and stigma [3], and the number of clinicians trained in evidence-based treatments is grossly inadequate to meet current needs. The urgency of these needs has stimulated the development of 2 broad strategies for increasing access to care: stepped care approaches and digital provision of therapies.

    Increasing Access to Care Using Stepped Care Approaches and Digital Provision of Therapies

    In stepped care models, typically, all patients begin with low-intensity treatments and only nonresponders are moved to the next (more intensive) step of care [4]. Low-intensity treatments often include digital or online therapies, with or without support, as multiple trials targeting anxiety and depressive disorders have shown that these treatments can be as effective as face-to-face therapy [5-8]. While this “fail-first” approach more efficiently allocates resources than traditional approaches that strive to provide the highest level of clinician-delivered care to all patients, it results in delays to more intensive treatment for those who need it [9], and such delays may lead to symptom worsening and increased disability [10], discouragement for new treatments after initial options fail [11], or dropout before a second course of treatment [12]. An emerging body of literature indicates that stratified models, in which individuals are matched to level of care based on predetermined criteria (typically current symptom severity), produce superior outcomes and are more cost-effective in treating depression and anxiety compared with fail-first stepped care models [13-16].

    Digital Technologies for Delivering Measurement-Based Care and Treatment Adaptation

    The use of data collected from patients during the course of treatment for clinical decision-making (measurement-based care) has the potential to greatly improve outcomes but remains the exception rather than the rule [17]. Specifically, measurement-based care typically outperforms usual care on both symptom reduction and reduced dropout rate [18] and is associated with decreased costs and lower odds of patient deterioration during treatment [19]. Without measurement-based care, clinicians have difficulty identifying patients who are at risk for nonresponse or deterioration [20,21]. Consequently, interventions become reactive to “crisis” needs rather than being proactive. Further, decisions about when treatment has been successfully completed tend to be determined by when the patient or their provider “feels ready” to terminate therapy, which may lead to extending treatment far beyond the point when symptom gains have been attained and maintained.

    Even when measurement-based care is utilized, guidelines on how to adjust treatment over time are lacking. As discussed, stepped care approaches typically personalize level of care after a patient has shown nonresponse. Waiting several months to make adaptation decisions could have dire consequences for individuals with depression and anxiety [22]. Moreover, typical stepped care approaches step up only for patients who remained actively engaged in treatment, thereby not addressing the critical issue of how to manage and engage individuals who prematurely discontinue treatment [23].

    Furthermore, typical care models fail to fully consider changes in clinical status after the end of acute treatment; relapse is not uncommon (especially for depression), yet rarely are patients monitored in order to identify for whom reinitiation of care is warranted. By identifying those at risk for nonresponse or worsening of symptoms during or after treatment, adaptations can be made to increase treatment intensity or reenter treatment and prevent further worsening of symptoms and their potential consequences. Routine monitoring of clinical status enables dynamic adaptation of treatment as needed, which has the potential to improve effectiveness and reduce attrition, as patients may be more engaged in treatment when they are receiving what they need most at the time they most need it [24-26]. Adaptive interventions can also enhance cost-effectiveness by increasing the efficiency of service delivery and reducing downstream service costs [27].

    Digital Technologies for Identification and Management of Suicide Risk

    Suicidal thoughts and behaviors are a major public health problem, particularly among young adults [28]. Rates of suicide attempts, nonsuicidal self-injury, and suicidal ideation increase markedly during adolescence [29-31], with suicide death rates increasing as youth move into adulthood. Across all adult age groups, the prevalence of serious suicidal thoughts is highest among young adults aged 18-25 (11%) [32]. There is a critical need to embed methods for detection of risk and strategies for suicide/self-harm prevention within systems of care, which go beyond relying on patients to reach out to providers or to call crisis lines, and beyond monitoring at frequencies that fail to capture tipping points toward high risk. Digital tools may be particularly useful in assessing and monitoring such risk, given that they can promote increased self-disclosure of sensitive topics compared with face-to-face assessment and allow for more rapid, in-the-moment identification and response [33].

    STAND: A Novel, Scalable, Dynamic, Digitally Assisted Evidence-Based Solution to Mental Health Care Delivery

    Screening and Treatment for Anxiety and Depression (STAND) is a stratified stepped care model that incorporates online screening, continuous symptom monitoring over ~10 months, and tiered treatment for anxiety and depressive symptoms, with ongoing suicide risk detection and management (Figure 1). After individuals complete a brief online adaptive assessment of symptom severity and suicide risk (selected modules from the Computer Adaptive Test for Mental Health [CAT-MH]; [34]), they are provided feedback, routed to an appropriate level of care, and scheduled for an orientation or intake session, all within minutes of completion of initial screening. The CAT-MH assessment continuously tracks symptoms (both during and after active treatment) to adapt treatment (ie, move to a higher level of care when symptoms worsened or to a lower level of care for maintenance and relapse prevention) and to rapidly detect and respond to signs of elevated suicide risk. For the open trial described herein, there were 4 levels of care, or tiers.

    Tier 0 was for those with no or minimal symptoms of depression or anxiety on the initial CAT-MH assessment. These participants were offered the option to continue completing the CAT-MH biweekly [34]. Tier 1 was offered to those with mild depression or mild anxiety on the CAT-MH, and included digital therapy. Coaching was offered to tier 1 participants given evidence for its positive effects on retention and clinical outcomes from digital therapy [8]. Tier 2 involved digital cognitive behavioral therapy (CBT) with advanced (doctoral student) coaches, and was offered to individuals with moderate depressive symptoms or moderate to severe anxiety symptoms. Evidence-based psychological treatment with option for medication management was offered to those with severe depressive (or manic) symptoms or suicidality (tier 3). Core active ingredients of CBT (see the “Methods” section) were selected to match each participant’s problems areas and were delivered by clinical psychology graduate students. Therapy was complemented by protocolized medication management as needed (modified from the STAR*D trial; [35]), delivered by psychiatry residents.

    Across all tiers, treatment was adapted according to weekly or biweekly CAT-MH scores (Figure 2). If participants’ scores indicated a need for a higher tier of care at any time, they were contacted to initiate switching to the appropriate level of care. Similarly, participants in tier 3 could be switched to a lower tier and were thus offered access to digital therapy materials when they completed a course of tier 3 treatment and symptoms had shown improvement. Moreover, within tier 3, clinicians used weekly CAT-MH scores to guide decisions regarding treatment strategy (such as switching from behavioral activation to cognitive restructuring).

    Frequent monitoring of suicide risk was conducted using a standardized protocol for responding to risk in real time (see the “Measures” section for details). This approach to risk management was implemented for all levels of care within STAND. In addition to outreach, a positive suicidal triggering alert indicated adaptation to tier 3 if the participant was in a lower tier.

    ‎
    Figure 1. The STAND system of care. STAND: Screening and Treatment for Anxiety and Depression.
    ‎
    Figure 2. Treatment adjustment of a simulated patient in clinical care. CAT-DI: Computerized Adaptive Test-Depression Inventory; STAND: Screening and Treatment for Anxiety and Depression.

    Summary of STAND Improvements to Clinical Practice Paradigms

    STAND improves standard clinical practice paradigms in several ways. First, STAND provides increased access to evidence-based care by utilizing digital tools for a large proportion of the patient experience from screening through service enrollment to service delivery and risk management. Second, the emphasis on adapting both the level of care and intervention type as needs change is a major improvement over existing approaches (eg, stepped care), which typically wait until nonresponse is established at predefined points that are months apart before offering an alternative, if at all [16]. Third, STAND focuses on maintaining gains and minimizing relapse, which are rare in systems of care, despite the many benefits to individuals and a health system [36]. Finally, the STAND program focuses on personalization of services based on continuous symptom monitoring and clinical decision-making (both with regard to tier triaging and adaptation as well as choice of treatment modality within tier 3, the highest level of care) that is expected to reduce attrition and improve symptomatic and functional outcomes [23].

    This study aimed to develop and evaluate the preliminary acceptability, feasibility, and effectiveness of the STAND program in a large sample of college students in an open trial. It was hypothesized that participants would report high satisfaction with the program, and would show significant improvement in anxiety, depression, and suicide risk severity scores over the course of participation in STAND.


    Participant Recruitment and Initial Triaging

    Recruitment and enrollment into the open trial occurred between Fall 2017 and Spring 2020. All University of California, Los Angeles (UCLA) students were made aware of the screener through announcements from UCLA’s Chancellor, tabling at campus events, campus residence hall flyers, classroom announcements, web-based advertisements, banners, and email blasts. Students completed the CAT-MH online and received immediate feedback regarding their scores and recommended tier of care. Those who were interested in and eligible for the study and who provided informed consent were then scheduled for a baseline visit. At the baseline visit, research staff gathered pertinent information about treatment history and other demographics, after which participants were notified about how to start care in their assigned tier, with ongoing completion of the CAT-MH and other assessments over the course of 40 weeks.

    Inclusion/Exclusion Criteria

    All matriculating UCLA students who were at least 18 years of age and had access to the internet were eligible to complete the initial screener. Additional inclusion criteria for screening were English fluency, willingness to install an app on their phone, and no plans (eg, upcoming long vacations or studying abroad) that would interfere with participation. Additional exclusion criteria following screening for those entering clinical care (ie, tier 3) were severe psychopathology requiring intensive day-treatment or inpatient care (eg, severe eating disorder or substance use disorder, multiple [2+] suicide attempts leading to hospitalization within the past 6 months, significant psychotic symptoms not part of a severe major depressive episode, or bipolar manic episode) and were determined at the initial assessment through further screening. Participants in tier 3 were also excluded if they were unwilling to provide a blood sample (for secondary analyses outside of the scope of this manuscript) or had a current psychiatrist or provider and were unwilling to transfer their care to the study team for the period that they will receive treatment in STAND.

    Interventions

    Tier 0

    Participants whose Computerized Adaptive Test-Depression Inventory (CAT-DI) and Computerized Adaptive Test-Anxiety Inventory (CAT-ANX) scores were in the normal range were offered tier 0 or monitoring of their status using biweekly CAT-MH.

    Tier 1

    Those whose depression or anxiety scores were in the mild range were offered tier 1 that included an evidence-based digital CBT called ThisWayUp [37]. ThisWayUp provides 6 lessons on CBT strategies for anxiety and depression including behavioral activation, cognitive restructuring, and exposure. This digital therapy program was selected, as randomized controlled trials have demonstrated evidence to support its efficacy [37,38]. Digital CBT was accompanied by the offer of up to 8 phone or video coaching sessions by extensively trained undergraduate student coaches (see the “Methods” section for details) to support the digital therapy material and to assist with troubleshooting (rather than to teach clinical content). As human support has been shown to increase efficacy and retention in digital therapies [8], coaching sessions were offered each week after participants independently completed a ThisWayUp lesson. Coach training had 4 sequential components (see [39] for details). Didactic, interactive classroom meetings and small-group coaching support meetings focused on content and general coaching skills. Discussions were held weekly with opportunities to practice coaching skills under close supervision and with small-group feedback, and to achieve core competencies in professionalism, diversity awareness, ethical standards, and reflective practice and self-care. Advancement to the Certified Coach status was required before coaches could provide video-chat coaching to study participants, under supervision from program faculty. Coaching was initially provided to participants in a group format, and later was provided in an individual 1-on-1 format.

    Tier 2

    Tier 2 was offered to those who scored in the moderate range for depression or moderate to severe range for anxiety. Participants were assigned to complete ThisWayUp and were strongly encouraged to attend weekly face-to-face, video-chat, or phone coaching sessions following their independent completion of each ThisWayUp lesson. In addition to being trained in ThisWayUp, advanced (doctoral students) coaches participated in the tier 3 clinical training (described below) to gain a deeper knowledge of cognitive and behavioral principles. Given their more advanced training, tier 2 advanced coaches had more flexibility to support the CBT material in the coaching sessions. They received weekly supervision by licensed clinical psychologists.

    Tier 3

    Evidence-based psychological treatment with option for medication management was offered to those with severe depressive (or manic) symptoms or suicidality. Evidence-based psychological treatments often consist of multicomponent packages of different cognitive and behavioral strategies, such as cognitive restructuring, self-monitoring, problem solving, relaxation training, assertiveness training, or exposure therapy. Such combination packages prohibit understanding of which ingredient was most accountable for therapeutic change for each individual and the processes responsible for therapeutic change [40,41]. Approaches that rely upon specific active ingredients may enable better treatment personalization and greater treatment efficacy (eg, [42]). Thus, core cognitive behavioral ingredients [43] (Table 1) were delivered to patients according to their individualized needs, as was ascertained during a functional assessment. The goal of this clinician-administered, semistructured interview assessment was to identify the primary processes (eg, fear/avoidance and deficits in extinction; inactivity/sadness and deficits in response-contingent positive reinforcement) that were driving or maintaining symptoms, and then to select the therapeutic strategy that most directly targeted those processes (eg, exposure therapy and behavioral activation). Therapy was complemented by protocolized medication management as needed (modified from the STAR*D trial; [35]), delivered by psychiatry residents. Weekly CAT-MH scores made available to clinicians on a dashboard enabled continuous monitoring of treatment response and need for treatment adaptation. For example, if CAT-MH depression scores did not show improvement within 6 weeks of behavioral activation, the functional assessment was repeated, followed by possible switching to either cognitive restructuring or mindfulness-based approaches. Similarly, medication management changes were considered in light of poor response on the CAT-MH scores.

    Weekly individual therapy sessions were delivered by clinical psychology doctoral students and psychiatry residents, as part of their in-house training programs. They were trained through a series of workshops and didactics on each of the evidence-based active ingredients (Table 1), consisting of didactic material, demonstrations of skills by the experts, and role-play practices of skills. They were taught to use the functional assessment and treatment decision-making algorithm to identify the principal process to target and the first-line treatment ingredient that mapped onto that target (eg, worry—cognitive restructuring; low activity/sadness—behavioral activation); and the process for monitoring symptoms to identify when a functional assessment and treatment plan should be updated. After initial training, psychology trainees participated in weekly supervision and an interdisciplinary case conference.

    Psychiatry trainees were trained in the STAND medication management algorithm and received regular supervision from attending psychiatrists. Psychiatrists also participated in the same weekly interdisciplinary case conference as psychology trainees, which also included members of the campus crisis response team who could assist with coordination of outreach for high-risk participants.

    Table 1. Matching of clinical care to patient problem area.
    Problem areaProcess targetedFirst-line therapy module + medication as appropriateProcess targetedSecond-line therapy module + medication as appropriate
    Low activity/sadnessLow response-contingent positive reinforcementBehavioral activation (mood monitor, activity schedule, problem solve barriers, sleep schedule for barriers)Cognitive distortions and ruminationCognitive restructuring or mindfulness and value-driven action and problem solving
    AnhedoniaReward hyposensitivityPleasant event scheduling (hedonic and eudaimonic rewards), memory specificity recountingReward hyposensitivityCognitive restructuring with positive focus and cultivating positivity
    Fear/phobiaDeficits in extinction, safety learning, avoidanceExposure therapyNegative cognitive bias and poor social skillsCognitive restructuring, mindfulness, value-driven action or social skills training
    WorryNegative cognitive biasCognitive restructuring or mindfulness, value-driven actionAvoidance (experiential and in vivo)Exposure therapy or mindfulness and value-driven action or social skills training
    Sleep dysregulationSleep dysregulationBrief behavioral therapy for insomniaNegative cognitive biasCognitive restructuring
    Trauma—fearDeficits in extinction, safety learning, avoidanceImaginal and in vivo exposureNegative cognitive biasCognitive restructuring and impact statement
    Trauma—guilt, shame, cognitive distortionsNegative cognitive biasTrauma narrative with cognitive restructuring and impact statementN/AaN/A
    Chronic suicidality, self-harm, affective instabilityLow tolerance of distressDistress tolerance skills in DBTbPoor emotion regulation and interpersonal difficultiesEmotion regulation skills and interpersonal effectiveness in DBT
    ManiaCircadian dysregulationBrief behavioral therapy for insomniaN/AN/A
    Major life stressors (any symptom profile)Poor copingProblem solving for controllable stressor, mindfulness, value-driven action for uncontrollable stressorN/AN/A
    Interpersonal relations (any symptom profile)Social skills deficitsInterpersonal effectiveness training in DBTN/AN/A

    aN/A: not applicable (ie, no second-line therapy module for trauma, mania).

    bDBT: dialectical behavioral therapy.

    Tier Adaptation

    Across all tiers, treatment was adapted according to weekly or biweekly CAT-MH scores. If participants’ scores indicated a need for a higher tier of care at any time (ie, showed a 30% worsening of symptom severity from baseline over 2 consecutive assessments), they were contacted to initiate switching to the appropriate level of care. Similarly, participants in tier 3 were switched to a lower tier when their symptoms remained consistently low (ie, in the moderate to normal range of severity following ~12-16 weeks of treatment). Thus, when participants completed treatment in tier 3 they were offered access to ThisWayUp materials from tiers 1 and 2. Moreover, within tier 3, clinicians used weekly CAT-MH scores to switch treatment strategies (such as switching from behavioral activation to cognitive restructuring).

    Suicide Risk Monitoring

    The CAT-MH [34] digitally identified participants at risk of suicide; scores above a preset threshold triggered an alert to initiate a risk management protocol. The CAT-MH includes (but is not limited to) the screener module from the Columbia Suicide Severity Rating Scale (CSSRS; [44]). The CSSRS yields a positive case as indicated by the presence of current suicide ideation plus current intent or current method or recent suicidal behavior (eg, suicide attempt, steps taken toward an attempt). A positive case automatically triggered a 24/7 crisis service that made up to 3 attempts to contact the participant to conduct a risk assessment and appropriate care management. All participants in the STAND program consented to be contacted by this crisis service at study entrance. Relevant STAND staff and clinicians were also informed, and could make additional outreach efforts if the crisis service outreach attempts were unsuccessful. In the event that the participant was unable to be reached by the crisis service or STAND staff and clinicians, the deployment of other community-based crisis units was considered on a case-by-case basis.

    Measures

    Clinical Outcomes

    The primary outcome measure was the CAT-MH [34]. The CAT-MH is a commercially available (Adaptive Technologies) computerized adaptive test for assessing a variety of symptoms including those of anxiety (CAT-ANX; [45]), depression (CAT-DI; [46]), and suicide risk severity (Computerized Adaptive Test-Suicide Scale [CAT-SS]; [47]). The CAT-MH scores included symptom severity as a cumulative score of endorsed symptoms (0-100) or severity categories per symptom cluster for the CAT-DI and CAT-ANX (normal, mild, moderate, and severe) based on empirically derived cut points.

    Depression scores (CAT-DI severity) of <50.0 were considered normal, 50.0-65.0 mild, >65.0-75.0 moderate, and ≥75.0 severe for the majority of the recruitment period (through June 2019) [46,48], and anxiety scores (CAT-ANX severity) of <35.0 were considered normal, 35.0-50.0 mild, >50.0-65.0 moderate, and >65.0 severe [45,48]. From June 2019 to June 2020 (ie, end of recruitment), CAT-DI categories were slightly changed based on findings from [48] that led to recommendations from the CAT-MH developer team (R. D. Gibbons, PhD, personal communication, June 9, 2019). During this latter period, the threshold for mild depressive symptoms was lowered such that scores <35.0 were considered normal, and the mild range was 35.0-65.0. The CAT-MH was administered weekly (tier 3) or biweekly (tiers 0-2) across the entire course of participation in STAND. The CAT-MH categories were used to triage participants into initial tier, and were used to inform adaptation to tiers (ie, tier switching) during the next 40 weeks. Therefore, the primary outcomes gathered from the CAT-MH (ie, CAT-ANX severity score, CAT-DI severity score, and CAT-SS severity score) were the dependent variables. See [34,45-47] for details regarding the excellent psychometric properties of the CAT-ANX, CAT-DI, and CAT-SS.

    Participant Acceptability

    Participants were asked to provide feedback on their experience in the STAND program when they reached the final 40-week assessment (this assessment was initiated halfway through the study). Participants were asked to rate how logical the program seemed (on a scale from 1 to 9, where 1=not at all logical, 5=somewhat logical, and 9=very logical), to what extent the program met their expectations (on a scale from 1 to 5, where 1=completely failed to meet expectations, 3=met expectations, and 5=greatly exceeded expectations), how satisfied they were with the program (on a scale from 1 to 7, where 1=extremely dissatisfied, 4=neither satisfied nor dissatisfied, and 7=extremely satisfied), and how likely they would be to recommend the program to a friend (on a scale from 1 to 7, where 1=extremely unlikely, 4=neither likely nor unlikely, and 7=extremely likely).

    Statistical Analysis

    Descriptive statistics were used to characterize the data and to identify indices of feasibility and acceptability. With regard to the primary outcome data, a series of hierarchical linear models (1 per tier per outcome measure) were conducted with time as the level 1 predictor and the outcome of interest (CAT-ANX, CAT-DI, and CAT-SS scores for tier 3, and CAT-ANX and CAT-DI scores for tiers 1 and 2). In line with an effectiveness paradigm, we included all participants who were assigned to a tier and had at least one CAT-MH datapoint, regardless of how many sessions/lessons were completed or how much missing data on the CAT-MH was observed. Parallel sets of analyses were run with and without a covariate (number of sessions/lessons completed on level 2). The survey data were not consistently collected for participants in tiers 1 and 2 after completion of the ThisWayUp program. Therefore, results for tiers 1 and 2 reflect CAT-MH scores over the course of ThisWayUp program completion only (ie, 8 weeks). Further, of the 11 participants who completed a baseline assessment for tier 0, only 5 participants completed more than 3 additional assessments. For this reason, CAT-MH scores are not reported for tier 0.

    As a secondary analysis, the percentages of those within each tier who achieved at least 30% improvement on each CAT-MH index (ie, CAT-DI, CAT-ANX, and CAT-SS) were calculated among participants considered at least moderately engaged (ie, completed at least four sessions/lessons).

    Ethics Approval

    The project was approved by the UCLA Institutional Review Board (approval numbers 17-001938, 16-001395, and 17-001365).


    Indices of Feasibility

    Coach and Clinician Training

    A total of 530 student coaches (including undergraduates in the resilience peer network for tier 1 and doctoral students for tier 2) were trained (to an adequate or better level, thus allowing them to deliver coaching; see [39]), and 47 psychology and psychiatry clinicians were trained to deliver psychological and psychiatric care in tier 3. For further details on student coach training and competency assessment, see [39].

    STAND Participant Uptake, Engagement, and Adaptations

    A total of 4845 unique students completed screening using the CAT-MH; 3580 out of the 4845 students who completed screening (73.89%) were eligible and offered care in tiers 1, 2, or 3; 516 out of the 3580 eligible students who were offered care (14.41%) initiated therapy in one of the tiers. As many as 327 individual suicide risk alerts were detected at screening, and additional risk assessments (after screening) were conducted for 1054 alerts indicating a risk of suicide or severely worsening depression over the course of the study.

    A total of 180 students received care (ie, completed at least one lesson) in tier 1 and 197 in tier 2. Those in tier 1 completed on average 4.22 (SD 1.91) out of 6 digital therapy lessons, which was similar or better than adherence reported in previous studies [37,38]. Tier 1 participants completed on average 0.88 (SD 2.07) coaching sessions. The correlation between number of coaching sessions and digital therapy lessons completed in tier 1 was small but significant and positive (r=0.25, P<.001). Students in tier 2 completed on average 4.09 (SD 2.06) out of 6 digital therapy lessons and 2.09 (SD 2.56) group or 1-on-1 coaching sessions. The correlation between number of coaching sessions and digital therapy lessons completed in tier 2 was moderately large and significant (r=.54, P<.001). With regard to “tier switching”/adaptation, only 5 participants who were in tiers 1 or 2 were moved up to tier 3 due to a symptom worsening or suicide risk alert. A total of 139 students received clinical care in tier 3 (ie, attended at least one therapy session), and attended on average 13.86 (SD 7.94) therapy sessions. Following the acute course of tier 3 treatment, all tier 3 participants were offered online CBT via ThisWayUp (tier 1 without coaching).

    Demographic, diagnostic, and baseline clinical data for tiers 0-3 are presented in Table 2.

    Table 2. Demographic characteristics.
    CharacteristicsaTier 0Tier 1Tier 2Tier 3
    Completed baseline assessment, n11225239144
    Age (years), mean (SD)22.91 (6.56)23.01 (5.81)21.93 (4.47)21.14 (3.36)
    Assigned sex at birth, n

    		Female5160184107

    		Male6635336
    Current gender identity, n

    		Female5159181105

    		Male6615135

    		Transgender0120

    		Does not identify as either0233
    Sexual orientationb, n

    		Asexual0101

    		Bisexual0122527

    		Heterosexual/straight8498281

    		Homosexual, gay or lesbian1257

    		Queer0366

    		Questioning or unsure18129

    		Not listed1030

    		Prefers not to answer0125
    Marital status, n

    		Single, never married10195211131

    		Living with partner11094

    		Domestic partnership0021

    		Married014114

    		Separated0000

    		Divorced0201

    		Do not know0011

    		Prefers not to answer0221
    First language, n

    		English11159161104

    		Spanish0222315

    		Other0425324
    Country of birth, n

    		United States10171174111

    		Other1526332
    Immigration status, n

    		Domestic11194201126

    		International0253113

    		Undocumented0442

    		Other0000

    		Prefers not to answer0012
    Racioethnic group, n

    		Hispanic1405936

    		Non-Hispanic White3857135

    		Non-Hispanic Black24104

    		Non-Hispanic Asian4727152

    		Non-Hispanic multiple1171712

    		Non-Hispanic Native American/Pacific Islander0000

    		Unknown0594
    Highest level of education, n

    		High school graduate2344443

    		General educational development or equivalent0100

    		Some college4859659

    		Associate’s degree (occupational, technical, or vocational)0130

    		Associate’s degree (academic program)0122714

    		Bachelor’s degree5664618

    		Master’s degree023188

    		Professional degree0120

    		PhD0010

    		Prefers not to answer0001
    Employment, n

    		Working3202616

    		Only temporarily laid off, sick leave, or maternity leave0001

    		Looking for work, unemployed0662

    		Disabled, permanently or temporarily0001

    		Student8194201121

    		Other0342
    Student enrollment status, n

    		Full time10215229138

    		Part-time0475
    College level, n

    		Freshman3303534

    		Sophomore0294125

    		Junior1426132

    		Senior2373626

    		Graduate student4705522

    		Professional student0954

    		Other0230
    Baseline CAT-MHc Depression

    		Depression severity, mean (SD)33.63 (9.04)49.90 (11.44)59.96 (12.25)73.21 (11.32)

    		Minimal to none depression, n1096365

    		Mild depression, n110911621

    		Moderate depression, nN/Ad196561

    		Severe depression, nN/AN/A2157
    Baseline CAT-MH Anxiety

    		Anxiety severity, mean (SD)18.42 (12.74)39.65 (12.51)55.60 (14.37)60.14 (16.10)

    		Minimal to none anxiety, n1082198

    		Mild anxiety, n11047432

    		Moderate anxiety, nN/A337945

    		Severe anxiety, nN/A66659
    Baseline CAT-MH Suicidality
     Suicidal ideation diagnosis, nN/A2624
     Suicidality severity, mean (SD)18.49 (12.95)39.62 (12.67)50.29 (11.99)62.19 (11.76)

    aParticipants were able to enroll in the study at both wave 1 and wave 2 of recruitment. Therefore, some participants may be represented twice if they chose to reenroll in the study.

    bThis question was introduced partway through the study, therefore not all participants had the opportunity to respond.

    cCAT-MH: Computer Adaptive Test for Mental Health.

    dN/A: not applicable.

    STAND Clinical Outcomes (Preliminary Effectiveness)

    Overview

    There was no difference in the pattern or significance of any findings when the covariate (ie, number of sessions/lessons completed) was included. Therefore, analyses without statistically adjusting for the number of sessions/lessons completed are reported below as a conservative approach. Specific trajectories for each tier/outcome combination are described below in more detail. Figures 3 and 4 show the decline slopes for the CAT-ANX and CAT-DI within each tier, and Figure 5 shows the CAT-SS decline slope in tier 3.

    ‎
    Figure 3. Tier 1 changes in CAT-DI and CAT-ANX scores over time. CAT-ANX: Computerized Adaptive Test-Anxiety Inventory; CAT-DI: Computerized Adaptive Test-Depression Inventory; SI: suicidal ideation.
    ‎
    Figure 4. Tier 2 changes in CAT-DI and CAT-ANX scores over time. CAT-ANX: Computerized Adaptive Test-Anxiety Inventory; CAT-DI: Computerized Adaptive Test-Depression Inventory.
    ‎
    Figure 5. Tier 3 changes in CAT-DI, CAT-ANX, and CAT-SS scores over time. CAT-ANX: Computerized Adaptive Test-Anxiety Inventory; CAT-DI: Computerized Adaptive Test-Depression Inventory; CAT-SS: Computerized Adaptive Test-Suicide Scale.
    Tier 1

    The CAT-DI and CAT-ANX scores significantly declined over the course of STAND participation (b=–2.03, t732=–10.71, P<.001 and b=–2.26, t732=–12.47, P<.001, respectively). Based on the regression lines, scores moved from the mild to normal range for both depression and anxiety severity (Figure 3). Among participants who completed at least four digital therapy lessons (n=110), 51 participants (46.4%) showed ≥30% improvement in CAT-DI scores and 62 participants (56.4%) showed ≥30% improvement in CAT-ANX scores, notable for a sample likely to have a floor effect, with minimal room for significant symptom improvement.

    Tier 2

    The CAT-DI and CAT-ANX scores significantly declined over the course of STAND participation (b=–2.10, t696=–9.64, P<.001 and b=–2.73, t696=–12.42, P<.001, respectively). As shown in Figure 4, although scores for depression began and ended in the mild range (following the adjustment of categories in June 2019), they moved from the top of the mild range into the lower end of the mild range (and into what was considered the normal range prior to June 2019). Anxiety severity scores moved from the moderate range to the mild range. Among participants who completed at least four digital therapy lessons (n=106), 33 participants (31.1%) showed ≥30% improvement in CAT-DI scores and 47 participants (44.3%) showed ≥30% improvement in CAT-ANX scores.

    Tier 3

    The CAT-DI and CAT-ANX scores significantly declined over the course of STAND participation (b=–0.61, t2805=–11.91, P<.001 and b=–0.51, t2805=–9.64, P<.001, respectively). The CAT-SS severity also declined over time (b=–0.52, t2805=–11.05, P<.001). As depicted in Figure 5, the regression line shows depression scores moved from the moderate range to the mild range (and what was considered the normal range prior to June 2019) and anxiety scores moved from the moderate range to the normal range. Among participants who completed at least four therapy sessions (n=131), 74 participants (56.5%) showed ≥30% improvement in CAT-DI scores, 81 participants (61.8%) showed ≥30% improvement in CAT-ANX scores, and 78 participants (59.5%) showed ≥30% improvement in CAT-SS scores.

    Participant Acceptability

    Results on the subsample (n=79) who completed the satisfaction survey are reported in Table 3. In sum, participants across tiers 1-3 rated their care as highly logical (scores ranging from 6.65 to 7.60 out of 9), that the program succeeded in meeting their expectations (scores ranging from 3.47 to 3.95 out of 5), that they were satisfied with the program (scores ranging from 5.24 to 5.90 out of 7), and that they would recommend it to a friend (scores ranging from 5.47 to 6.14 out of 7). Tier 0 participants also rated these categories highly but scores should be interpreted with caution, given the small sample size in tier 0 (Table 3).

    Table 3. Program feedback.
    VariablesTier 0Tier 1Tier 2Tier 3
    Participants, n3171742
    Logical, mean (SD)7.33 (0.58)6.65 (2.23)6.59 (1.28)7.60 (1.62)
    Expectation, mean (SD)3.00 (0.00)3.47 (1.01)3.76 (0.83)3.95 (1.04)
    Satisfaction, mean (SD)4.67 (0.58)5.24 (1.56)5.47 (1.01)5.90 (1.25)
    Recommendation, mean (SD)4.67 (0.58)5.65 (1.37)5.47 (0.94)6.14 (1.18)

    Summary of Outcomes

    This preliminary study demonstrated that STAND, a technology-assisted and scalable system of care, is feasible and effective on a college campus. STAND facilitated the deployment of a significant number of suicide risk assessments as needed in real-time, a critical index of risk prevention. As part of the STAND model, a large number of coaches (n=530) and clinicians (n=47; in psychiatry and psychology) were trained to provide support for or deliver evidence-based treatment for anxiety and depression, respectively. Care was delivered to 516 UCLA students (ie, those who completed at least one session of therapy/digital therapy lesson), who, on average, experienced significant reductions in their anxiety and depression symptoms across all tiers. Participant acceptability on the satisfaction survey revealed moderately high ratings of satisfaction with participants’ assigned tier. Importantly, the stratified model of initial triaging to the appropriate level of care was successful in that the vast majority of participants showed significant improvement in their originally assigned tier. Indeed, only 5 participants required a higher level of care during their initial participation in a lower tier.

    Although only a small percentage of participants who completed the initial CAT-MH screening assessment and were offered care actually initiated care in one of the tiers (516/3580, 14.41%), there is a wide range of uptake reported in similar digital interventions, with the rate in this study falling within that range [5,49,50], and promising outcomes emerged among those who did participate. Additionally, a prior analysis of racial/ethnic differences in STAND treatment uptake midway through the recruitment period found that students from ethnic/racial minority groups were not less likely to enroll or engage in treatment compared with non-Hispanic White students (see [51]), indicating that STAND has the potential to reach and be acceptable to a diverse student population, which is important for our current work that aims to implement STAND in underserved communities. Of note, although significant improvement over time was observed across all outcome measures within each tier, a relatively lower percentage of participants in tier 2 showed clinically meaningful change across outcomes than tiers 1 and 3, suggesting that there may be some participants who were assigned to tier 2 who may have benefited more from a higher level of care. Our current work is examining whether including other variables (eg, life stress, trauma, demographics) in multivariate predictive models alongside symptom severity can improve upon and personalize tier triaging to improve clinical outcomes.

    Taken together, these findings support the feasibility, acceptability, and preliminary effectiveness of STAND; and this initial demonstration of the STAND model of care provided an opportunity to identify areas for improvement and refinement of the model, including increasing engagement and retention in care.

    Limitations

    The data presented represent the initial implementation of this system in an open trial. Therefore, conclusions about the effectiveness of STAND compared with another treatment cannot be drawn. Further, as is common practice with initial implementations that aim to iteratively refine and improve upon methodology, some minor data-driven changes were made throughout the course of the study (eg, changing CAT-DI categories based on incoming psychometric information, shifting from group to individual coaching sessions). Another limitation was the low response rate, particularly in tier 0 and following the ThisWayUp intervention in tiers 1 and 2. This low response rate precludes our ability to evaluate the longer-term outcome in the lower tiers. Finally, with regard to booster treatment during the monitoring period following the initial acute phase of care, in the present demonstration of STAND, we opted to provide the digital therapy to all participants who had completed tier 3 treatment. However, we did not track usage during this preventative/maintenance phase. Future work we have planned will more precisely identify those who may need this additional booster and at what point in their trajectory. Finally, we did not systematically collect demographic data (eg, gender, race/ethnicity) on coaches and clinicians, which would have been interesting to examine both descriptively and correlations between participant/clinician match on demographic factors and clinical outcomes/engagement. Our current work is now collecting these data on coaches in tier 2 as well as conducting a randomized trial to examine the potential impact of racial/ethnic matching between coaches and participants on engagement and clinical outcomes.

    Lessons Learned and Current Directions in STAND

    Two notable observations in this initial implementation of STAND were the low initial uptake of STAND services, and the relatively modest ongoing engagement in the digital therapy tiers. With regard to the former, the vast minority of students who were screened and eligible to receive services in STAND actually engaged in care. Despite our multipronged approach to recruitment, and the possibility that a lack of financial incentive to participants in the study may have contributed to some extent in low uptake, improvements to our recruitment strategies are clearly needed. Specifically, additional research is needed to identify recruitment strategies (possibly tailored to each new population that adopts STAND) that provide messaging that resonates with the student population. We are currently partnering with social marketing experts to use inclusive, nonstigmatizing messaging and improve upon our outreach efforts; and current research projects underway are seeking feedback from students who did not engage in care to identify barriers and develop solutions.

    With regard to the latter, consistent with prior evidence [49,52], retention in digital therapy was somewhat problematic, and utilization of coaching support was particularly low. This is a ubiquitous challenge in digital therapy research for which creative solutions are being developed and evaluated (eg, the use of gamification, avatars, personalization, feedback, and individual support; see [53] for a review). Although our digital therapy approach already incorporates several of these features, we have made several changes to improve retention in digital therapy and coaching, which are currently being implemented and evaluated in ongoing trials. These include automatic scheduling of all digital therapy participants for coaching sessions, rather than making it an add-on option, and personalizing digital therapy by switching from a unified, one-size-fits-all approach to a modularized suite of digital components that are selected using embedded measurement systems to match an individual’s presenting concerns. The change to a personalized approach to module selection is expected to increase retention given evidence for students to prefer tailored online therapy [54]. Current studies are also underway to develop and evaluate engagement strategies (with an emphasis on culturally responsive messaging) and to implement text messaging as a method to increase engagement.

    Other changes to our current iteration of STAND include collapsing tiers 1 and 2 into 1 tier (tier 2), with streamlined coach training and supervision, as well as provision of a self-guided wellness digital program in the original tier 0 (now called tier 1). Given that we found such low engagement in tier 0 (the monitoring-only program), we were unable to analyze data collected from tier 0 participants. Possibly, students did not see the benefit of participating in a monitoring-only program when they had minimal to no symptoms (and were not being financially compensated). Our expectation is that by providing digital prevention tools framed as stress management and wellness, students with no symptoms will be more interested in participating in STAND.

    Conclusions

    The need for effective and scalable treatments for depression and anxiety is enormous given the high (and rising) rates of these disorders and pervasive lack of availability of accessible, evidence-based care. The Screening and Treatment of Anxiety and Depression (STAND) system of care was developed to address this need. STAND is based on the principles of stratified stepped care, adaptation of care in response to continuous symptom monitoring, and routine detection and prevention of suicidality. The STAND system of care was tested in a large sample of UCLA students, where it led to significant reductions in depression and anxiety symptoms and suicide risk. Efforts are underway to continue to refine and enhance the STAND system of care, while increasing its reach, with an emphasis on reaching underserved, diverse populations, conducting randomized clinical trials to identify the optimal ways to deliver STAND, and developing a pathway for sustainability of STAND as it is rolled out on a larger scale. These sustainability efforts will include stakeholder input and economic analyses to identify the specific resources needed to carry out each component of STAND, including ensuring that a sufficient clinical and coaching workforce is available and trained to fidelity in partnership with a college campus that wishes to implement STAND.

    Acknowledgments

    We thank Ariel Seroussi, MD, Jonathan Heldt, MD, Amanda Gorlick, PhD, and Raphael Rose, PhD, for their clinical supervisory roles in the project, as well as the numerous clinicians, coaches, and research staff who supported this project.

    Conflicts of Interest

    None declared.

    1. Fang Y, Ji B, Liu Y, Zhang J, Liu Q, Ge Y, et al. The prevalence of psychological stress in student populations during the COVID-19 epidemic: a systematic review and meta-analysis. Sci Rep. Jul 15, 2022;12(1):12118. [FREE Full text] [CrossRef] [Medline]
    2. Lipson S, Lattie EG, Eisenberg D. Increased Rates of Mental Health Service Utilization by U.S. College Students: 10-Year Population-Level Trends (2007-2017). Psychiatr Serv. Jan 01, 2019;70(1):60-63. [FREE Full text] [CrossRef] [Medline]
    3. Andrade L, Alonso J, Mneimneh Z, Wells J, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol. Med. Aug 09, 2013;44(6):1303-1317. [FREE Full text] [CrossRef]
    4. Cross S, Hickie I. Transdiagnostic stepped care in mental health. Public Health Res Pract. Apr 27, 2017;27(2):1. [FREE Full text] [CrossRef] [Medline]
    5. Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital Mental Health Interventions for Depression, Anxiety, and Enhancement of Psychological Well-Being Among College Students: Systematic Review. J Med Internet Res. Jul 22, 2019;21(7):e12869. [FREE Full text] [CrossRef] [Medline]
    6. Moshe I, Terhorst Y, Philippi P, Domhardt M, Cuijpers P, Cristea I, et al. Digital interventions for the treatment of depression: A meta-analytic review. Psychol Bull. Aug 2021;147(8):749-786. [CrossRef] [Medline]
    7. Pauley D, Cuijpers P, Papola D, Miguel C, Karyotaki E. Two decades of digital interventions for anxiety disorders: a systematic review and meta-analysis of treatment effectiveness. Psychol. Med. 2023;53(2):267-279. [FREE Full text] [CrossRef]
    8. Schueller S, Tomasino KN, Mohr DC. Integrating human support into behavioral intervention technologies: The efficiency model of support. Clinical Psychology: Science and Practice. Mar 2017;24(1):27-45. [FREE Full text] [CrossRef]
    9. Trivedi MH, Rush AJ, Wisniewski SR, Nierenberg AA, Warden D, Ritz L, et al. STAR*D Study Team. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry. Jan 2006;163(1):28-40. [CrossRef] [Medline]
    10. Gunn J, Wachtler C, Fletcher S, Davidson S, Mihalopoulos C, Palmer V, et al. Target-D: a stratified individually randomized controlled trial of the diamond clinical prediction tool to triage and target treatment for depressive symptoms in general practice: study protocol for a randomized controlled trial. Trials. Jul 20, 2017;18(1):342. [FREE Full text] [CrossRef] [Medline]
    11. LaLonde L, Good J, Orkopoulou E, Vriesman M, Maragakis A. Tracing the missteps of stepped care: Improving the implementation of stepped care through contextual behavioral science. Journal of Contextual Behavioral Science. Jan 2022;23:109-116. [FREE Full text] [CrossRef]
    12. Wells J, Browne MO, Aguilar-Gaxiola S, Al-Hamzawi A, Alonso J, Angermeyer MC, et al. Drop out from out-patient mental healthcare in the World Health Organization's World Mental Health Survey initiative. Br J Psychiatry. Jan 2013;202(1):42-49. [CrossRef] [Medline]
    13. Delgadillo J, Ali S, Fleck K, Agnew C, Southgate A, Parkhouse L, et al. Stratified Care vs Stepped Care for Depression: A Cluster Randomized Clinical Trial. JAMA Psychiatry. Feb 01, 2022;79(2):101-108. [FREE Full text] [CrossRef] [Medline]
    14. Firth N, Barkham M, Kellett S. The clinical effectiveness of stepped care systems for depression in working age adults: a systematic review. J Affect Disord. Jan 01, 2015;170:119-130. [FREE Full text] [CrossRef] [Medline]
    15. Ho FY, Yeung W, Ng TH, Chan CS. The Efficacy and Cost-Effectiveness of Stepped Care Prevention and Treatment for Depressive and/or Anxiety Disorders: A Systematic Review and Meta-Analysis. Sci Rep. Jul 05, 2016;6:29281. [FREE Full text] [CrossRef] [Medline]
    16. van Straten A, Hill J, Richards D, Cuijpers P. Stepped care treatment delivery for depression: a systematic review and meta-analysis. Psychol. Med. Mar 26, 2014;45(2):231-246. [FREE Full text] [CrossRef]
    17. Zimmerman M, McGlinchey JB. Why don't psychiatrists use scales to measure outcome when treating depressed patients? J Clin Psychiatry. Dec 2008;69(12):1916-1919. [CrossRef] [Medline]
    18. de Jong K, Conijn JM, Gallagher RA, Reshetnikova AS, Heij M, Lutz MC. Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis. Clin Psychol Rev. Apr 2021;85:102002. [FREE Full text] [CrossRef] [Medline]
    19. Lewis C, Boyd M, Puspitasari A, Navarro E, Howard J, Kassab H, et al. Implementing Measurement-Based Care in Behavioral Health: A Review. JAMA Psychiatry. Mar 01, 2019;76(3):324-335. [FREE Full text] [CrossRef] [Medline]
    20. Constantino MJ, Boswell JF, Bernecker SL, Louis Castonguay MG. Context-Responsive Psychotherapy Integration as a Framework for a Unified Clinical Science: Conceptual and Empirical Considerations. Journal of Unified Psychotherapy and Clinical Science. 2013;2(1):1-20. [FREE Full text]
    21. Hickie I, Scott EM, Cross SP, Iorfino F, Davenport TA, Guastella AJ, et al. Right care, first time: a highly personalised and measurement-based care model to manage youth mental health. Med J Aust. Nov 2019;211 Suppl 9:S3-S46. [CrossRef] [Medline]
    22. Carpenter JS, Iorfino F, Cross SP, Davenport TA, Hermens DF, Rohleder C, et al. Combining clinical stage and pathophysiological mechanisms to understand illness trajectories in young people with emerging mood and psychotic syndromes. Medical Journal of Australia. 2019;211(supplement 9):S12-S22. [CrossRef]
    23. Cross SP, Davenport TA, Scott EM, Iorfino F, Sawrikar V, Hickie IB. A service delivery model to support highly personalised and measurement-based care in youth mental health. Med J Aust. 2019;211(Supplement 9):S42-S46. [FREE Full text]
    24. Hamilton M, Hetrick SE, Mihalopoulos C, Baker D, Browne V, Chanen AM, et al. Identifying attributes of care that may improve cost-effectiveness in the youth mental health service system. Med J Aust. Nov 20, 2017;207(10):S27-S37. [CrossRef] [Medline]
    25. Hickie I, Carpenter JS, Iorfino F, Scott E, Cross S, Hermens DF. The Utility of Clinical Staging in Youth Mental Health Settings: Neurobiological and Longitudinal Data from Sydney-Based Studies of Transdiagnostic Cohorts. In: McGorry P, Hickie I, editors. Clinical Staging in Psychiatry: Making Diagnosis Work for Research and Treatment. Cambridge, MA. Cambridge University Press; 2019;81-102.
    26. Iorfino F, Cross SP, Davenport T, Carpenter JS, Scott E, Shiran S, et al. A Digital Platform Designed for Youth Mental Health Services to Deliver Personalized and Measurement-Based Care. Front Psychiatry. 2019;10:595. [FREE Full text] [CrossRef] [Medline]
    27. Adrian M. Adaptive Interventions Are Required for Efficient and Personalized Adolescent Depression Treatment. J Am Acad Child Adolesc Psychiatry. Jan 2019;58(1):16-17. [CrossRef] [Medline]
    28. Centers for Disease Control and Prevention (CDC). Web-based Injury Statistics Query and Reporting Syste4m (WISQARS). CDC. 2020. URL: https://www.cdc.gov/injury/wisqars/index.html [accessed 2022-12-01]
    29. Jacobson CM, Gould M. The epidemiology and phenomenology of non-suicidal self-injurious behavior among adolescents: a critical review of the literature. Arch Suicide Res. 2007;11(2):129-147. [CrossRef] [Medline]
    30. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. Jun 2005;62(6):593-602. [CrossRef] [Medline]
    31. Klonsky ED. Non-suicidal self-injury in United States adults: prevalence, sociodemographics, topography and functions. Psychol. Med. Jan 05, 2011;41(9):1981-1986. [CrossRef]
    32. Substance Abuse and Mental Health Service Administration. Key substance use and mental health indicators in the United States: Results from the 2018 National Survey on Drug Use and Health (HHS Publication No. PEP19-5068, NSUDH Series H-54). Substance Abuse and Mental Health Service Administration. Rockvile, MD. Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; 2019. URL: https:/​/www.​samhsa.gov/​data/​sites/​default/​files/​cbhsq-reports/​NSDUHNationalFindingsReport2018/​NSDUHNationalFindingsReport2018.​pdf [accessed 2023-07-19]
    33. Braciszewski J. Digital Technology for Suicide Prevention. Advances in Psychiatry and Behavioral Health. Sep 2021;1(1):53-65. [FREE Full text] [CrossRef]
    34. Achtyes ED, Halstead S, Smart L, Moore T, Frank E, Kupfer DJ, et al. Validation of Computerized Adaptive Testing in an Outpatient Nonacademic Setting: The VOCATIONS Trial. Psychiatr Serv. Oct 2015;66(10):1091-1096. [FREE Full text] [CrossRef] [Medline]
    35. Rush AJ, Trivedi MH, Wisniewski SR, Stewart JW, Nierenberg AA, Thase ME, et al. Bupropion-SR, Sertraline, or Venlafaxine-XR after Failure of SSRIs for Depression. N Engl J Med. Mar 23, 2006;354(12):1231-1242. [CrossRef]
    36. World Health Organization (WHO). WHO menu of cost-effective interventions for mental health. WHO. Geneva, Switzerland. WHO; 2021. URL: https://apps.who.int/iris/handle/10665/343074 [accessed 2023-07-11]
    37. Newby J, Mewton L, Williams AD, Andrews G. Effectiveness of transdiagnostic Internet cognitive behavioural treatment for mixed anxiety and depression in primary care. J Affect Disord. Aug 2014;165:45-52. [CrossRef] [Medline]
    38. Mahoney A, Li I, Haskelberg H, Millard M, Newby JM. The uptake and effectiveness of online cognitive behaviour therapy for symptoms of anxiety and depression during COVID-19. J Affect Disord. Sep 01, 2021;292:197-203. [FREE Full text] [CrossRef] [Medline]
    39. Rosenberg B, Kodish T, Cohen ZD, Gong-Guy E, Craske MG. A Novel Peer-to-Peer Coaching Program to Support Digital Mental Health: Design and Implementation. JMIR Ment Health. Jan 26, 2022;9(1):e32430. [FREE Full text] [CrossRef] [Medline]
    40. Craske M. Honoring the Past, Envisioning the Future: ABCT's 50th Anniversary Presidential Address. Behav Ther. Mar 2018;49(2):151-164. [CrossRef] [Medline]
    41. Holmes EA, Ghaderi A, Harmer CJ, Ramchandani PG, Cuijpers P, Morrison AP, et al. The Lancet Psychiatry Commission on psychological treatments research in tomorrow's science. The Lancet Psychiatry. Mar 2018;5(3):237-286. [CrossRef]
    42. Craske MG, Herzallah MM, Nusslock R, Patel V. From neural circuits to communities: an integrative multidisciplinary roadmap for global mental health. Nat. Mental Health. Jan 17, 2023;1(1):12-24. [FREE Full text] [CrossRef]
    43. Craske MG. Cognitive Behavior Therapy, 2nd Edition. New York, NY. American Psychological Association; 2017.
    44. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. Dec 2011;168(12):1266-1277. [FREE Full text] [CrossRef] [Medline]
    45. Gibbons R, Weiss DJ, Pilkonis PA, Frank E, Moore T, Kim JB, et al. Development of the CAT-ANX: a computerized adaptive test for anxiety. Am J Psychiatry. Feb 2014;171(2):187-194. [FREE Full text] [CrossRef] [Medline]
    46. Gibbons R, Weiss DJ, Pilkonis PA, Frank E, Moore T, Kim JB, et al. Development of a computerized adaptive test for depression. Arch Gen Psychiatry. Nov 2012;69(11):1104-1112. [FREE Full text] [CrossRef] [Medline]
    47. Gibbons R, Kupfer D, Frank E, Moore T, Beiser DG, Boudreaux ED. Development of a Computerized Adaptive Test Suicide Scale—The CAT-SS. J. Clin. Psychiatry. Dec 27, 2017;78(9):1376-1382. [FREE Full text] [CrossRef]
    48. Graham A, Minc A, Staab E, Beiser DG, Gibbons RD, Laiteerapong N. Validation of the Computerized Adaptive Test for Mental Health in Primary Care. Ann Fam Med. Jan 2019;17(1):23-30. [FREE Full text] [CrossRef] [Medline]
    49. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the Trial: Systematic Review of Real-World Uptake and Engagement With Digital Self-Help Interventions for Depression, Low Mood, or Anxiety. J Med Internet Res. Jun 06, 2018;20(6):e199. [FREE Full text] [CrossRef] [Medline]
    50. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol. Med. Sep 24, 2008;39(5):705-712. [FREE Full text] [CrossRef]
    51. Kodish T, Lau AS, Gong-Guy E, Congdon E, Arnaudova I, Schmidt M, et al. Enhancing Racial/Ethnic Equity in College Student Mental Health Through Innovative Screening and Treatment. Adm Policy Ment Health. Mar 2022;49(2):267-282. [FREE Full text] [CrossRef] [Medline]
    52. Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. J Anxiety Disord. Apr 2018;55:70-78. [FREE Full text] [CrossRef] [Medline]
    53. Saleem M, Kühne L, De Santis KK, Christianson L, Brand T, Busse H. Understanding Engagement Strategies in Digital Interventions for Mental Health Promotion: Scoping Review. JMIR Ment Health. Dec 20, 2021;8(12):e30000. [FREE Full text] [CrossRef] [Medline]
    54. Batterham PJ, Calear AL. Preferences for Internet-Based Mental Health Interventions in an Adult Online Sample: Findings From an Online Community Survey. JMIR Ment Health. Jun 30, 2017;4(2):e26. [FREE Full text] [CrossRef] [Medline]

    ‎
    CAT-ANX: Computerized Adaptive Test-Anxiety Inventory
    CAT-DI: Computerized Adaptive Test-Depression Inventory
    CAT-MH: Computer Adaptive Test for Mental Health
    CAT-SS: Computerized Adaptive Test-Suicide Scale
    CSSRS: Columbia Suicide Severity Rating Scale
    STAND: Screening and Treatment for Anxiety and Depression
    UCLA: University of California, Los Angeles

    Edited by J Torous; submitted 01.03.23; peer-reviewed by A Ghosh, S Locke; comments to author 16.05.23; revised version received 25.05.23; accepted 26.05.23; published 24.07.23.

    Copyright

    ©Kate Wolitzky-Taylor, Richard LeBeau, Inna Arnaudova, Nora Barnes-Horowitz, Elizabeth Gong-Guy, Scott Fears, Eliza Congdon, Nelson Freimer, Michelle Craske. Originally published in JMIR Mental Health (https://mental.jmir.org), 24.07.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.