Kaiser Permanente Northern California (KPNC) is a large, diverse, integrated health care delivery system with approximately 4.6 million members. The membership is largely representative of the region, with less representation at the extreme ends of the income distribution [19]. Mental health services are primarily provided internally, both individually and in group format. Services are also available by referral to external community providers as needed.

KPNC offers multiple DTx tools to its members at no cost. During the study time frame, the portfolio included Calm, myStrength, Headspace (Headspace Health, Inc), SilverCloud (Amwell), Thrive (Waypoint), and Whil (RethinkFirst) [20]. Providers can recommend DTx tools to patients based on their clinical judgment using smart text elements in the EHR, which are inserted into electronic secure messages to the patients with an associated link to the DTx. This electronic secure message contact is documented in the after-visit summary if providers use these smart text elements.

This study received approval from the KPNC Institutional Review Board (1899252) and was determined exempt, with a waiver of consent for the EHR-based analyses and a waiver of written consent for the anonymized provider survey, which did not gather protected health information. No compensation was provided to the providers per health system policy.

Of the 335,250 adults eligible during the study time period, 16% (53,546/335,250) were recommended a DTx by a provider (Table 1). Patients recommended versus not recommended to a DTx were more likely to be younger (age 18‐44 years: 37,744/53,546, 70.5% vs 164,456/281,704, 58.4%; P<.001), Asian or Hispanic (20,360/53,546, 38% vs 88,827/281,704, 31.6%; P<.001), and female (38,338/53,546, 71.6% vs 193,063/281,704, 68.5%; P<.001). Additionally, there was a higher proportion of patients in the lowest (“0“) category of the Charlson Comorbidity Index (40,261/53,546, 75.2% vs 188,414,/281,704, 66.9%; P<.001) and a higher proportion of commercially insured patients (44,960/53,363, 84.3% vs 205,425/280,546, 73.2%; P<.001) among those who received recommendations versus those without a recommendation. Most DTx recommendations originated from therapy providers (48,916/53,546, 91.4%).

Given the significant demographic differences between patients recommended a DTx versus those not recommended, we performed a matched analysis to compare mental health diagnoses and symptoms between these 2 groups. Patients recommended to DTx versus not recommended had higher proportions of anxiety (22,247/53,545, 41.5% vs 19,039/53,545, 35.6%) and adjustment disorders (11,024/53,545, 20.6% vs 7592/53,545, 14.2%) and lower proportions of depression (13,151/53,545, 24.6% vs 16,517/53,545, 30.8%), bipolar (588/53,545, 1.1% vs 2685/53,545, 5%), or psychotic diagnoses (173/53,545, 0.3% vs 1239/53,545, 2.3%; all P<.001; Table 2). Additionally, more patients recommended to DTx had PHQ-9 depressive symptom scores in the mild (5-9: 10,643/37,282, 28.5% vs 6926/25,749, 26.9%), moderate (10-14: 10,597/37,282, 28.4% vs 7104/25,749, 27.6%), and moderate severe (15-19: 7513/37,282, 20.2% vs 5115/25,749, 19.9%) ranges compared to patients not recommended to DTx (P<.001 for all). The distribution of GAD-7 anxiety scores was similar between patients recommended and not recommended to DTx, albeit with the distribution of recommended patients having a slightly higher proportion of mild (5-9: 4852/23,251, 20.9% vs 2579/13,080, 19.7%) and lower proportion of severe (≥15: 9283/23,251, 39.9% vs 5391/13,080, 41.2%) GAD-7 scores (P=.003 for all).

Since approximately two-thirds of patients recommended to DTx were female, we conducted ad hoc analyses examining symptom burden stratified by sex among those who received a recommendation to DTx. Female patients recommended to DTx had higher proportions of PHQ-9 depressive symptom scores in the moderate (7575/26,416, 28.7% vs 3022/10,867, 27.8%), moderate severe (5414/26,416, 20.5% vs 2099/10,867, 19.3%), and severe (3246/26,416, 12.3% vs 1220/10,867, 11.2%) range compared to male patients; and male patients had higher proportions of PHQ-9 scores in the normal (2804/26,416, 10.6% vs 1260/10,867, 11.6%) and mild (7377/26,416, 27.9% vs 3266/10,867, 30.1%) ranges (P=.003; Table 3). Female patients recommended to DTx had higher proportions of GAD-7 scores in the severe (6839/16,526, 41.4% vs 2444/6725, 36.3%) range compared to male patients; and male patients had more representation in the normal (516/16,526, 3.1% vs 341/6725, 5.1%) and mild (3296/16,526, 19.9% vs 1556/6725, 23.1%) ranges for GAD-7 (P<.001).

A total of 211 individuals responded to the provider survey on DTx prescribing patterns for a participation rate of 8.4% (211/2505). Of those, 19.6% (40/204) identified as prescribers, 68.6% (140/204) as nonprescribers, and the remainder as having an administrative or managerial role in the clinic. Of respondents, 91.9% (192/209) were between the ages of 30 and 59 years, 54.1% (113/209) identified as White, 17.2% (36/209) as Asian, 12% (25/209) as Hispanic or Latino, and 6.2% (13/209) as Black. Nearly all respondents reported working at least half-time, and two-thirds of respondents reported working a full-time schedule.

Overall, a higher proportion of nonprescribers (127/164, 77.4%) indicated they recommend DTx to patients compared to prescribers (22/39, 56.4%; P=.008). Of providers who recommend DTx, 83.6% (122/146) do so via an electronic secure message to patients, and only 12.3% (18/146) do so during a clinic appointment. Calm was the most commonly recommended DTx, followed by myStrength, Headspace, SilverCloud, Thrive, and Whil. Only 29.4% (45/153) of providers indicated using the suggested internal EHR–based smart text to recommend DTx. Prescribers were more likely than nonprescribers to report that they did not know how to make patient recommendations using EHR-based protocols (P=.01). Of providers recommending DTx to patients, 64.1% (98/153) report that they follow up with patients to inquire whether the resource was helpful. Only 38.4% (58/151) of providers recommend specific modules of DTx resources, and of those 58.6% (34/58) report following up with patients on these specific recommendations. On a scale of 0‐10 regarding the perceived effectiveness of DTx resources to impact symptoms (0=none and 10=extreme), both prescribers and nonprescribers rated cognitive behavioral therapy and mindfulness content between 6 and 7 or moderate to highly effective.

These are some of the first pragmatic results regarding provider recommendation patterns, characteristics, and attitudes toward mental health DTx. In this large, integrated health system with mental health DTx tools included as a benefit, a very modest percentage of patients with mental health disorders received recommendations to DTx tools. Those who did were more likely to be young, female, Asian or Hispanic, with less medical comorbidity, and to have commercial insurance compared to patients not recommended to DTx. Recommendations for DTx were more often made for anxiety or adjustment diagnoses and for patients with less severe mental health symptoms, although female patients recommended to DTx tools had a higher proportion of anxiety symptoms compared to male patients. A provider survey showed that DTx are recommended more frequently by nonprescribers and via electronic secure messages rather than during clinical appointments. Of providers recommending DTx, a modest majority follow up with the patient regarding DTx effectiveness, and few knew how to follow the recommended EHR protocols to do so.

Previous studies reporting demographic characteristics of patients using DTx have been from pilot trials or RCTs [7-9]. Using real-world data collected in the course of clinical care, we found that patient characteristics were similar to those of DTx trials. Overall, patients who received DTx recommendations were younger with less severe mental health symptoms, suggesting that providers are likely tailoring their recommendations to those they feel are most suitable. This may also reflect the initial emphasis of the health system DTx implementation focusing on patients with mild to moderate symptoms of depression or anxiety [20]. These results highlight a gap in the evidence base regarding the effectiveness of DTx for patients with mild to severe mental health symptoms or multiple comorbidities.

Asian or Hispanic patients were more likely to receive DTx recommendations. There are many potential drivers of this finding, including provider beliefs about patient DTx uptake, patient ability to interface with DTx, if providers felt the symptom severity reported by these patients was more appropriate for DTx, or other unrecognized factors. This finding merits further investigation. Of note, the significant association for insurance type with a DTx recommendation suggests slightly more patients with commercial insurance receiving recommendations. This may reflect provider-perceived barriers to care (eg, differences in disease severity or mobile phone access) for patients with other types of insurance coverage.

The finding that a higher proportion of female patients with severe mental health symptoms were referred to DTx compared to male patients could reflect a bias to implement more tools to treat symptoms in female patients, a reporting bias of symptoms by male or female patients, refusal of the DTx recommendation by male patients, or other sex-associated biases in care [26]. These findings warrant further research.

Provider survey results indicate that a higher proportion of nonprescribers recommend DTx and reported familiarity with EHR-based protocols to recommend DTx compared to prescribers. These results could reflect uneven training by provider type during the initial rollout or a lack of awareness or interest in the DTx tools by prescribers. Nonprescribers may appreciate having another treatment option to recommend beyond their typical strategies, given the therapeutic role DTx can fill and the silos of mental health care (ie, “therapy” vs “medication management” tasks). Interestingly, a minority of providers reported using the EHR-based protocols to recommend DTx to patients, with very few recommending specific modules or follow-up on module recommendations. Providers may view DTx tools as a passive, adjunctive resource rather than part of an active mental health treatment plan. Further, only a modest majority of providers reported asking patients if they found the DTx effective. A similar follow-up rate for a recommended or prescribed medical device, medication, or therapeutic program such as intensive outpatient or partial hospitalization would likely be regarded as very low. It is unclear why providers perceive DTx offerings in this way; it is possible that providers do not see DTx as having robust effectiveness data, which may impact their rate of follow-up. Similarly, providers may not view DTx resources as part of clinical care or within their scope of practice to monitor. This may reflect the early stage of growth and implementation of DTx in mental health and may evolve over time. If DTx are to be integrated into clinical mental health care, it may be more effective as a formalized part of the treatment plan and followed in the same way as other treatment aspects.

This study was conducted in an integrated health care system, which limits generalizability to providers and patients in different systems of care. The survey participation rate was low, given constraints related to direct dissemination and reapproaching for response, though consistent with reported response rates in similar investigations [15]. While our integrated health system made multiple DTx available at no cost to patients during the study time period, such resources may not be as easily available to patients in other health care systems. Recommendations that were in person only or sent in messages without using the EHR protocols and smart text were not captured, nor were recommendations related to later visits, although our intent was to identify options presented earlier in the treatment course. The large sample size of EHR-based recommendation data makes it likely to detect associations that may have statistical significance but may not have meaningful clinical implications. The data we explored are observational and exploratory; future work warrants further examination into these initial findings along with an examination of patient-level outcomes associated with DTx.

The use of DTx is growing at a rapid pace, and health systems and patients see these as potentially valuable resources. However, the extent of provider recommendations to DTx is modest, as is provider knowledge about DTx and their follow-up with patients about DTx use, which suggests a largely passive DTx uptake by providers. Given the likely continued interest by patients, health care systems, and industry, future research on how to effectively implement these tools is critical.