Telehealth Services for Substance Use Disorders During the COVID-19 Pandemic: Longitudinal Assessment of Intensive Outpatient Programming and Data Collection Practices

Introduction

Substance use disorder (SUD) is a chronic relapsing disease associated with numerous psychosocial harms and health sequalae. Addiction was a leading global cause of disability and death prior to the COVID-19 pandemic [1], which has since disproportionately impacted individuals suffering from SUD. Recent studies indicate that individuals with SUD may be more susceptible to severe disease and have higher rates of mortality and postvaccination breakthrough infections [2-4]. Isolation, uncertainty, and financial instability have also compounded substance use and the challenges of early recovery [5-7]. These vulnerabilities have reinforced the critical need for ongoing and safe access to treatment throughout the course of the pandemic through virtual services.

Prior to the COVID-19 pandemic, virtual services showed promise but were slow to develop. Early applications have shown promise as a means of preventing premature dropout from SUD treatment [8]. The pandemic rapidly accelerated the implementation of telehealth services for mental health and substance use treatment [9], and both providers and participants have viewed these types of services favorably [10,11]. Unfortunately, little is known about the actual efficacy or effectiveness of individual SUD treatment in telehealth treatment settings [12-14], and even less about traditional group treatment formats [15].

The onset of the pandemic became a catalyst for addiction treatment programs to quickly pivot to provision of services through telehealth formats despite limited data to guide their delivery. Change was facilitated by paradigm shifts in federal, state, and local policies and in organizational and provider practices [16,17]. Although these policies allowed for the continuity of care through available technology, stakeholders within the addiction field are now facing decisions on which elements of policies and programs to sustain, adapt, or discontinue. Continuation of these policies is dependent upon rigorous assessment of clinical data to define the new standard of SUD treatment through virtual platforms. Unfortunately, the pandemic had a devastating impact on research, with ongoing disruptions to recruitment and study progress, as well as a dramatic reduction in survey participation and response rates across many fields of study [18-20].

There is still a significant need for research related to the application and assessment of telehealth for SUD. Unfortunately, best practices for patient outcome collection for SUD treatment in mixed settings have yet to be established. In this paper, we describe how the COVID-19 pandemic presented a novel opportunity to bridge the gap and assess the effectiveness of a virtual intensive outpatient programming (IOP) for substance use treatment through the examination of short-term postprogram outcomes of adults who received IOP services through different delivery formats at the largest SUD treatment provider in the United States.

Methods

Study Design and Population

The Hazelden Betty Ford Foundation (HBFF) is the largest national provider of addiction services in the United States. The HBFF utilizes evidence-based practices through a multidisciplinary and integrated approach to addiction treatment across varying levels of care. In 2019, the HBFF piloted a single virtual intensive outpatient group with planned expansion of virtual services in 2020, as informed by routine outcome monitoring (ROM) data. The HBFF has an established infrastructure and process for collection of ROM data that has been used and refined since 1974. ROM data provide an understanding of real-world conditions, offering applied generalizability to community-based treatment settings where the majority of care is provided [21,22]. These data can be designed as a feedback loop, intended to quickly translate findings into treatment implementation [23,24]. In a rapidly evolving global pandemic, this type of real-world feedback is invaluable to informing the refinement of virtual treatment, despite potentially lower response rates than a formal randomized controlled trial [22].

This study presents 3-month findings (n=1060, 29.1%) from a 12-month longitudinal assessment of patients, 18 years and older, who were discharged from IOP between January 2020 and March 2021 (N=3642). Patients were separated by 3 distinct treatment delivery settings in response to pandemic changes: (1) in-person care only (n=957, 26.3%); (2) hybrid, in-person, and virtual care (n=541, 14.9%); and (3) virtual care only (n=2144, 58.9%).

Ethics Approval

The study was reviewed and approved by Emory University’s Institutional Review Board (STUDY00001822) and was determined to have met the human research exemption since all data were collected within the context of the HBFF’s standard ROM practices.

Data Collection Procedures

Trained research data collection specialists (RDCS) utilized a systematic and manualized process for data collection. Web-based surveys were automatically assigned to IOP patients at 1, 3, 6, 9, and 12 months postdischarge, with survey completion windows open for 30 days. Survey links were emailed to patients, with reminder prompts every 3 days for up to 2 weeks. Patients were contacted by an RDCS every 4-7 days to complete the survey over the phone or to encourage patients to complete it online if not initially completed. Patients were still prompted to provide responses if admitted to a different level of care.

Impact of the Pandemic on Data Collection

In response to a notable decline in response rates at the beginning of the pandemic, RDCS spent over 1200 hours attempting to contact patients to complete the 3-month survey throughout the course of the virtual IOP study. The timing of data collection was also impacted. Virtual IOP began in March 2020, while data collection began in May 2020, largely due to reallocation of resources to facilitate the transition of direct care to virtual services. To retrospectively capture an in-person IOP comparison group, all patients discharged from an in-person IOP on or after January 1, 2020, were opted into receiving IOP outcome surveys. However, due to the 30-day survey windows, the majority of in-person and hybrid patients were excluded from completing the baseline survey at admission and the 1-month postdischarge survey, impacting response rates for those time points (Figure 1).

‎View this figure

Measures

Demographic information was collected from patient electronic medical records. The majority of the full sample of patients were White (n=3296, 91.3%) and male (n=2258, 62.0%), with a mean age of 39.1 (SD 13.5) years.

Outcome Measures

A variety of self-reported outcome measures were used to assess for health and well-being at 3-month follow-up. Continuous abstinence from drugs and alcohol (abstinent since discharge vs relapsed) during the follow-up period was assessed using a question from a modified Form 90 Alcohol Questionnaire (Form 90-AQ) [25], adapted to ask about the use of any substances and to include the specific time period for clarity: “Have you used any drugs or alcohol since your last survey on (last survey date)?” Compliance to prescribed anticraving medications (eg, buprenorphine, naltrexone, or acamprosate) was assessed with a single-item, binary question: “Have you taken your anticraving medication, as prescribed?” Peer support group engagement was measured by 1 item from the Alcoholics Anonymous (AA) Involvement Scale [26] and adapted to include a reference to other peer group support beyond AA: “About how often have you been attending 12-step/peer support/mutual aid group meetings since you were discharged?”. Participants rated their frequency of attendance on a 6-point scale: daily, 4 or more times per week, 1-3 times per week, 2-4 times per month, once a month or less, and never.

Quality of life was measured using the 4-item self-reported Centers for Disease Control Healthy Days Survey [27,28]. An additional question assessing overall quality of life was also added: “How would you rate your overall quality of life?” Patients were asked to rate their overall quality of life and quality of general health using a 5-point Likert rating, from 1 (poor) to 5 (excellent), and indicated the number of days out of the previous 30 that they experienced poor mental or physical health. Higher numbers of unhealthy days indicated a lower quality of life. Psychological well-being was assessed by the summed composite of the 8-item Flourishing Scale [29]. For each item, patients rated their level of agreement on a 7-point Likert scale, from 1 (strongly disagree) to 7 (strongly agree). The scale yielded high internal consistency (α=.94). Financial well-being was measured using the 5-item Consumer Financial Protection Bureau (CFPB) Financial Well-Being Scale, with higher scores illustrating greater perceived financial well-being [30]. The CFPB Financial Well-Being Scale showed good internal consistency (α=.85). Patients’ confidence in their ability to stay sober was measured using an adapted form of the Brief Situational Confidence Questionnaire (BSCQ) to create a sobriety self-efficacy scale [31]. The 7-point Likert response categories were reworded to maintain consistency across the different scales, and the original BSCQ question 5, “I could probably go back to social drinking or other moderate drug use if I wanted to,” was removed as initial interitem correlations and α values indicated that this question did not adequately add to the measure of sobriety self-efficacy. After removal of question 5, the adapted scale of sobriety self-efficacy showed high internal consistency (α=.89).

Statistical Analyses

Analyses were performed using IBM SPSS Statistics version 28 [32]. Data were examined using chi-square tests of independence and one-way ANOVAs to ascertain the relationship between IOP delivery setting and patient outcomes. Direct comparisons between the different settings of IOP were not indicated because the virtual IOP study was not prospectively designed and was instead reactively implemented as a result of the pandemic. This pandemic reality resulted in differences in the timing of care, akin to a cohort effect within a single year, where those in-person care reached the 3-month survey earlier in the pandemic (eg, May-July 2020), while those in virtual-only care completed the 3-month survey over a much longer period (eg, June 2020-June 2021).

Results

Sample Characteristics

Sample characteristics are reported by IOP modality in Table 1. Differences between IOP settings emerged in biological sex, age, and length-of-stay distributions. In comparison to in-person and virtual groups, the hybrid group members were more likely to be male (133/192 [69.3%] in the hybrid group vs 161/256 [62.9%] in the in-person group and 363/612 [59.3%] in the virtual group). The virtual group had a greater number of individuals aged from 45 to 64 years (293/612 [47.9%] in the virtual group vs 92/256 [35.9%] in the in-person group and 67/192 [34.9%] in the hybrid group). Individuals in the hybrid group had significantly longer lengths of stay (mean 74.67 [SD 41.78] days) than those who participated in in-person IOP (mean 53.88 [SD 34.79] days) or virtual IOP (mean 54.67 [SD 33.31] days). No significant differences were detected between formats by race, ethnicity, marital status, employment status, education level, whether a patient was discharged against staff/medical advice, use of insurance for services, or type or number of active SUD diagnoses.

Demographic and clinical characteristics of noncompleters of the 3-month survey were compared with those who completed the survey (Table 2). There were no significant differences in completer status in regard to biological sex, race, identification as Latinx, or the highest level of education attained. However, a few differences emerged in age, marital/relational status, and employment type. Completers of the 3-month survey were slightly older (mean age 42.26 [SD 12.93] years) than noncompleters (mean age 38.26 [SD 13.17] years). In addition, completers were more likely to be employed full-time (672/1060 [63.4%] vs 1400/2503 [55.9%]) and to be married/cohabiting (550/1060 [51.9%] vs 985/2503 [39.8%]).

A greater number of differences arose in regard to clinical characteristics. Those who completed the 3-month survey were less likely to have multiple active SUDs (336/1060 [31.7%] vs 1014/2503 [40.5%]) and to get discharged against staff/medical advice (107/1060 [10.1%] vs 531/2503 [21.2%]). Completers also showed a longer length of stay in IOP care (mean 58.10 [SD 36.16] days) than noncompleters (mean 49.57 [SD 38.73] days). There was no difference between those who stepped down into IOP from a higher level of programming within the HBFF (eg, residential vs day treatment).

Multivariate Comparisons

A few differences emerged between IOP settings across multiple domains of functioning (Table 3). There was no significant difference by setting in self-reported continuous abstinence, with over two-thirds of the sample (680/960, 70.8%) reporting no drug or alcohol use since discharge. Approximately one-third (332/1060, 31.3%) overall reported still being prescribed an anticraving medication. Of those, no difference in medication compliance emerged between in-person, hybrid, or virtual IOP respondents. Individuals across all settings reported attending peer support meetings, on average, of 1 or 2 times per week. Further, there were no differences across settings in the overall perceived quality of life or in the total number of poor physical and mental health days at 3-month follow-up. Finally, there were no differences detected between IOP setting and the individuals’ confidence in their ability to stay sober, financial well-being, or psychological well-being.

The only significant difference by IOP setting that emerged was in the overall quality of one’s general health, where those in the hybrid group (mean 4.08 [SD 0.75]) were more likely than those in the virtual group (mean 3.89 [SD 0.83]) to report a higher level of general health.

Discussion

Principal Findings

This study is the first of its kind to assess telehealth for SUD in the IOP setting in a large cohort of patients (N=1000+). No meaningful differences in outcome measures were identified between delivery settings at 3-month follow-up, with individuals reporting similar levels of continuous abstinence, quality of life, and social/emotional well-being. Our findings in regard to continuous abstinence were consistent with previous studies following patients at 3-6 months postdischarge from IOP (eg, 65/103 [63.1%]) [33]. These results are promising and suggest a potential continuing role for virtual IOP as an effective component in addiction treatment settings. Advocacy is needed to maintain these services as a standard offering within the SUD treatment continuum.

Historically, peer-based connections and the therapeutic milieu have been integral parts of addiction treatment. Concern has been expressed by the addiction treatment community regarding the shift to virtual services and its impact on group engagement and patient-centered outcomes [34]. These preliminary results demonstrate the feasibility of offering services virtually. Further research is necessary to obtain feedback on patient experience and measures of group cohesion, such as secure emotional expression, as they apply to virtual addiction treatment [35].

Our findings aid in establishing a platform for future evaluation of data collection processes that inform the effective development of standardized protocols for routine outcomes data practices, including frequency of contact, method of outreach, and training of staff. Standardized protocols must consider the context for accurate interpretation of collected data. For example, differences in response rates emerged in this study based on the timing of data collection in relation to the global pandemic and due to unanticipated staff burden and should be interpreted in this context. At 3-month follow-up, response rates were lowest for those in in-person IOP (256/957, 26.8%). These rates are likely attributable to these being completed by patients between May and August 2020, timing that coincided with major city- and statewide lockdowns and great uncertainty about the unknowns presented by the pandemic. Furthermore, because of the large opt-in of the in-person cohort at the beginning of data collection (May 2020), many participants at 3-month follow-up were given access to the survey well into the 30-day response window, reducing the likelihood that they would have adequate time to complete it prior to survey close. In contrast, response rates for those in virtual IOP (612/1532, 28.5%) were most impacted by a higher-than-projected admission rate of individuals into IOP throughout the study period, resulting in a larger sample size than originally anticipated. As a result, there were proportionally lower staffing levels than would be typically allocated for the final sample size, which may explain the lower response rates for the virtual group.

Strengths and Limitations

A key strength of this study is the breadth of data collected from such a large number of patients receiving SUD treatment during a period of extensive change. Although these results are mainly descriptive, these analyses are necessary to carefully evaluate the impacts of a global shift in treatment approach. A number of limitations should be considered when interpreting results. Similar to existing research used with ROM data [36-39], response rates at 3-month follow-up were low. Nonresponse bias is a risk inherent to survey analysis. However, even in studies with high response rates, research has shown that nonresponse rates are not always directly predictive of nonresponse bias [37,40-42]. Research supports that highly resource-intensive recruitment deployed to capture late responders does not necessarily alter the outcomes found at lower, less resource-intense response rates, and this type of recruitment can be both cost- and time-prohibitive [40,42]. Despite COVID-induced high nonresponse rates, we expect these data to accurately reflect the effectiveness of IOP services during the COVID-19 pandemic. Substance use at follow-up was based on retrospective self-report of use. Utilizing additional methods for verification, such as urine drug analyses, would strengthen the validity of these reports in future studies. As discussed earlier, due to the sudden onset of the pandemic and subsequent data collection for this unique cohort, baseline and 1-month follow-up data were unable to be collected from a substantial portion of the full sample. The missing data influenced our ability to directly compare the effectiveness of IOP across delivery type, given the inherent confounding effect of the timing of patient care in relation to the unfolding public health crisis. We recommend future prospective studies be designed to compare in-person and virtual treatment directly, with inclusion of a formal evaluation of the ideal conditions for patient success (ie, dosage, treatment duration, frequency).

Finally, although our sample was representative of HBFF program participants, it differs from the general population in a few specific ways that are important to acknowledge when considering the generalizability of the findings. First, patients in the sample were primarily White and non-Latinx. As a result, there may have been too few non-White/Latinx participants to detect differences. However, this sample directly compares to findings from the 2020 National Survey on Drug Use and Health (NSDUH) [43] in terms of full- and part-time employment (United States: 67.3% vs sample: 67.0%) and bachelor’s degree attainment (United States: 27.7% vs sample: 29%) among adults 18+ years old with a SUD in the past year. This demonstrates that addiction affects individuals across varying educational and employment statuses. This is in contrast to stereotypes of individuals struggling with addiction, which may characterize this as a disease of the uneducated and unemployed. Even though a higher percentage of patients employed full-time completed the 3-month survey (672/1060 [63.4%] vs 1400/2503 [55.9%]), post hoc analyses revealed no difference in outcomes based on employment status; therefore, we believe this had minimal impact on our findings. Future research should endeavor to improve the representation of racial and ethnic minorities in order to improve generalizability across a wider cross section of demographic variables.

Conclusions

Results from this study suggest that virtual outpatient care for the treatment of SUDs is a feasible alternative to in-person care, leading to similar rates of self-reported continuous abstinence, health, and well-being in patients at 3-month follow-up. This study should serve as a baseline for the assessment and refinement of the role of virtual services in the field of addiction treatment in order to better understand under what circumstances telehealth can function as an effective alternative to the established in-person standard of care.