Published on in Vol 12 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/73617, first published .
Digital Contingency Management for Substance Use Disorder Treatment: 12-Month Quasi-Experimental Design

Digital Contingency Management for Substance Use Disorder Treatment: 12-Month Quasi-Experimental Design

Digital Contingency Management for Substance Use Disorder Treatment: 12-Month Quasi-Experimental Design

Authors of this article:

Xiaoni Zhang1 Author Orcid Image ;   Valerie Hardcastle2 Author Orcid Image

1Department of Management, Information Systems and Quantitative Methods, The University of Alabama at Birmingham, CSB 371 710 13th Street South, Birmingham, AL, United States

2Institute for Health Innovation, Northern Kentucky University, Highland Heights, KY, USA

Corresponding Author:

Xiaoni Zhang, PhD


Background: Although contingency management has shown some efficacy in substance use disorder treatment, digital contingency management (DCM) needs more evidence supporting its value in treating substance misuse.

Objective: This study aimed to evaluate the effectiveness of DCM in treating substance use disorder by examining 2 key outcome variables—abstinence and appointment attendance.

Methods: A 12-month quasi-experimental design was conducted by enrolling patients into 2 groups using an alternating assignment process: one group receiving treatment-as-usual plus DCM and the other receiving treatment as usual with no contingency management. Propensity score matching was conducted to match groups on covariates. After matching, t tests were conducted to examine the difference between groups on urine abstinence and appointment attendance rates.

Results: Two cohorts of propensity-matched patients (66 interventions and 59 controls) were analyzed. Abstinence was significantly higher in the DCM group (mean 0.92, 95% CI 0.88‐0.96) than in the treatment-as-usual group (mean 0.85, 95% CI 0.79‐0.90; P<.01). Appointment attendance also demonstrated significant differences between the groups, with the DCM group achieving a mean rate of 0.69 (95% CI 0.65‐0.74) compared with 0.50 (95% CI 0.45‐0.55) in the treatment-as-usual group (P<.001). This notable increase highlights the role of DCM in fostering engagement with care, an essential factor for successful treatment outcomes.

Conclusions: The results suggest that DCM can be an effective treatment modality for substance use disorder.

JMIR Ment Health 2025;12:e73617

doi:10.2196/73617

Keywords



Background

Substance use disorders (SUDs) are a worldwide public health concern that impacts millions of people and weighs heavily on health care systems, families, and communities [1,2]. Over the past decade, substance use problems have been linked to a significant rise in both sickness and death rates globally [3]. Even with the presence of proven treatments, treatment-related attendance and follow-up remain significant barriers to recovery, and the rate of dropout ranges between 20% and 70% for different treatment programs [4,5]. Treatment adherence (ie, appointment attendance) and continued substance abstinence are 2 crucial elements that impact the outcome of treatment [6,7]. However, traditional treatment protocols struggle with multiple problems, such as low appointment attendance as well as long-term abstinence and relapse rates [8].

Contingency management (CM) is one of the most research-validated behavioral treatments to increase abstinence and treatment adherence in SUD treatment [9]. The foundation of CM is operant conditioning, which holds that actions leading to favorable outcomes are more likely to be repeated, whereas actions that result in unfavorable outcomes are less likely to be repeated. CM can be used to address several health behaviors, including medication adherence, substance use, weight loss, and quitting smoking. Based on behavioral economics and operant conditioning, this methodology is a systematic offering of material reward or incentive, depending on verified target outcomes such as passing drug tests or attending appointments [10]. The efficacy of CM has been consistently shown in meta-analyses for a range of SUDs, and in some studies, CM can boost medication adherence, therapy attendance, and abstinence rates [8]. Despite this solid evidence base, traditional CM implementation has limitations, including excessive administrative overheads, frequent in-person testing of target behavior, cumbersome reward systems, and policy obstacles, such as US federal antikickback and beneficiary inducement monetary and criminal sanctions [11].

Previous Work

Digital contingency management (DCM) combines the traditional CM paradigm with digital health tools, which is the next generation of SUD treatment delivery. DCM involves using remote, technology-based techniques (such as the internet and smartphones) to track drug status using biochemical sensors (such as breath alcohol and carbon monoxide monitors) and to provide desired outcomes, such as monetary rewards depending on drug abstinence [12,13]. Such online enhancements are not only efficient at providing CM but also offer unparalleled data collection and program optimization.

Digital technologies offer multiple benefits for CM over other forms of implementation. First among these is greater accessibility as digital services transcend geographic boundaries and transportation expenses, especially for patients living in congested urban, rural, or underserved communities. Digital platforms remove much of the administrative cost as automated solutions cut staff time compared with manual CM systems while keeping similar verification accuracy rates. In addition, DCM systems’ real-time data collection capabilities allow dynamic program optimization: machine learning algorithms can recognize patterns in patient usage and adjust reward schedules to optimize treatment. Data show that personalized digital incentives better drive program retention than traditional fixed-schedule solutions [14]. DCM implementations may be affordable, with programs averaging per-patient costs less than traditional CM while achieving equal or better outcomes.

Even with the promising potential of digital CM, there is still a lack of comprehensive understanding of how it compares with traditional CM methods. Existing literature provides substantial evidence supporting the efficacy of traditional CM in SUD treatments, particularly for smoking cessation and alcohol use disorders [15]. However, more research is needed to evaluate its effectiveness in treating drug use. While DCM shows significant potential as an innovative approach, there remains a critical need for robust empirical evidence to validate its efficacy and broader applicability [15].

Study Objectives

CM is underutilized in clinical settings [12]. If proven effective, DCM could present a better modality for widespread adoption. There is a pressing need to understand the interrelationships between appointment attendance, abstinence outcomes, and DCM interventions [10,16]. This understanding is crucial for developing more effective, integrated treatment approaches to better support individuals in their recovery journey [17].

In this study, we attempted to fill critical gaps in understanding the impact of DCM on SUD treatment by evaluating its effectiveness on 2 key outcomes: appointment attendance and abstinence. In particular, we compared a standardized DCM intervention with treatment as usual across multiple sites in a time-randomized controlled trial. These results will be part of the scientific evidence base for digital behavioral interventions in SUD treatment and should guide the development of more effective, accessible, and long-term treatment solutions. Here, we aim to provide some evidence-based recommendations for enhancing SUD treatment protocols. This research aimed to answer the following research questions: (1) How does DCM impact treatment adherence, as measured by appointment attendance? (2) How does DCM impact treatment outcomes, as measured by substance test abstinence rates?


Study Design

DynamiCare Health provided the DCM platform. We conducted a 12-month quasi-experimental study of DCM in a SUD population seeking treatment in partnership with providers located in the Greater Cincinnati Metropolitan Statistical Area. This quasi-experimental design used 2 groups to compare treatment outcomes between a non-DCM treatment-as-usual control group and a DCM experimental group. Participants were assigned to their group using an alternating assignment process based on the sequence of their enrollment in the program, rather than through random assignment. This approach was adopted due to pragmatic and ethical constraints in operating in a real-world clinical setting. Pure randomization was not feasible at intake because clinicians had to obtain informed consent before assigning a patient to the experimental group. To mitigate bias, intake coordinators were instructed to follow a strict alternating pattern, regardless of patient characteristics or interest, with no discretion in who was offered participation.

The study was carried out over a 4-year period from 2021 to 2024. Clients used the DynamiCare Health mobile app on smartphones to check into their treatment appointments (verified by GPS) and track any financial rewards they earned. These rewards were provided on a smart debit card that blocked access to charges in bars, liquor stores, and cash withdrawals.

Due to practical constraints of this real-world implementation study, a formal a priori power analysis was not conducted. The sample size was determined by the number of eligible patients who consented to participate in the study during the enrollment period. While our final matched sample (66 interventions and 59 controls) was relatively modest for a 12-month study, it proved sufficient to detect statistically significant differences in our primary outcomes. However, we acknowledge that the relatively small sample size may have limited our ability to detect smaller effect sizes, particularly for secondary outcomes or subgroup analyses.

In our study, missing data and attrition were minimal. Participants’ appointment attendance and urine test results were extracted from electronic health records, which systematically document all scheduled appointments and urine screens, including missed visits and missed tests. Therefore, nonattendance and missing urine tests were captured as part of the outcome measurement rather than treated as missing data.

Participants who disengaged from treatment or failed to provide urine samples were included in the denominator when calculating appointment attendance rates and urine test success rates, in accordance with an intention-to-treat principle. No participants were excluded due to missing data. As a result, there was no attrition bias requiring adjustment.

Procedure

Patients were assigned to groups using an alternating assignment process. During intake, an intake coordinator described the DCM project to the first patient. If this patient were interested in participating, they were consented and placed in the experimental group. The next patient the intake coordinator saw would automatically be placed in the treatment-as-usual control group without receiving any project description. This alternating process continued, offering participation in the project to every other patient. The intake coordinator then informed therapists of each patient’s group assignment, and therapists explained the DynamiCare system to those in the experimental group during their second appointment. Participants in the control group were not given any further instructions; they just received treatment as usual. The electronic health record system documented all treatment events in which the experimental and control groups participated and their medical histories.

The enrollment order assignment is not true randomization and could introduce bias. To address this, we used patient ID as a proxy for enrollment sequence and tested whether our outcomes showed systematic trends over the enrollment period. To address this, we conducted time series analyses to check for secular trends in our outcome measures during the enrollment period. We found no significant temporal patterns in either abstinence rates (P=.59) or appointment attendance (P=.47) across the study time frame for the experimental group. We conducted the same analyses for the control group with similarly nonsignificant results (P=.49 for urine abstinence rate and P=.28 for appointment rate). These findings provide strong evidence that time-dependent confounding was not a concern in our study. The lack of significant temporal trends in our outcomes suggests that the sequential assignment of participants did not introduce systematic bias beyond what was addressed by our propensity score matching approach.

Intervention DCM

There were only 2 differences between the experimental group and the control group. First, the experimental group received financial awards for completing prescribed treatment activities and for remaining sober. The control group engaged in the same treatment and testing regimen; however, it received no financial rewards for success. And second, the experimental group used a phone app as part of its treatment protocol, while the control group did not use a phone app as part of its treatment protocol. The experimental group’s app allowed the participants to check into their appointments online, which DynamiCare could then use to verify attendance by a location tracker. The app also tracked the cash value to the experimental group participants’ debit card as they received rewards for complying with their prescribed treatment activities and purchased allowable items or services. The DynamiCare company disbursed the reward funds as each of the experimental participants completed each prescribed activity successfully.

Patients in the experimental group used their own iOS or Android smartphones in this study. The intake coordinator assisted patients in downloading and logging into the app for the first time. At the time of primary data collection for this study, the HIPAA (Health Insurance Portability and Accountability Act)-compliant mobile DynamiCare app was used only to locate patients during scheduled therapeutic appointments and to reward patients completing specific, verified, and scheduled activities (the app now includes many interactive features).

In the experimental group, cash rewards were promptly provided for attending scheduled counseling appointments and receiving a negative outcome in urine screens. The system recorded negative “weeks since onboarded” as an automatic process from the account creation. Usually, patients downloaded the app shortly after receiving the invitation, but there were occasions when the users did not access the app for weeks (or at all). Four sites implemented the intervention between October 16, 2020, and September 13, 2023.

Participants could earn up to US $600 in rewards over a 12-month period, with incentives front-loaded during the first 3 months to foster engagement and a pattern of active effort. During this initial period, participants could earn US $100 per month, with the rewards tapering down in subsequent months. Specifically, months 4‐6 offered US $50 per month, months 7‐9 provided US $40 per month, and months 10‐12 offered US $25 per month.

Both experimental and control group participants underwent the same therapy and urine testing sequences, following the protocols of their individual behavioral health units. Both the experimental and control groups participated in clinical activities, as per the protocols of the individual treatment facilities (these did not differ by group, although they could differ by individual, depending on medical recommendations). But the recommendations were not determined by app usage; rather, in both groups, participants were treated as they ordinarily would have been, regardless of the DCM intervention.

Outcome Variables

The outcome variables were abstinence and appointment attendance. Abstinence was measured using objective methods, specifically, urine drug screening. Urine test success is measured as the number of negative urine test results or medically consistent results divided by the total number of urine tests logged by providers in their electronic medical records. A test was marked consistent, or abstinent, if it met medical expectations: negative for illicit substances, with positive results for prescribed substances allowed. Appointment attendance was calculated as the number of attended appointments divided by the total number of scheduled appointments.

All participants who enrolled in the program were included in the analysis regardless of their level of engagement. Missed appointments and missing urine screens were treated as part of the outcome definitions rather than missing data, using the following formula for appointment attendance rate: appointment attendance/(total number of missed appointments + total number of attended appointments). No participants were excluded due to missing information, ensuring a complete analysis under an intention-to-treat framework. We examined the potential time-dependent confound arising from our enrollment order assignment and did not find any significant temporal trends in our outcomes.

While we did not conduct a formal a priori power analysis due to the pragmatic nature of this implementation study, we assessed the adequacy of the sample size post hoc by evaluating the observed effect sizes and associated P values for our primary outcomes. We used G*Power (version 3.1.9.7) to determine the statistical power of the study. To detect a medium effect size (Cohen d=0.50) with a 1-tailed independent samples t test at a significance level of α=.05, and using our matched sample sizes (66 in the treatment group and 59 in the control group), we achieved a power of 87.1%. This means that there is an 87.1% probability of detecting a true medium-sized effect if one exists, which exceeds the conventional 80% threshold typically considered acceptable in most fields. Our matched sample of 125 participants (66 treatment and 59 control) thus provides sufficient power to detect medium effects in our primary outcomes. We have added this explanation to the “Study Design” section.

Propensity Score Matching

Propensity score matching (PSM) was used to remove confounding bias [18-22] as our experimental study was randomized based only on the time of program entry. To control for selection bias, we used nearest neighbor 1:1 matching. A binary logistic regression was used to estimate the propensity score. Variables’ association with treatment and outcome, as well as both treatment and outcome, was considered and reported. The variables used in the PSM model included gender, race, relationship status, and insurance. Selection of these covariates was based on previous research [23] reporting on the impact that various sociodemographic factors have on appointment attendance and substance use outcomes among individuals receiving treatment for SUDs. In particular, women are less likely to receive SUD treatment than men [24]. Marriage was associated with lower rates of SUD [25]. Race and ethnicity have been linked to treatment completion [26]. Health insurance coverage seems to be linked to better abstinence outcomes among US adults with mental health and substance use disorders [27].

This study aimed to estimate the effect of DCM on abstinence and appointment attendance. The challenge in estimating these effects is to plausibly separate them from confounders that might affect patients’ outcomes and to isolate those associated with digital CM. PSM was performed to design balanced samples. We selected an experimental group for each patient receiving DCM that had similar observed characteristics at intake. PSM was performed using a 1:1 nearest neighbor matching algorithm with replacement with distances determined by logistic regression using the Stata package PSMatch2. Standardized mean differences (SMDs) were used to evaluate the balance between treated patients and matched controls. Before and following matching, this measure is used to balance the variables [18]. SMDs less than 0.2 suggested a reasonable balance for group comparison following matching [21]. After matching, standardized differences between covariates were generally well balanced with values ranging from 0.01 to 0.08, within the recommended threshold of less than 0.2. However, gender maintained a moderate imbalance with an SMD of 0.23, slightly exceeding our predefined threshold. This residual imbalance was considered in subsequent analyses and interpretation.

To assess the quality of matching, SMDs were calculated and checked for all variables included in the propensity score model. Following Austin’s recommendation, we considered SMDs less than 0.2 to indicate adequate balance between treated and control groups. To evaluate the robustness of our findings to potential unmeasured confounding, we conducted postmatching sensitivity analysis using Rosenbaum bounds. This approach helped determine how significant an unmeasured confounder would need to be to negate the observed treatment effects. The sensitivity analysis tested a range of gamma values (Γ) from 1 to 3, with Γ=1 representing no hidden bias and higher values indicating increasingly strong unmeasured confounding.

Statistical Analysis

Group comparison and linear regression were used to assess the average treatment effect of DCM on abstinence and appointment adherence. Statistical analyses were performed in Stata (version 18; StataCorp) and 2-sided P value of <.05 indicated statistical significance. P values for continuous values were calculated using a weighted t test, and a chi-square test was used for categorical comparisons. We also conducted the robustness check by conducting a sensitivity analysis.

Ethical Considerations

Institutional review board (IRB) approval was obtained from the Northern Kentucky University IRB (IRB number 914) under expedited review according to 45 CFR 46.110: (4) collection of data through noninvasive procedures and (7) research on individual or group characteristics or behavior. Participant consent was obtained at the treatment facilities during patient onboarding; all possible participants could opt out of the study at any time. Consent language included: “Your participation is completely voluntary; you are free to change your mind at any time and quit the study. Whatever you decide will in no way result in loss of medical or health benefits or services to which you are otherwise entitled.”

There were no secondary analyses using data with primary consent. All patient information and data were deidentified using random numbers as identifiers. No one engaged in data analysis had access to any identifying information. All data associated with this study are kept on a secure HIPAA-compliant server. Because participant compensation comprises part of the study design, it is described in the “Intervention DCM” subsection.


Overview

Figures 1 and 2 show the consistent urine rate and appointment attendance rate for patients using DCM.

Figure 1. Urine consistency rate with digital contingency management.
Figure 2. Appointment attendance rate with digital contingency management.

Cohort Characteristics

Table 1 shows the demographic and insurance-related characteristics of the experimental group (n=74) and the control group (n=119) in the unmatched sample. The P values indicate statistical significance, and standardized differences provide a measure of the imbalance between groups. Females were more prevalent in the experimental group (44/74, 59.5%) than in the control group (30/74, 42.86%). Males were more prevalent in the control group (68/119, 57.14%) than in the experimental group (30/74, 40.5%). With P=.025 and SMD=−0.31, there is a significant difference between groups in terms of gender. Race and education exhibit comparable characteristics between the unmatched cohorts. Relationship status shows a moderate imbalance (P<.05; SMD=−0.58), while insurance type shows a significant imbalance (P<.02; SMD=0.35).

Table 1. Comparison of characteristics of study cohorts in the original unmatched sample.
CharacteristicsTreatment (n=74)Control (n=119)P valueStandardized differences
Sex, n (%).03−0.31
Female44 (59.50)51 (42.86)
Male30 (40.50)68 (57.14)
Race, n (%).370.11
Black16 (21.62)N/Aa
Hispanic or Latino2 (2.7)20 (16.8)
Other race2 (2.7)3 (2.5)
White54 (72.97)96 (80.7)
Education, n (%).740.02
Associate’s degree3 (4.05)3 (11.11)
Bachelor’s degree2 (2.70)1 (3.7)
High school diploma28 (37.84)7 (25.93)
Less than high school19 (25.68)6 (22.22)
Master’s degree1 (1.35)0 (0.00)
Some college16 (21.62)8 (29.63)
Unknown5 (6.76)94 (78.99)
Relationship, n (%).05−0.58
Divorced7 (9.46)5 (4.2)
In a relationship4 (5.41)0 (0.00)
Married11 (14.86)7 (5.88)
Single50 (67.57)38 (31.93)
Unknown2 (2.7)64 (53.78)
SeparatedN/A5 (4.2)
Insuranceb, n (%).020.35
Commercial5 (6.76)21 (17.65)
Medicaid58 (78.38)50 (42.02)
Medicare9 (12.16)11 (9.24)
Not insured2 (2.7)4 (3.36)
Medicaid and Medicare,N/A1/119 (0.84)
Urine rate, mean (95% CI)0.92 (0.77-1)0.86 (0.71-1.0)N/A0.34
Appt rate, mean (95% CI)0.65 (0.44-0.87)0.62 (0.39-0.84)N/A0.43

aN/A: not applicable.

bThis is insured by both Medicare and Medicaid.

Table 2 shows the characteristics of matched cohorts. After matching, the sample size for the treatment was 66, whereas the sample size for the control was 59. All the characteristics are well balanced between the matched cohorts. While most variables achieved excellent balance after matching (SMDs <0.1), gender retained a moderate imbalance (SMD=0.23) despite the matching procedure. This moderate imbalance in gender distribution represents a limitation of our matching process that warrants consideration when interpreting the results.

Table 2. Comparison of characteristics between treatment and control participants in the propensity score matched sample.
CharacteristicsTreatment (n=66)Control (n=59)P valueStandardized difference
Sex, n (%).440.23
Female37 (56.06)29 (49.15)
Male29 (43.94)30 (50.85)
Race, n (%).400.02
Black14 (21.21)16 (27.12)
Hispanic or Latino2 (3.03)0 (0.00)
Other race1 (1.52)3 (5.1)
White49 (74.24)40 (67.8)
Education, n (%).790.06
Associate’s degree3 (4.55)3 (11.11)
Bachelor’s degree2 (3.03)1 (3.7)
High school diploma/GED25 (37.88)7 (25.93)
Less than high school16 (24.24)6 (22.22)
Master’s degree1 (1.52)0 (0.00)
Some college14 (21.21)8 (29.63)
Unknown5 (7.58)2 (7.41)
Relationship, n (%).150.08
Divorced6 (9.09)5 (8.47)
In a relationship1 (1.52)0 (0.00)
Married11 (16.67)7 (11.86)
Single46 (69.7)38 (64.41)
Unknown2 (3.03)4 (6.78)
Separated0 (0.00)5 (8.47)
Insurance, n (%).350.03
Commercial5 (7.58)4 (14.81)
Medicaid50 (75.76)21 (77.78)
Medicare9 (13.64)2 (7.41)
Not insured2 (3.03)N/Aa
Urine rate, mean (95% CI)0.92 (0.76-1.09)0.84 (0.64-1.05)N/A0.34
Appointment rate, mean (95% CI)0.69 (0.5-0.89)0.49 (0.3-0.69)N/A0.43

aN/A: not applicable.

Abstinence

After PSM, we conducted t tests to compare urine rate between the control and experimental groups for the matched sample of 125 observations (59 controls+ 66 treated). Table 3 shows the urine test consistency rate test statistics between the treatment and control cohorts. This suggests that the treatment intervention likely had a positive impact on urine rate outcomes. The urine test abstinence rate, as indicated by the urine rate, was significantly higher in the DCM group (mean 0.92, 95% CI 0.88‐0.96) than in the treatment-as-usual group (mean 0.85, 95% CI 0.79‐0.90; P<.01).

Table 3. t test results after matching.
GroupObservedMean (SD)95% CIP value
Urine rate.01
Control590.85 (0.21)0.79-0.90
Treatment660.92 (0.16)0.88-0.96
Appointment rate.001
Control590.50 (0.20)0.45-0.55
Treatment660.69 (0.19)0.65-0.74

Appointment Attendance Rate

Table 3 shows the appointment attendance rate statistics between the treatment and control cohorts. The appointment adherence rate was substantially higher in the DCM group (mean 0.69, 95% CI 0.65‐0.74) than in the treatment-as-usual group (mean 0.50, 95% CI 0.45‐0.55), with an even stronger statistical significance (P<.001).

Sensitivity Analysis

We conducted a sensitivity analysis using Stata’s mhbounds command, which implements Rosenbaum bounds to evaluate the robustness of treatment effects to potential unmeasured confounding. This method assesses how much hidden bias (Γ) would be required to invalidate the statistical significance of the observed treatment effect. Our findings demonstrate that the treatment effects for both urine test rate and appointment attendance rate are robust to unmeasured confounding across a wide range of Γ values (1-3).

Complete sensitivity analysis results are shown in Tables S1 and S2 in Multimedia Appendix 1. Table S1 in Multimedia Appendix 1 on urine test rate shows that both sig+ and sig− values remain at 0 across all Γ values (1-2), indicating that the treatment effect maintains statistical significance even under strong assumptions of unmeasured confounding. The treatment effect estimate (t-hat) shows remarkable stability, ranging from 0.90 at Γ=1 to 0.86 (t-hat+) and 0.93 (t-hat−) at Γ=2. This minimal change suggests that the effect is highly robust. The CI values remain relatively narrow throughout (0.88‐0.91 at Γ=1 to 0.84-0.95 at Γ=2), further supporting the stability of the findings.

Table S2 in Multimedia Appendix 1 on appointment rate shows that both sig+ and sig− values remain at 0 across all Γ values, indicating sustained statistical significance. The treatment effect ranges from 0.67 at Γ=1 to 0.58-0.74 at Γ=2, showing slightly more variation than the urine rate analysis but still maintaining clinical significance. While the CI values widen somewhat with increasing Γ values (0.63‐0.69 at Γ=1 to 0.55-0.77 at Γ=2), they remain within interpretable bounds.

For both outcomes, the experimental group consistently outperformed the control group with statistically significant differences (P<.05). The minimal changes in treatment effect estimates and CI values across increasing levels of Γ suggest that our results are highly robust to potential unmeasured confounding. These findings support the conclusion that the intervention was likely effective in improving both urine test abstinence and appointment attendance rates.


Principal Findings

Our study aimed to evaluate the effectiveness of DCM in comparison with treatment-as-usual for individuals with SUD. The findings reveal statistically significant differences in both urine test abstinence and appointment attendance among participants in the DCM group compared with the treatment-as-usual group. Patients in the DCM group showed higher urine abstinence rates (92% vs 85%; P<.01) and substantially better appointment attendance rates (69% vs 50%; P<.001). These results align with a previous study [28], which found that digital health interventions can effectively enhance medication adherence among individuals with opioid use disorder. Also, DeFulio [14] found that inner-city patients with SUD who were mostly treated with buprenorphine and the DynamiCare app showed significantly higher rates of treatment attendance in study months 2-4 (9.6%‐18.0% increases) and odds ratio of 4.84 for greater drug abstinence and medication adherence than the treatment-as-usual group (P<.05). These results highlight the potential of DCM as an innovative intervention for improving key treatment outcomes in SUD care. Our findings extend the existing literature on traditional CM approaches and contribute to the much-needed evidence on the efficacy of DCM for drug treatment, supporting the broader transition toward digital therapeutic interventions in addiction treatment [12].

These results suggest that integrating digital tools into CM can enhance both treatment abstinence and engagement in care, consistent with previous studies highlighting the promise of digital interventions in substance misuse treatment [10,14]. This finding suggests that DCM interventions are effective in promoting abstinence, a critical metric in SUD treatment. Such gains align with previous findings on CM’s efficacy in motivating behavior change. Mobile apps can better communicate between the care team and the patients and could potentially save lives [29].

A 19% increase in appointment attendance with DCM is clinically meaningful, consistent with previous findings on digital interventions on treatment adherence [25]. This increase highlights the role of DCM in fostering engagement with care, an essential factor for successful treatment outcomes. By leveraging digital tools to deliver incentives, DCM may reduce the impact of logistical challenges, thus promoting consistent participation in scheduled appointments [30]. Compared with patients who received treat-as-usual alone, those who had treatment-as-usual coupled with app-based CM had a noticeably higher chance of remaining in treatment for longer [31]. These results corroborate earlier research that concluded that receiving rewards motivates patients to maintain treatment compliance [10].

The digital platform can decrease obstacles to incentive receipt and help remove infrastructure barriers that may impact attendance rates. Urban-rural disparities exist in CM availability [32]. Digital interventions could expand access to a broader patient population [33,34], suggesting that DCM could help reduce this disparity. Thus, DCM could help address the inequalities in health care support for the underserved groups [35,36].

According to earlier studies, digital interventions decrease the impact of logistical obstacles to participation, such as schedule conflicts and travel time, which are frequent problems in conventional therapy settings. The results support these views, which highlight DCM’s capacity to encourage long-lasting behavioral improvements in SUD populations [14,37]. In sum, DCM interventions align with behavioral reinforcement principles by providing timely rewards for desirable behaviors, thus enhancing adherence and reducing substance misuse [10].

Limitations

Despite these promising findings, certain limitations warrant consideration. First, the study did not explore differences in outcomes across social determinants of health, motivation levels, or comorbidity. Future studies will explore the efficacy of DCM on SUD treatment relative to educational level, economic stability, transportation availability, and housing stability. Nevertheless, our sample’s high proportion of Medicaid and noncommercial insureds suggests that patients at risk due to social determinants of health challenges may benefit. The gender imbalance may affect the generalizability of our findings. Future research should include larger samples, powered to support more robust gender-stratified analyses.

Second, the study may have excluded individuals less familiar with digital technology. App features and user experience could affect engagement with DCM. This digital divide may disproportionately affect certain demographic groups, such as older adults or those from lower socioeconomic or educational backgrounds, potentially limiting the generalizability of the findings to all individuals with SUD. However, the front-loading of rewards is designed to motivate patients to overcome tech unfamiliarity.

Third, the study focused on outcomes, specifically drug abstinence and appointment adherence, during the intervention. While these measures are critical indicators of treatment success, the study did not assess the sustainability of behavior changes following the conclusion of the intervention. Follow-up was determined to be ineffective due to changing patient contact information and availability to be able to continue data collection post intervention. The 12-month duration of the study, however, exceeds that of many previous trials.

Fourth, this study used a quasi-experimental design with alternating assignment rather than true randomization. The selection of participants was based on a sequential allocation process where every other patient was offered participation in the DCM intervention. The nonrandom assignment process introduces selection bias. Patients who chose to participate in the DCM project may differ systematically from those who declined to do so, particularly in terms of motivation, technological aptitude, or other unmeasured characteristics that could influence treatment outcomes. Self-selection into the experimental group could mean that more motivated or tech-savvy patients were overrepresented in the experimental group, which could impact the generalizability of results. However, we conducted postmatching analyses to reduce observable confounding, and the statistically significant differences observed in primary outcomes suggest that the intervention still had a meaningful impact despite this limitation.

While our PSM approach attempted to balance observed characteristics between groups, it cannot account for unmeasured confounders that might have influenced both group assignment and outcomes. Although we conducted sensitivity analyses to assess the robustness of our findings to unmeasured confounding, selection bias remains an inherent limitation of our design. At the same time, anecdotally, it does not appear that technology was a limiting feature for those opting out. In addition, the variability observed within the appointment and urine rates suggests that individual differences, including socioeconomic status, comorbid conditions, and personal motivation, may affect treatment outcomes. Future research should explore these contextual factors to identify and address potential barriers to successful implementation and should use randomized controlled designs when feasible.

Finally, while our PSM successfully balanced most covariates between treatment and control groups, a moderate imbalance in gender (SMD=0.23) persisted after matching. This exceeds our predetermined threshold of 0.2 and represents a potential limitation. The potential influence of this residual gender imbalance on our results cannot be fully ruled out and should be explored in future studies with larger sample sizes that would enable gender-stratified analyses. Future studies should use more stringent matching algorithms or consider exact matching on gender to eliminate this potential source of bias.

Conclusions

This article supports the validity of DCM as an intervention to improve treatment adherence and outcomes for people with SUD. For the DCM group, urine test abstinence and appointment attendance were significantly higher than those in treat-as-usual recipients. These results suggest that technology-enabled interventions can improve patient compliance and engagement.

The improvements in both abstinence and appointment attendance suggest that DCM could be a cost-effective addition to SUD treatment, especially considering associated potential reductions in health care utilization, emergency services, and societal costs associated with SUDs. DCM additionally allows for greater accessibility while also removing administrative overhead due to automation. As cell phone usage continues to grow, potential equity concerns should be diminished.

The findings could provide additional support for enacting policies that reimburse DCM as a viable medical protocol that improves treatment compliance and outcomes. Our retention and abstinence outcomes support the inclusion of DCM in emerging value-based payment models. Currently, most state Medicaid programs do not have specific billing codes for CM. However, DynamiCare platform’s verification features address a key concern in current reimbursement policies by providing robust documentation of incentive distribution and behavioral verification, directly meeting Medicaid’s increasing requirements for service documentation and accountability. At the same time, we recommend that state and federal regulatory bodies establish clear guidelines for DCM implementation that address privacy concerns and verification standards.

Acknowledgments

The authors are grateful to DynamiCare Health, Talbert House, Journey Recovery Care of St. Elizabeth Healthcare, the Greater Cincinnati Behavioral Health Services, and the Center for Addiction Treatment in Cincinnati for their interest, support, and participation in this research project. Funding was provided by Realworks (grant 4001541), an Ohio-based nonprofit. The authors are grateful for it. Generative artificial intelligence (Copilot) was used to proofread a small portion of the manuscript for grammar check and readability.

Data Availability

The data are not publicly available per the data use agreements with each of the participating sites and the funder.

Authors' Contributions

XZ contributed to writing and analysis of the original draft. VH contributed to conceptualization, reviewing, and editing of the manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Rosenbaum bounds sensitivity analyses on urine test and appointment rates.

DOCX File, 21 KB

  1. 2023 national survey on drug use and health (NSDUH): methodological summary and definitions. Substance Abuse and Mental Health Services Administration (SAMHSA). URL: https:/​/www.​samhsa.gov/​data/​sites/​default/​files/​reports/​rpt47098/​Methodological%20Summary%20and%20Definitions/​2023-nsduh-method-summary-defs.​htm [Accessed 2024-12-31]
  2. Scaling up care for mental, neurological, and substance use disorders: mhGAP. World Health Organization (WHO). URL: https://www.who.int/activities/scaling-up-mental-health-care [Accessed 2024-12-31]
  3. Global health. National Institute on Drug Abuse. URL: https://nida.nih.gov/research-topics/global-health [Accessed 2025-07-27]
  4. Bouchard M, Lecomte T, Cloutier B, Herrera-Roberge J, Potvin S. Dropout rates in psychosocial interventions for people with both severe mental illness and substance misuse: a systematic review and meta-analysis. Front Psychiatry. 2022;13:842329. [CrossRef] [Medline]
  5. Lappan SN, Brown AW, Hendricks PS. Dropout rates of in-person psychosocial substance use disorder treatments: a systematic review and meta-analysis. Addiction. Feb 2020;115(2):201-217. [CrossRef] [Medline]
  6. Alessi SM, Rash CJ, Pescatello LS. Reinforcing exercise to improve drug abuse treatment outcomes: a randomized controlled study in a substance use disorder outpatient treatment setting. Psychol Addict Behav. Feb 2020;34(1):52-64. [CrossRef] [Medline]
  7. de Mattos BP, Mattjie C, Ravazio R, Barros RC, Grassi-Oliveira R. Craving for a robust methodology: a systematic review of machine learning algorithms on substance-use disorders treatment outcomes. Int J Ment Health Addiction. 2024:1-28. [CrossRef]
  8. Bolívar HA, Klemperer EM, Coleman SRM, DeSarno M, Skelly JM, Higgins ST. Contingency management for patients receiving medication for opioid use disorder. JAMA Psychiatry. Oct 1, 2021;78(10):1092. [CrossRef]
  9. Ginley MK, Pfund RA, Rash CJ, Zajac K. Long-term efficacy of contingency management treatment based on objective indicators of abstinence from illicit substance use up to 1 year following treatment: a meta-analysis. J Consult Clin Psychol. Jan 2021;89(1):58-71. [CrossRef] [Medline]
  10. Petry NM, Alessi SM, Olmstead TA, Rash CJ, Zajac K. Contingency management treatment for substance use disorders: how far has it come, and where does it need to go? Psychol Addict Behav. Dec 2017;31(8):897-906. [CrossRef] [Medline]
  11. Proctor SL. Rewarding recovery: the time is now for contingency management for opioid use disorder. Ann Med. Dec 2022;54(1):1178-1187. [CrossRef] [Medline]
  12. Dallery J, Ives L, Knerr A. Toward an era of impact of digital contingency management in the treatment of substance use disorders. Prev Med. Nov 2023;176:107518. [CrossRef] [Medline]
  13. Hallgren KA, Duncan MH, Iles-Shih MD, et al. Feasibility, engagement, and usability of a remote, smartphone-based contingency management program as a treatment add-on for patients who use methamphetamine: single-arm pilot study. JMIR Form Res. Jul 6, 2023;7:e47516. [CrossRef] [Medline]
  14. DeFulio A. Dissemination of contingency management for the treatment of opioid use disorder. Perspect Behav Sci. Mar 2023;46(1):35-49. [CrossRef] [Medline]
  15. Hammond AS, Sweeney MM, Chikosi TU, Stitzer ML. Digital delivery of a contingency management intervention for substance use disorder: a feasibility study with DynamiCare Health. J Subst Abuse Treat. Jul 2021;126:108425. [CrossRef] [Medline]
  16. Sugarman DE, Campbell ANC, Iles BR, Greenfield SF. Technology-based interventions for substance use and comorbid disorders: an examination of the emerging literature. Harv Rev Psychiatry. 2017;25(3):123-134. [CrossRef] [Medline]
  17. Murphy SM, Campbell ANC, Ghitza UE, et al. Cost-effectiveness of an internet-delivered treatment for substance abuse: data from a multisite randomized controlled trial. Drug Alcohol Depend. Apr 2016;161:119-126. [CrossRef] [Medline]
  18. Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med. Feb 20, 2007;26(4):734-753. [CrossRef] [Medline]
  19. Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. Nov 10, 2009;28(25):3083-3107. [CrossRef] [Medline]
  20. Kane LT, Fang T, Galetta MS. Propensity score matching: a statistical method. Clin Spine Surg. Apr 2020;33(3):120-122. [CrossRef] [Medline]
  21. Rosenbaum PR, Rubin DB. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am Stat. Feb 1985;39(1):33-38. [CrossRef]
  22. Ali MS, Groenwold RHH, Belitser SV, et al. Reporting of covariate selection and balance assessment in propensity score analysis is suboptimal: a systematic review. J Clin Epidemiol. Feb 2015;68(2):112-121. [CrossRef] [Medline]
  23. van Emmerik-van Oortmerssen K, Blankers M, Vedel E, et al. Prediction of drop-out and outcome in integrated cognitive behavioral therapy for ADHD and SUD: results from a randomized clinical trial. Addict Behav. Apr 2020;103:106228. [CrossRef] [Medline]
  24. Greenfield SF, Brooks AJ, Gordon SM, et al. Substance abuse treatment entry, retention, and outcome in women: a review of the literature. Drug Alcohol Depend. Jan 5, 2007;86(1):1-21. [CrossRef] [Medline]
  25. Kahle EM, Veliz P, McCabe SE, Boyd CJ. Functional and structural social support, substance use and sexual orientation from a nationally representative sample of US adults. Addiction. Mar 2020;115(3):546-558. [CrossRef] [Medline]
  26. Stahler GJ, Mennis J, DuCette JP. Residential and outpatient treatment completion for substance use disorders in the U.S.: moderation analysis by demographics and drug of choice. Addict Behav. Jul 2016;58:129-135. [CrossRef] [Medline]
  27. Creedon TB, Wayne GF, Progovac AM, Levy DE, Cook BL. Trends in cigarette use and health insurance coverage among US adults with mental health and substance use disorders. Addiction. Feb 2023;118(2):353-364. [CrossRef] [Medline]
  28. Schramm ZA, Leroux BG, Radick AC, et al. Video directly observed therapy intervention using a mobile health application among opioid use disorder patients receiving office-based buprenorphine treatment: protocol for a pilot randomized controlled trial. Addict Sci Clin Pract. Jul 31, 2020;15(1):30. [CrossRef] [Medline]
  29. Flickinger TE, Waselewski M, Tabackman A, et al. Communication between patients, peers, and care providers through a mobile health intervention supporting medication-assisted treatment for opioid use disorder. Patient Educ Couns. Jul 2022;105(7):2110-2115. [CrossRef] [Medline]
  30. Rawson RA, Erath TG, Chalk M, et al. Contingency management for stimulant use disorder: progress, challenges, and recommendations. J Ambul Care Manage. 2023;46(2):152-159. [CrossRef] [Medline]
  31. Langdon KJ, Ramsey S, Scherzer C, Carey K, Ranney ML, Rich J. Development of an integrated digital health intervention to promote engagement in and adherence to medication for opioid use disorder. Addict Sci Clin Pract. Apr 29, 2020;15(1):16. [CrossRef] [Medline]
  32. Marino EN, Karns-Wright T, Perez MC, Potter JS. Smartphone app-based contingency management and opioid use disorder treatment outcomes. JAMA Netw Open. Dec 2, 2024;7(12):e2448405. [CrossRef] [Medline]
  33. Joudrey PJ, Chen K, Oldfield BJ, Biegacki E, Fiellin DA. Drive time to addiction treatment facilities providing contingency management across rural and urban census tracts in 6 US states. J Addict Med. 2023;17(5):615-617. [CrossRef] [Medline]
  34. Hampton J, Mugambi P, Caggiano E, et al. Closing the digital divide in interventions for substance use disorder. J Psychiatr Brain Sci. 2024;9(1):e240002. [CrossRef] [Medline]
  35. McPherson SM, Burduli E, Smith CL, et al. A review of contingency management for the treatment of substance-use disorders: adaptation for underserved populations, use of experimental technologies, and personalized optimization strategies. Subst Abuse Rehabil. 2018;9:43-57. [CrossRef] [Medline]
  36. Lee EW, McCloud RF, Viswanath K. Designing effective eHealth interventions for underserved groups: five lessons from a decade of eHealth intervention design and deployment. J Med Internet Res. Jan 7, 2022;24(1):e25419. [CrossRef] [Medline]
  37. Gilbert MK, Daughton AR, Chilcoat HD, Laffont CM, Strafford S, DeVeaugh-Geiss AM. Social listening for patient experiences with stopping extended-release buprenorphine: content analysis of reddit messages. J Med Internet Res. Apr 25, 2025;27:e71245. [CrossRef] [Medline]


CM: contingency management
DCM: digital contingency management
HIPAA: Health Insurance Portability and Accountability Act
IRB: institutional review board
PSM: propensity score matching
SMD: standardized mean difference
SUD: substance use disorder


Edited by John Torous; submitted 07.03.25; peer-reviewed by Chiamaka Pamela Agu, Cordia Ogbeta; final revised version received 06.06.25; accepted 07.06.25; published 02.09.25.

Copyright

© Xiaoni Zhang, Valerie Hardcastle. Originally published in JMIR Mental Health (https://mental.jmir.org), 2.9.2025.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.