This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Research supports the effectiveness of technology-based treatment approaches for substance use disorders. These approaches have the potential to broaden the reach of evidence-based care. Yet, there is limited understanding of factors associated with implementation of technology-based care approaches in different service settings.
In this study, we explored provider and staff perceptions of facilitators and barriers to implementation of a mobile phone substance use recovery support app with clients in 4 service settings.
Interviews were conducted with leadership and provider stakeholders (N=12) from 4 agencies in the first year of an implementation trial of the mobile phone app. We used the Consolidated Framework for Implementation Research as the conceptual foundation for identifying facilitators and barriers to implementation.
Implementation process facilitators included careful planning of all aspects of implementation before launch, engaging a dedicated team to implement and foster motivation, working collaboratively with the app development team to address technical barriers and adapt the app to meet client and agency needs, and consistently reviewing app usage data to inform progress. Implementation support strategies included training all staff to promote organization awareness about the recovery support app and emphasize its priority as a clinical care tool, encouraging clients to try the technology before committing to use, scaling rollout to clients, setting clear expectations with clients about use of the app, and using peer coaches and consistent client-centered messaging to promote engagement. Perceived compatibility of the mobile phone app with agency and client needs and readiness to implement emerged as salient agency-level implementation facilitators. Facilitating characteristics of the recovery support app itself included evidence of its impact for recovery support, perceived relative advantage of the app over usual care, the ability to adapt the app to improve client use, and its ease of use. The mobile phone itself was a strong motivation for clients to opt in to use the app in settings that provided phones. App access was limited in settings that did not provide phones owing to lack of mobile phone ownership or incompatibility of the app with clients’ mobile phones. Individual differences in technology literacy and provider beliefs about substance use care either facilitated or challenged implementation. Awareness of patient needs and resources facilitated implementation, whereas external policies and regulations regarding technology use introduced barriers to implementation.
The conceptually grounded facilitators and barriers identified in this study can guide systematic targeting of strategies to improve implementation of mobile phone interventions in community treatment settings. Results also inform the design of technology-based therapeutic tools. This study highlights directions for research with regard to implementation of technology-based behavioral health care approaches.
Addiction and mental health treatment programs have been particularly slow to adopt evidence-based practices [
There is strong and growing evidence to support the effectiveness of technology-based treatment approaches for substance use disorders across the care continuum, including screening and assessment [
Mobile phone technologies offer a promising platform for delivery of substance use treatment approaches. Use of mobile phone technologies continues to rapidly grow across age, race ethnicity and geography, and consumers increasingly rely on the Internet and mobile phone–based tools for health information [
Technology-based substance use treatment approaches offer the potential for on-demand access to care across time and geographic location. These tools can also extend the reach of services to traditionally underserved and disfranchised populations who perceive stigma regarding service use, such as those with substance use conditions or mental illness. There is also growing support for the cost-effectiveness of technology-based substance use treatment approaches [
Despite strong empirical evidence to support the effectiveness of technology-based therapeutic approaches to substance use, the field is relatively nascent with regard to guidance on the process of implementing these approaches in community care settings. In this qualitative study, we explored provider and staff perceptions of implementation of the A-CHESS mobile recovery support app with clients in 4 addiction service settings. By identifying facilitators and barriers to implementation, we can begin to develop clearer guidelines to support adoption and implementation of technology-based tools in diverse settings.
We used the Consolidated Framework for Implementation Research (CFIR) [
The first author’s institutional review board (IRB) approved the study. Participating service sites were members of the Comprehensive Health Enhancement Support System Health Education Consortium (CHEC), organized by the A-CHESS app development researchers at the University of Wisconsin, Madison (UW). The purpose of the CHEC was to study how A-CHESS would be used by service organizations in naturalistic implementations. As part of consortium membership, agencies made a donation ($10,000) to support consortium activities and agreed to participate in studies generated from consortium member interests, as possible. The UW team provided member agencies with access to A-CHESS and ongoing technical assistance. Technical assistance included training materials for program setup and monthly telephone support.
Eight agencies participated in the first year of the consortium, representing substance use treatment, community behavioral health, and drug court settings located in the Northeast (2), Midwest (2), South (2), and West (2) of the United States. Each agency committed to make A-CHESS available to up to 100 clients over a 1-year period. Agencies determined how and with whom A-CHESS would be used at their organization as well as how to engage clinical and administrative staff in the implementation process.
Dartmouth researchers presented the plan for this implementation process study to the 8 consortium agencies during a regular CHEC monthly teleconference and sent a separate follow-up email invitation to each consortium member. Of the 8 sites, 4 (50%) agreed to participate in this study. Four consortium agencies elected not to participate, primarily owing to lack of time and delays in implementation of A-CHESS. Agencies that participated in the study did not differ from those that did not participate on key demographic indicators, including type of setting, services provided, and client demographics.
All 4 participating organizations were within the first year of implementation of A-CHESS. Agency details include:
1. A northeastern addiction recovery center that specializes in services to veterans used A-CHESS in combination with medication-assisted treatment for veterans with a high rate of alcohol detox admissions. The agency provided mobile phones and data plans to clients, through funding from a federal grant, to foster standardization of client experience and internal technical support. The medical director, nurse case manager, and director of information technology were interviewed for this study.
2. A northeastern drug court program integrated A-CHESS into their substance abuse treatment program for offenders. Mobile phones and data plans were provided to clients through a federal grant. Interviews were conducted with the change leader, the caseworker, and the peer recovery coach hired to support implementation.
3. An addiction treatment center based outside a major northeastern city offered A-CHESS to “alumni” of its inpatient treatment program. Only clients with compatible mobile phones were offered the A-CHESS app. Interviews were conducted with the alumni services coordinator, the inpatient administrator, and the training director involved in implementation.
4. An outpatient behavioral health agency in the Midwest offered A-CHESS as a resource for posttreatment support for clients with mobile phones. Interviews were conducted with the agency program director, training director, and clinical supervisor overseeing implementation of A-CHESS at the agency.
For each participating agency, 3 stakeholders were invited to participate in the study. Stakeholders represented leadership and clinical perspectives on the implementation of A-CHESS. Because the study focused on agency efforts to promote implementation of the mobile phone app, clients were not included as stakeholders. All stakeholders who were invited agreed to participate.
An interview guide was created to elicit stakeholder perspectives on the implementation process of the A-CHESS mobile phone app. The guide included probes associated with the decision to become a consortium member, preimplementation planning and preparation strategies, implementation experiences, monitoring of progress and success, experiences of technical assistance and support, and plans for sustainability.
A total of 12 stakeholders were interviewed for the study (3 from each agency). Participants were 50% female, predominantly white (91%), and ranged in age from 25 to 53 (mean: 36.7) years. A postdoctoral researcher trained in qualitative methods conducted the 20- to 30-minute interviews in early 2013. All interviews were audio-recorded and transcribed, with the exception of one wherein the audio recording failed. In this instance, interview summary notes were created immediately after the interview. Because summary notes were not conducive to further coding, we analyzed 11 interviews for this study.
Researchers trained in qualitative methods (coauthors SM and SL) reviewed and coded interview transcripts using a deductive, consensus-based directed content analysis approach to strengthen the trustworthiness of the analysis [
Results are described first in terms of the implementation process for each agency, followed by description of contextual facilitators and barriers to implementation that emerged across agencies and stakeholders. Represented CFIR coding themes across agencies and stakeholders are depicted in
This agency offered a set of integrated services to clients including A-CHESS, naltrexone, individual therapy, and a recovery coach. At the time of the interviews, 45 of 50 eligible clients were using A-CHESS. The agency fostered engagement through creation of a dedicated implementation team that included the following: (1) a nurse case manager who identified and trained eligible clients and (2) the information technology director who set up the technology infrastructure, oversaw all internal technical assistance and troubleshooting, and managed the data collected through A-CHESS.
Implementation of A-CHESS was marked by detailed planning before the launch, including: (1) identifying strategies for recruitment of clients, obtaining consents, and training of all agency providers and clients, (2) identifying the types of mobile phones and data plans that would be most compatible with the agency service area (working with a local phone vendor), and (3) determining processes for monitoring and maintaining online app features (eg, discussion board) and for how to effectively use data to inform implementation. Clearly defined roles, regular meetings, and open communication between team members and with clients allowed for adaptability during implementation to improve compatibility with agency and client needs.
We saw actually a tipping point...one veteran had reached out on the discussion board and there was radio silence…the member of our team managing A-CHESS had a talk with the veterans and the next week, the same veteran sent out a distress signal and 20 vets descended upon the cellphone, and that was the tipping point. Now it’s instant support.
Ongoing collaboration with the UW app development researchers to address technical issues and adaptations as needed to improve fit with the agency and clients also facilitated implementation, as did routine reviews of data to evaluate implementation and client engagement.
… our CEO, he is definitely the one who is…taking the numbers and talking to other places where he thinks this could be beneficial…It could be a grant, it could become a VA benefit…trying to show the VA that it’s gonna cost them a lot less to keep these people in the A-CHESS program than paying for five detoxes a year…inpatient stays are always more expensive.
A-CHESS and mobile phones were offered to all drug court participants. A dedicated team was created that included a certified change leader, a caseworker, and a peer recovery coach. At the time of the interviews, 40 clients were using the recovery support app. Familiarity with A-CHESS from a prior pilot facilitated preimplementation activities. Careful planning of client training, execution of implementation, and tailoring of the app to meet client needs facilitated implementation. The agency worked closely with the UW researchers and their IRB to address concerns about features that could compromise client privacy and confidentiality (with potential legal implications) to ensure protection of clients, including prohibiting probation officers from accessing client information.
Regular team meetings, open communication between team members and with clients, close attention to client needs, and ongoing review of data to monitor client usage, risk, and outcomes all helped to create an organization culture that valued use of A-CHESS and made it a routine aspect of treatment. Client-centered implementation strategies to promote engagement included staged introduction to new clients and use of peer recovery coaches, establishing contracts with clients to set clear expectations about app usage, encouraging clients to try the app before committing to use it, using consistent client-centered messaging about the app, and supporting peer-driven management of app features (eg, discussion board).
It is client-to-client…our staff step back…and we let the peers run it. They police it…Because we don’t dominate it, they own it. They feel very empowered…
The more people you have using the app, the more benefits others get out of it…You have to encourage clients to use it. Once they start using it…the clients will start encouraging the other clients to use it…Make sure clients know it’s for their benefit, it’s for them. The minute they think it’s for your benefit you will find they are resistant…
The team at this agency offered the A-CHESS app to alumni clients of the inpatient treatment program as one of several postdischarge resources. Access was available only to those with a compatible mobile phone. Inpatient alumni were targeted owing to disinterest in the A-CHESS project among outpatient treatment clinicians. The focus on this client subgroup also allowed for a more manageable implementation process. At the time of the interviews, approximately 40 clients were using the app.
The agency engaged a dedicated team that included the alumni services coordinator, the inpatient administrator for clinical outreach, and the training director. The team paid careful attention to planning implementation rollout, including designing a course of action for training clients with the app, tailoring implementation for client subgroups (eg, older clients, those with learning disabilities), setting expectations for use of the discussion board, and identifying response strategies to “Panic” outreach. Clients were oriented to the app as part of inpatient discharge planning to build awareness and interest in use of the app as a postdischarge resource.
We always talk about the alumni stuff, and we always talk about the app and just kind of put it in their head that there is this app out there and it’s really cool…After they’ve been there a few weeks, that’s when I start meeting with them and talking about aftercare…and just letting them play around with the app on my phone…Before their discharge we meet again to discuss if they want to get [the app] and then I will actually download
Other process facilitators included working collaboratively with the UW app development researchers on adaptations to improve client engagement (eg, notification feature for discussion board) and ongoing use of data to monitor client engagement and implementation progress. For example, early review of client usage data and time spent in relapse suggested that relapse times were shorter for those clients more actively using the app, i.e. agency staff was able to intervene quickly because of notification about relapse. Agency stakeholders used this information to promote engagement of clients and to highlight the value of the mobile phone app to administrators to promote ongoing adoption.
This agency offered A-CHESS as an ancillary posttreatment support for clients with a substance use disorder who had compatible mobile phones. At the time of the interviews, 5 clients had been set up to use the recovery support tool. A clinical supervisor was in charge of implementation efforts. The agency was familiar with the app from successful participation in prior projects wherein clients were given mobile phones. In this study, there were challenges recruiting clients with mobile phones.
Recruitment strategies shifted to target younger college-aged clientele, with limited success. The clinical supervisor also described general plans to make A-CHESS a more central component of treatment planning for clients. Stakeholders did not elucidate specific planning strategies for implementation of A-CHESS, ways in which data could inform implementation, or whether they engaged the UW researchers to promote implementation.
Salient barriers and facilitators to implementation of the app across settings and stakeholders are described in the following section (see
Across settings and stakeholders, perceived compatibility of A-CHESS with client needs emerged as a salient facilitator to implementation. Themes associated with the perceived compatibility included the ability to communicate when clients needed it the most (eg, “clients reach to their phones as a way to interrupt a bad moment” [Veterans Substance Use Treatment]), and to promote client empowerment (eg, “…it puts the power of recovery in the hands of the individual. It is the quintessential strength-based, person-centered model.” [Behavioral Health]
Noted barriers included app features that were incompatible with client populations. For example, stakeholders from the veterans’ treatment setting noted that the name of the outreach feature (Panic Button) was not compatible with military training (ie, “Soldiers don’t panic”), and was thus a feature this client subgroup was ambivalent about using. Drug Court stakeholders noted that the location-tracking feature could be used to violate client privacy and result in additional legal issues.
Agency-level compatibility barriers included concerns about lack of reimbursement for use of the mobile phone app in the care process, provider resistance to use of technology with clients (eg, concerns about therapeutic boundaries and 24/7 liability), and organization policies restricting use of mobile phones.
Implementation readiness marked by clear leadership support and availability of resources to support implementation (eg, training, dedicated team) facilitated implementation of A-CHESS with clients. Clear and consistent messaging to staff about the relative priority of the mobile phone app, open communication among staff, and a positive learning climate that supported workflow adaptation to improve implementation all facilitated implementation.
Characteristics of the A-CHESS app itself were salient for implementation. Design quality and packaging and evidence of strength and quality were the two most referenced characteristics.
Design quality and packaging is defined as how the intervention is bundled, presented, and assembled [
Accessibility was a significant barrier to implementation in agencies that did not issue mobile phones both by virtue of clients not owning mobile phones or having phones that were incompatible with the A-CHESS android app (ie, iPhone or Blackberry). A number of clinical barriers were noted. For example, drug court staff were concerned about adverse effects of relapsing clients’ posts on other clients in the online community.
A stakeholder from the addiction treatment setting expressed concern about potential iatrogenic triggering of relapse by the location-based alert feature that makes a user “aware of every liquor store in your area.” Liability issues were also noted, including concern about the need for ongoing monitoring and response to client postings on the discussion board in the event of actionable incidents (eg, suicide threats). Other noted barriers included limited flexibility of the online administrative feature for providers, navigation challenges (eg, duplicate login areas) and sporadic technical problems with features that required wireless access (particularly in rural areas), and the influence of rapidly changing technologies.
Positive perceptions about the quality of the mobile phone app as a recovery support tool facilitated implementation across care settings (ie, “We’ve found this to be a great tool to stay connected, and it’s all recovery-based, which is great” [Addiction Treatment]). As a drug court stakeholder noted,
The relapse prevention model of A-CHESS itself is really well demonstrated with clients…the fact that clients can reach out 24/7 to peers…that interrupts the moment…the relapse prevention moment, they had something to do instead of sitting in their own head…it really does play out that clients reach to their phone as a way to interrupt a bad moment.
True to the mobile phone app’s purported mandate—to improve continuing care for individuals with substance use disorders by offering ongoing emotional and instrumental support—these quotes all reference contributions to adaptive functioning.
Salient barriers with regard to strength and quality of the app included limited longer term client engagement with the app (“Like a lot of other apps, you use it for a month…and then you start losing touch with it” [Addiction Treatment]) and questions about impact with dissemination to a broader client base (“I think if you have too many people on this…you kind of get lost in the whole cyberspace thing of who is this person—anybody could be on there…you want to keep it small so people feel comfortable actually interacting and putting messages on there…you get more interaction when people know each other” [Addiction Treatment]).
Perceptions of relative advantage of the recovery support app over usual care also facilitated implementation across service sites, particularly with regard to ongoing client support:
A lot of people who deal with drug issues are pretty solitary people…so it’s hard to get them to open up. When you’re doing it through the application, it’s somewhat anonymous…you’re not standing face-to-face…and it makes it a little bit easier to develop friendship.
Stakeholders generally did not indicate perceived relative advantages of the app with regard to their own workflow (eg, “Does it save me…on paperwork or phone calls? I would not say either. It’s probably just the same” [Addiction Treatment]).
Perceived ease of use of the app was a facilitator to implementation across settings (eg, “It is a very safe app; anything you press it easily directs you to go right back to it” [Addiction Treatment]).
The ability to try the app and adapt it to improve workflow and client use emerged as important implementation facilitators. For example, to overcome usability barriers with the discussion board due to duplicate login navigation, the implementation teams worked with UW development researchers to improve access to the discussion board and create notifications to alert clients to new material.
Cost was a perceived barrier to sustained implementation in the Veteran Substance Use Treatment Center, Drug Court, and Community Behavioral Health agency. Long-term sustainability of providing mobile phones to clients and paying for technical assistance from the UW development team were the most salient cost concerns.
Providers’ treatment philosophy and beliefs about the use of technology influenced implementation (eg, “this is not how I work with people”). Individual differences in clients’ technology and reading literacies, and degree of learning disability impacted the level of training required for implementation. The intuitive ease of use of the app aided in self-learning.
Once they got the hang of it they really don’t need more instruction. They loved it once they got playing with it. It was a surprise; people we thought would not use it much did use it
Overall agency awareness of patient needs and resources was positively associated with implementation. Agencies that modified implementation strategies to improve compatibility with clients’ needs experienced implementation success. The behavioral health agency, which had positive experience with A-CHESS in a prior pilot, referenced unawareness of patient needs and resources, specifically with regard to clients’ relative lack of access to mobile phones. This factor contributed significantly to the agency’s challenges with client recruitment to use the app.
External policies and regulations regarding technology use also introduced barriers to implementation. For example, in the drug court, concerns about client privacy and regulations prohibiting mobile phone use were only circumvented through IRB study provisions and a plan to exclude probation officers from access to client information from the mobile phone app. Such exclusion could be more difficult outside a research context.
Planning
Identify what mobile phones to use (if providing phones) to maximize compatibility with clientele and agency needs
Develop client recruitment strategies, when and how to introduce the app to clients, and training protocol
Create clear plans for monitoring program features to ensure client safety and privacy
Identify indicators of implementation success and develop plan for consistent monitoring and use of data to inform implementation and agency practices
Engagement
Create a dedicated internal team with clear role and responsibilities to lead implementation
Identify staff with positive attitudes toward the mobile phone recovery support approach to serve as champions to promote buy-in among clinicians and clients
As possible, collaborate with the mobile phone app development team to address technical issues and create adaptations to improve client engagement and fit with agency and client needs
Work with technology vendors to ensure that mobile phones to be issued are compatible with the mobile phone app software, the local service area, and the technology infrastructure of agency
Orient clients to the app early to build awareness and interest (eg, as a postdischarge resource during inpatient care)
Use contracts with clients to set mutual expectations about mobile phone app use
Seed discussion forums with conversation content to build client engagement
Execution
Scale rollout to work out implementation challenges
Meet regularly to review implementation and adjust workflow as needed
Obtain client feedback regarding app experiences to guide implementation adaptations
Reflection/evaluation
Conduct ongoing review of data to monitor client usage, risk, and client outcomes
Use data to adapt workflow processes to promote implementation
Inner Setting Characteristics
Adapt technology and implementation plans to maximize compatibility with agency workflow and needs
Train clinicians and staff to promote awareness about the mobile phone app and emphasize priority of the app as routine part of care
Develop and clearly communicate standards and practices to ensure protection of clients and clinicians with regard to use of the mobile phone recovery support app
Communicate clear indicators of implementation success to all staff to build perceptions of relative advantages of the technology-based approach and elicit staff engagement to help overcome implementation barriers
Intervention characteristics
Provide mobile devices to foster standardization of client experience and technical support
Preprogram important client support contacts to foster easy outreach when needed
Seek funding (eg, donations, minigrants) and leverage state Medicaid billing codes to subsidize hardware and software purchases
Trial the mobile phone app with end users to promote buy-in
Characteristics of individuals
Provide ongoing technical assistance and education to staff and clinicians to increase perceptions of the app as compatible with client needs and effective care
Train clients individually to accommodate differences in technology literacy and learning disabilities
Tailor the app to meet client needs
Tailor implementation for client subgroups
Highlight features to aid those with literacy challenges (eg, speech-to-text functionality)
Outer setting characteristics
Maintain client-centered approach to care that prioritizes client needs and resources
Assess technology ownership and use among an agency’s client base to provide foundation for planning of technology-based service delivery initiatives
Use data from successful implementation to inform administrative policy decisions
Change agency policies if necessary to accommodate use of mobile care approaches
The CFIR conceptual framework provided a valuable lens through which to identify key barriers and facilitators to implementation of A-CHESS with clients from 4 service settings. Results of the study contribute to the field in 3 substantive ways: (1) informing practice for implementation of mobile phone technology approaches for addiction treatment and behavioral health care more broadly, (2) informing development of mobile phone apps to optimize implementation success, and (3) informing directions for implementation science research with regard to use of mobile phone technologies for behavioral health care.
Successful implementation of the recovery support app was marked by careful attention to planning for implementation before launch, including what mobile phones to use, what client populations could benefit the most from program use, when to introduce the app to clients, processes for training staff and clientele, how to monitor program features most effectively in terms of staff time and expertise, and how to effectively use data to inform implementation. A key component to planning was early engagement of a dedicated, appropriately trained internal team to foster implementation buy-in among agency providers and clients and facilitate implementation. Iterative and ongoing technical assistance from the app development team to improve the fit of A-CHESS with the agencies and their respective clientele also facilitated implementation. One agency also worked with a local technology vendor to ensure that the mobile phones to be issued to clients were compatible with the mobile app software, the local service area, and the technology infrastructure of the agency. This pre-implementation preparation paved the way for standardization of client experience and internal technical assistance. The use of internal change agents and external supports to promote implementation in systems of care is central to a number of implementation models [
Consideration of client needs was central to implementation and informed adaptations to address subgroup needs (eg, learning disability, literacy, drug court involvement), as well as strategies to support client engagement, such as training all staff about the app so that they would be better prepared to respond to clients’ questions regarding their own experiences with the app. The most successful implementation occurred in agencies that actively created a culture that positioned the mobile phone app as a routine part of relapse prevention care. The community behavioral health agency had the most challenges with implementation in large part due to lack of awareness about the technology resources of its clientele. As the only participating agency that did not focus exclusively on addiction treatment, competing service demands may have made it more difficult to prioritize the mobile phone recovery support app as a central part of care. Assessing technology ownership and use among an agency’s client base would create a foundation from which to plan technology-based service delivery initiatives.
Perceptions of A-CHESS as compatible with client needs were central to stakeholders’ assessment of implementation. Such compatibility was noted across substance use diagnostic categories, that is, alcohol disorders as well as other substance use disorders (eg, drug court). Compatibility, or fit, of interventions with client and organizational needs is a key aspect of many implementation models [
Consistent with other research with provider stakeholders in behavioral health care settings [
The potential of mobile phones for enhancing continuing care was salient across agencies and stakeholders. Access to mobile phones was a powerful motivator to client buy-in to use the app in agencies that offered phones for A-CHESS implementation. Cost was a barrier to sustained ability to provide mobile phones to clients. In agencies that did not provide phones, accessibility to the A-CHESS app was limited to those clients that owned app-compatible mobile phones. Fortunately, mobile phone ownership continues to increase across demographics [
Strategies to promote sustainable client access to evidence-based mobile apps such as A-CHESS are essential. When possible, agencies can seek funding (eg, donations, mini-grants) and leverage state Medicaid billing codes to subsidize hardware and software purchases [
The online peer discussion forum was the most popular feature of A-CHESS for clients, as reported by stakeholders across the 3 agencies that successfully implemented the mobile phone app. This feature empowered clients and fostered active engagement in their recovery process through opportunities to both receive and offer support to others. The online forum created a way for individuals to connect with others and overcame traditional barriers to in-person recovery support groups, including difficulty finding meetings or inability to attend at scheduled times, and perceived stigma or discomfort with in-person meetings.
There were differences between agencies in the level of monitoring and seeding of the forum by the care teams. Stakeholders from the drug court set expectations for forum use and client privacy protection and encouraged client ownership of the forum; stakeholders from the Veterans and addiction treatment settings were more involved in seeding the forum to encourage client engagement. One way to address concerns that A-CHESS communities might get too large is to create multiple groups as volume increases based on obvious delineations within the community-at-large (ie, separate forums for probationers and post-probationers or for younger and older adults). Research is needed with clients in addiction recovery to broaden our understanding of the role and benefits of online support communities, such as those offered in A-CHESS, for recovery outcomes. Key research questions include when and for whom these online support networks are most helpful and how online communities compare with inperson self-help groups in terms of mechanisms of influence on client outcomes (eg, bonding and support, goal direction and structure, promotion of non-drug–using norms, fostering self-efficacy and coping skills) [
Variability in implementation was primarily due to provider attitudes regarding use of technology with clientele and to individual differences in technology literacy of clients who required different levels of training. Although achieving buy-in for service innovations from all clinical staff is unlikely at the outset, agencies can optimize buy-in by carefully selecting internal champions as part of implementation team that will promote use of the mobile phone app by clients. Ongoing sharing of data that supports implementation success among clients can help persuade others about the relative advantages of the technology. The intuitive interface design and ease of use of the A-CHESS app allowed clients to easily learn to use the app with practice.
Agency policies prohibiting client use of mobile phones and more stringent privacy regulations for particular subpopulations (ie, drug court–involved) can introduce barriers to implementation of mobile substance use treatment technologies. In some cases, liabilities related to specific technology features, such as location tracking, may need to be turned off to proceed with use of the tool for other capacities. In many cases, agency policies can also be changed. Thinking systemically, an innovation can be adapted to improve fit during implementation; organizations can also adapt to accommodate the innovation [
The focus of this implementation process study was a single point in time relatively early in implementation. In a separate study of A-CHESS sustainability among consortium members, 3 agencies had sustained use of the recovery support tool at 24 months, 2 of which were agencies studied here (Veterans Substance Use Treatment and Addiction Treatment) [
Study results can guide developers to create mobile applications to optimize implementation success. A-CHESS was developed based on evidence-based relapse prevention practices, and evidence of these elements promoted implementation. To optimize adoption, features should be developed to align with evidence-based practices, target audience needs and characteristics, and organizational workflow. To the degree possible, apps should be developed to be cross-platform and Web-accessible. User-centered design practices that include iterative feedback from target end users during the development process help ensure that the app is easy to use and can promote client engagement. Ongoing evaluation of data collected from technology-based approaches can help agencies monitor client engagement related to their care and implementation progress. The way in which important implementation data from technology-based care approaches are displayed to stakeholders is often underappreciated. Improving visualization of data to make it accessible and meaningful to stakeholders is critical.
This study also highlights directions for future research with regard to implementation frameworks for technology-based care approaches. A challenge to achieving code consensus using the CFIR framework largely reflected overlap of constructs within and between domains as applied to the mobile phone recovery support app (eg, compatibility of the mobile app for clients as a characteristic of the intervention; compatibility with provider practices as characteristic of the organization). The CFIR model also positions implementation as a traditional unidirectional process—an intervention is delivered to clients by providers or clinicians in a given setting. However, in the case of mobile treatment approaches, the mobile phone itself is a context for implementation, as is the sociocultural environment in which mobile apps are used. Implementation frameworks such as the CFIR need to be expanded or adapted to align with the capacities and multidimensional, dynamic nature of technology-based treatment approaches that actively engage clients at various stages of motivation and treatment and expand care beyond clinic walls [
Moving science to service is inherently an active process and the implementers, those people who put an innovation to use, are active recipients of these innovations [
There are several limitations to this implementation process study. First, the study was conducted in 4 settings, the sample size was small and only reflected the perspectives of provider and staff stakeholders, and data were collected at a single point in time. Future efforts to explore implementation would benefit from longitudinal data collection from a larger, more diverse range of settings and stakeholders, including those who decided not to use the technology. Second, although providers and staff provide a valuable lens on implementation from the perspective of the larger client base and organization, clients’ experiences with the app would lend valuable perspective on strategies for implementation and ongoing engagement with the app and should be a focus of future research. Finally, it is possible that the relationships between the consortium participants and the UW-Madison team may have inflated the overall impressions of the tool owing to the collective investment in the consortium itself. This possibility is countered by the fact that 2 of the 4 agencies did not sustain use of A-CHESS as demonstrated in later work. Despite these limitations, the study yielded important findings that may generate further research and aid in practice-based efforts to implement mobile substance use treatment approaches.
Addiction-Comprehensive Health Enhancement Support System
Consolidated Framework for Implementation Research
Comprehensive Health Enhancement Support System Health Education Consortium
institutional review board
This research study was supported by a Center of Excellence grant from the National Institute on Drug Abuse (1P30DA029926-01: Lisa A. Marsch, PhD, Principal Investigator).
AR conducted the qualitative interviews. SM and SL coded and analyzed the transcribed interviews. KJ and SD led the consortium study, facilitated site recruitment for the study described in this paper, and provided valuable feedback on the paper.
Dr. Johnson and Ms. Dinauer are shareholders in CHESS Mobile Health, a company that markets A-CHESS. The remaining authors have no conflicts of interest to declare.