This paper is in the following e-collection/theme issue:

Innovations in Mental Health Systems (398)

Published on in Vol 5, No 2 (2018): Apr-Jun

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/9600, first published .
Implementing Internet-Based Self-Care Programs in Primary Care: Qualitative Analysis of Determinants of Practice for Patients and Providers

Implementing Internet-Based Self-Care Programs in Primary Care: Qualitative Analysis of Determinants of Practice for Patients and Providers

Implementing Internet-Based Self-Care Programs in Primary Care: Qualitative Analysis of Determinants of Practice for Patients and Providers

Authors of this article:

Eric Hermes1, 2 Author Orcid Image ;   Laura Burrone1 Author Orcid Image ;   Elliottnell Perez1 Author Orcid Image ;   Steve Martino1, 2 Author Orcid Image ;   Michael Rowe2 Author Orcid Image
Article Authors Cited by (7) Tweetations (15) Metrics

    Original Paper

    1VA Connecticut Healthcare System, Veterans Health Administration, West Haven, CT, United States

    2Department of Psychiatry, Yale University School of Medicine, New Haven, CT, United States

    Corresponding Author:

    Eric Hermes, MD

    VA Connecticut Healthcare System

    Veterans Health Administration

    950 Campbell Ave

    West Haven, CT, 06106

    United States

    Phone: 1 2039325711 ext 7422

    Email: eric.hermes@yale.edu


    Background: Access to evidence-based interventions for common mental health conditions is limited due to geographic distance, scheduling, stigma, and provider availability. Internet-based self-care programs may mitigate these barriers. However, little is known about internet-based self-care program implementation in US health care systems.

    Objective: The objective of this study was to identify determinants of practice for internet-based self-care program use in primary care by eliciting provider and administrator perspectives on internet-based self-care program implementation.

    Methods: The objective was explored through qualitative analysis of semistructured interviews with primary care providers and administrators from the Veterans Health Administration. Participants were identified using a reputation-based snowball design. Interviews focused on identifying determinants of practice for the use of internet-based self-care programs at the point of care in Veterans Health Administration primary care. Qualitative analysis of transcripts was performed using thematic coding.

    Results: A total of 20 physicians, psychologists, social workers, and nurses participated in interviews. Among this group, internet-based self-care program use was relatively low, but support for the platform was assessed as relatively high. Themes were organized into determinants active at patient and provider levels. Perceived patient-level determinants included literacy, age, internet access, patient expectations, internet-based self-care program fit with patient experiences, interest and motivation, and face-to-face human contact. Perceived provider-level determinants included familiarity with internet-based self-care programs, changes to traditional care delivery, face-to-face human contact, competing demands, and age.

    Conclusions: This exploration of perspectives on internet-based self-care program implementation among Veterans Health Administration providers and administrators revealed key determinants of practice, which can be used to develop comprehensive strategies for the implementation of internet-based self-care programs in primary care settings.

    JMIR Ment Health 2018;5(2):e42

    doi:10.2196/mental.9600

    Keywords

    cognitive behavioral therapy; internet-based therapy; health information technology; behavioral intervention technology; internet; Veterans 


    Background

    Internet-based self-care programs (ISPs) belong to a category of health information technology, which consist of personalized, self-guided interventions delivered electronically to improve knowledge, awareness, or behavior change for mental or behavioral health conditions [1]. ISPs present interactive therapeutic material in rich formats such as audio, video, and text. Participants use ISPs at a pace and in a setting of their choosing, while being provided varying levels of support [2,3]. Evidence-based ISPs have shown efficacy for the treatment of many common conditions including depression, anxiety, substance use, and insomnia [4,5].

    The potential health service benefits of ISPs are pronounced, as they may mitigate chronic and systemic problems associated with access to mental health services [6]. Access to services is a challenge even for integrated health care systems such as Veterans Health Administration (VHA), which are accountable to provide a coordinated range of services to a defined population [7,8]. One solution may lie in the implementation of ISPs, the benefits of which include reduced travel barriers, improved access to evidence-based therapies, and reduced stigma [9]. ISPs may also increase self-care and add to health care system productivity [10,11]. Within the VHA, ISPs may especially benefit patients where travel time and distance to VHA facilities or the lack of trained providers are limiting factors [12].

    Despite the advantages of ISPs and evidence of general acceptability, ISP use in US health care systems, including the VHA, is not well documented and presumed low [13,14]. Although ISP use in the United Kingdom, Europe, and Australia is more routine and to some extent institutionalized in policy, there have been notable difficulties with ISP implementation at the point of care with patient and providers in the primary care settings [15-17]. For instance, a recent major pragmatic trial implementing ISPs for depression in primary care sites across the United Kingdom resulted in relatively low use of the programs by patients [18].

    The lack of ISP use in US health care settings and the relative difficulty of implementation in other countries suggest the need for comprehensive strategies to address ISP implementation. Such strategies are dependent on the determinants of practice for a given setting—defined as the human and system factors, which determine to what extent and manner interventions are used. Determinants of practice have been referred to elsewhere by alternative terms such as barriers, obstacles, enablers, and facilitators [19]. For instance, in a study implementing evidence-based practices for the treatment of chronic diseases in primary care, the use of checklists by providers and self-management among patients were identified as important determinants of practice [20]. However, unlike other interventions, determinants of practice for ISPs rely on the acceptance and use of both technology and self-care by patients, providers, staff, and administrators [21].

    Objective

    The identification of determinants of practice for ISP use will be a key step in the development of effective strategies for ISP implementation [22]. The objective of this study was to identify determinants of practice for ISP implementation perceived by VHA primary care and integrated mental health care providers and administrators. The identification of determinants will aid in the development of comprehensive strategies for ISP implementation.


    Study Design

    VHA primary care provider and administrator perspectives on ISP implementation were explored through qualitative analysis of semistructured interviews. The institutional review board of the VA Connecticut Healthcare System preapproved this study.

    Sample

    To identify determinants encountered in care, participants were primary care providers, primary care mental health providers, clinic/facility administrators, and VHA system administrators who were known to be using or have used ISPs in patient care, interested in using ISP in primary care settings, or otherwise knowledgeable on the subject of ISP use. VHA does not systematically track or record patient or provider use of ISPs, and as yet, there is no validated way to search VHA electronic health records for evidence of use. Therefore, a provider sampling strategy based on in-depth investigator knowledge of the VHA system was used. Investigators identified an initial group of VHA key informants in ISP use. This group was expanded by asking interviewees for referrals to additional key informants. Recruitment was through e-mail contact that included a project description. Physicians, psychologists, nurse practitioners, nurse administrators, and social workers with both clinical and administrative responsibilities were recruited. A total of 20 interviews were conducted; 100% of those approached agreed to participate.

    Data Gathering

    The principal investigator (EH) conducted semistructured interviews using a guide that first defined ISPs, giving examples of VHA-developed ISPs and other programs if needed, such as Moving Forward, PTSD Coach, and Sleep Healthy Using the Internet [23-25]. Determinants were then explored using 9 core question areas, with subquestions to explore specific topics that emerged. Question areas were derived from constructs (performance expectancy, effort expectancy, social influence, and facilitating conditions) from the Unified Theory of Acceptance and Use of Technology, a theory that describes factors associated with technology use [21]. Interviews were audio recorded and conducted over the phone, lasting between 45 and 90 min.

    Data Analysis

    Interviews were transcribed verbatim. Thematic analysis, a foundational approach to the analysis of in-depth interviews, was used to analyze interview content through a 5-step group process: (1) familiarization with transcribed data; (2) generation of initial codes; (3) collating codes into themes; (4) reviewing, discussing, and modifying themes in relation to interview extracts; and (5) defining and developing a typology of themes [26]. Codes were first generated by individual investigators, and then discussed as a group, referencing the interview content to develop a final set of codes. Discrepancies in coding were addressed through group discussions led by the principal investigator (EH). Similarly, themes were then developed from codes using a group discussion process. Codes within themes were reviewed using the entire dataset to identify examples and counterfactuals. Authors EH, LB, and EP participated in steps 1 through 3, whereas all authors participated in steps 4 and 5. Themes were organized according to socio-ecological levels within the integrated health care system: patients, providers, and the organization [27]. This manuscript describes patient- and provider-level themes associated with determinants of practice for ISP use at the point of care. Determinants active at the VHA organizational level will be described in future work. Data were managed and analyzed using Atlas/ti. version 4.2 software (ATLAS.ti Scientific Software Development GmbH).


    Interviewee Characteristics

    Of the 20 individuals participating, more males than females were interviewed, while the training, background, and primary work locations of interviewees were relatively balanced (Table 1). Interviews began with questions concerning interviewees’ and their colleagues’ experience with using ISPs in clinical practice. ISP use among interviewees was relatively low. Although 16 (80%, 16/20) interviewees had used an ISP or discussed ISPs with patients at some point in practice, only 6 (30%, 6/20) were currently using them. One primary care provider stated:

    I have [used ISPs], but not typically, this isn’t something that’s kind of a normal or typical thing for me to do.

    Despite this level of use, all 20 interviewees (100%, 20/20) made statements in support of ISP use in primary care. One primary care provider stated:

    I’m all for it. I think that it’s a great modality to utilize especially as we are getting…a lot more of the younger guys…who are much more computer tech savvy.

    Perceived Determinants Associated With Patients

    Attitude Toward Treatment

    A total of 10 interviewees (50%, 10/20) identified patient interest in, motivation for, or willingness to engage in ISPs or mental health treatment in general as a determinant of ISP use. Interest in ISPs was generally described as high, but variable. One primary care provider with administrative responsibilities said most patients are using the internet for health information already:

    A lot of them…do a lot of exploration of their problems and try things out.

    Interviewees also discussed how patients must adjust to the idea of self-care and are variably motivated to engage in self-care. Consequently, providers assess patient interest and motivation before recommending ISP use. One provider in primary care mental health said:

    I meet with the patient and I pull up the course and we go through some of it and I get a sense of whether or not they have the interest in continuing to do it on their own.

    Another said:

    Motivation is a big one for everything. The big barrier is if they’re not really motivated to do whatever, then they’re likely not going to do it.
    Table 1. Characteristics of interviewees.
    CharacteristicInterviewees (N=20), n (%)
    Gender

    		Male13 (65)

    		Female7 (35)
    Training

    		Doctor of Philosophy9 (45)

    		Medical Doctor6 (30)

    		Registered Nurse3 (15)

    		Social work2 (10)
    Primary work location

    		Primary care8 (40)

    		Primary care mental health8 (40)

    		ISP implementation researcha4 (20)
    Primary dutyb

    		Patient care12 (60)

    		Administration8 (40)

    aInterviewees with direct experience with VA patients and providers in VA primary care and primary care mental health settings as part of ISP research.

    bAll interviewees had both clinical and administrative duties. This category identifies their primary duty activity.

    Literacy

    Low technology and reading literacy were discussed in 8 interviews (40%, 8/20), especially in older Veterans. As a clinic administrator stated:

    I think that [the use of] technology-based care for sure is going to be based upon the age of the individual and their experience with technology.

    Two other interviewees, who were primary care providers, said reading literacy was a barrier, as most programs require participants to read and understand written language on screens. However, as a counterfactual, one mental health provider noted that oversimplifying ISP content might alienate literate individuals:

    Any time you are trying to get accessible language that most people understand, you risk inadvertently alienating folks who feel they are being talked down to.
    Access to the Internet

    In 6 interviews (30%, 6/20), patients’ lack of reliable home or community Wi-Fi or hard-wired internet connections were discussed. Where such connections were available, internet speeds may be too slow to support ISP use. Lack of access also included limited bandwidth on cellular phone connections. Interviewees explained that reduced access was influenced by age, personal finance, and locations where cable, cellular, or public networks were not available. One primary care mental health provider said:

    For certain members of the older population, lack of consistent, decent access to the internet [is a problem]. I’ve got some guys who are homeless.
    Internet-Based Self-Care Program Compatibility With Patient Experience

    A total of 5 interviewees (25%, 5/20) identified compatibility between patients’ experience of their mental health condition and ISP program content as a determinant of ISP use. Many ISPs deliver versions of manualized therapies, utilizing structured and sequenced formats to teach content and skills for common issues. However, ISPs may not be able to address unique patient experiences. Consequently, some patients may find that ISPs do not align with their specific mental health complaints. A primary care mental health provider shared the following issue:

    Sometimes [the patient’s reaction is] like, “Hey, what I’m experiencing doesn’t really fit to this framework.”...I think that sometimes we risk alienating people who feel like we are not appealing to people at their level.
    Patient Expectations

    A total of 4 interviewees (20%, 4/20) discussed unrealistic expectations among patients about the time and effort required for ISP engagement, the application of content to their lives, and symptom improvement. One provider with administrator responsibilities said:

    The expectation when you’re on line or using your phone is that things happen quickly. Learning how to relax still takes time. Actually being able to practice and incorporate the techniques still takes time.

    Perceived Determinants Associated With Providers

    Familiarity With Internet-Based Self-Care Programs

    A total of 18 interviewees (90%, 18/20) identified providers’ familiarity with ISPs as a factor influencing use in clinical practice. Familiarity with ISPs ranges from awareness of available programs to regular and judicious use by providers of effective programs for the right patients. Moreover, 10 interviewees stated that they or other providers had limited awareness of any ISPs. One provider with VHA-wide administrative responsibilities stated:

    I don’t even think that local staff know what internet-based self-help options out there are.

    In addition, 3 interviewees stated that they were aware of programs but were not familiar with their content or had not tried them. One administrator believed that providers must know about all aspects of ISPs to refer patients to them:

    As we move forward, the skill of front-line clinical providers is going to be just as much to know the potential benefits, risks, side-affects, value, [and] impact of digital health tools, as they currently know about pharmaceuticals.

    For another primary care provider, improving the “findability” of ISPs was key, described as the ability to access the right ISP at the right time for a specific patient.

    Changes to the Traditional Delivery of Care

    A total of 11 interviewees (55%, 11/20) discussed how ISPs could improve current models of care delivery by increasing self-care, providing more patient-generated data, and affording different modes of communication. Interviewees indicated that some providers are reluctant for patients to take part in self-care or treatment outside the traditional provider-patient dyad. One provider who was also an ISP researcher stated:

    Some clinicians are reluctant to allow things to happen inside the black box…if the clinician isn’t entirely comfortable with what the rules are, what’s happening inside the app…they’re not going to use it.

    Providers were also concerned about how a patient’s clinical status would be monitored and how workload credit for a provider’s time to support ISP use would be tracked. Other interviewees discussed the shift to using patient-generated health data in encounters:

    There is a culture shift that needs to happen with providers to help them recognize that patient generated health data has equal value to healthcare system-generated data.

    Others focused on possible changes to provider-patient communication, using secure messaging as an example:

    Secure messaging impacted their practice. The fear from the provider side was now the Veterans could have an unfiltered access to me and I’m going to spend all day filtering through these e-mails.
    Competing Demands and Lack of Time

    A total of 8 interviewees (40%, 8/20), chiefly primary care and primary care mental health providers, said that competing demands and a lack of time during patient appointments were barriers to ISP use. One said:

    It’s just one of many, many things you’re trying to do in the clinic.

    Another one said:

    I think there would be concern about extra time, like one more thing. More and more keeps getting added on.

    Still another provider stated:

    Clinicians are not necessarily going to have time to sit there with the patient and show ’em how the thing works, and kinda guide them through it.

    Perceived Determinants Associated With Patients and Providers

    Human Contact

    A total of 10 interviewees (50%, 10/20) discussed human contact as critical for ISP use in primary care settings. One provider who was also an ISP researcher said:

    My overall impression is that these are materials that cannot be used without some sort of provider support…I don’t think that sending people to the course or having people download the app is sufficient for those to have any kind of clinical utility.

    Interviewees made several points regarding the need to introduce ISPs to patients, elicit interest, assess willingness to engage, and match patients to specific programs. Interviewees also highlighted human contact to support patient progress. One mental health provider said:

    I routinely ask, when they come in, “Hey, were you able to use that? What was your experience?” If they didn’t, I try to find out why.

    Four interviewees with a range of duties made suggestions about individuals, other than providers, who could support ISP use as a clinical intermediary, such as peer specialists, nurses, care managers, and masters-level clinicians.

    Age

    Age was discussed as a moderator of ISP use for patients in 6 interviews (30%, 6/20) and providers in 3 interviews (15%, 3/20). For patients, interviewees stated that age influences access to the internet, technology literacy, and preference for face-to-face care. One clinic administrator referenced the relatively low use of VHA’s MyHealtheVet website by older Veterans [28]:

    I welcome those programs [but] I think it’s challenging. A lot of our Veterans, it’s such an older population, I don’t see a majority of them turning to the internet. I know some will. For example, MyHealtheVet, we thought we’d get a higher population of enrollment for that.

    Regarding provider age, interviewees said that the use of technology other than phones for communicating with patients could be a barrier for older providers.


    Study Objective

    The aim of this study was to explore primary care provider and administrator perspectives on ISP implementation to identify important perceived determinants of practice for ISP use. We identified 10 themes for determinants of practice at both patient and provider levels. An understanding of these determinants is vital to the development of strategies for ISP implementation in primary care. Comprehensive implementation strategies for ISPs have yet to be clearly defined or rigorously tested. Our investigation of determinants suggests functions for such an ISP implementation strategy, which we feel can be performed by a clinical intermediary (Table 2).

    Internet-Based Self-Care Program Use in Practice

    There is little information on rates of ISP use among US providers or within US health care institutions, and we presume this use to be low. Thus, the relatively infrequent use of ISPs reported by this group of interviewees is not too surprising. Nonetheless, interviewees were generally supportive of ISP use, in spite of identifying a number of determinants of practice that can be considered barriers to ISP use. The support for ISPs identified in this study is consistent with the general acceptance of and support for health information technology among providers and patients [13,29]. Support for ISPs relative to actual use may suggest an emerging tipping point for the increased use of ISPs in the United States, as has occurred in other countries. For example, a 2008 survey of mental health providers in the United Kingdom revealed that 99% used ISPs in care and 39% had been trained to use them [30]. At a minimum, our findings indicate that provider and patient attitudes toward ISPs are not strong barriers to implementation.

    Patient-Level Determinants

    Perceived patient-level determinants of ISP use including literacy, internet access, and age are well documented, as in a recent analysis of interviews with VHA service users who stopped using an ISP for insomnia before program completion [31-34]. However, conflicting evidence suggests that internet use and the acceptability of ISPs among VHA service users are on par with those of the general population [35,36]. Given these differences and the relative difficulty of modifying the above determinants, compared with other patient-level determinants identified here and in similar studies (ie, interest/motivation, patient expectations, and ISP compatibility with patients), other determinants may be more amenable to mitigation and suitable for incorporation into strategies used to implement ISPs (see discussion below and Table 2) [37].

    Table 2. Description of how a clinical intermediary can function in an internet-based self-care program implementation strategy to address key determinants of practice.
    Determinant of practiceAction of clinical intermediary
    Determinants associated with providers

    		Provider familiarityProvide recurring education to providers on ISPa availability and content

    		Changes to the traditional delivery of careProvide supported self-care by mediating communication between the patient and provider regarding clinical questions about the program and progress in the program

    		Time/competing demandsRelieve provider workload by sharing responsibility with the provider for case identification, patient education, and program support
    Determinants associated with patients

    		Patient technology literacyPerform basic education on program access and navigation such as website location and login. Provide human contact for technical assistance

    		Internet accessIdentify community internet access points. Develop and maintain clinical internet access points within the health care facility

    		Interest/motivationSupplement the provider in assessing interest in and motivation to engage in behavioral treatment through ISPs using motivational interviewing

    		Patient expectationsSupplement provider education about ISPs to ensure realistic expectations on the effort required and time course of symptom improvement

    		Patient experience of ISP fitProvide insight on how program content may fit with an individual’s experience and assist providers in identifying individuals who are failing ISPs or require a higher level of care
    Determinants associated with both patients and providers

    		Human contactMeet patients individually or in groups to provide face-to-face support

    aISP: internet-based self-care program.

    Provider-Level Determinants

    Two themes at the provider level—familiarity with available programs and change to the traditional delivery of care—stand out as key in this study and the few others that have evaluated provider perspectives on ISP use [38,39]. ISP familiarity as a determinant of practice suggests that educating providers about available ISPs and having them experience the platform should be a component of a comprehensive ISP implementation approach. Educational strategies to implement ISPs in health care settings must be flexible to account for varying levels of baseline ISP familiarity among providers. The likely downstream effects of increased provider familiarity with ISPs are the changes more ISP use could bring to the traditional delivery of care [40]. Providers must become more comfortable recommending not only technology use but also self-care to patients. Similarly, a system of stepped care must be in place so that patients who are not appropriate for or fail initial attempts at ISP treatment are identified and referred to other forms of care (eg, more provider interactions) [16].

    A Strategy for Internet-Based Self-Care Program Implementation

    The determinants of practice identified here and those from other studies of ISP use outside of primary care, mental health, and VHA settings suggest the need for comprehensive strategies for ISP implementation [31,38,39]. Given the current level of support for ISPs identified among providers in this study, the function of such a strategy should not rely solely on educating providers about ISPs to gain support. Rather, key strategy functions should be more nuanced and include increasing provider familiarity with specific programs, addressing competing demands, providing internet access, and modulating patient expectations, among others (Table 2). Also identified among these determinants is the need for human contact during ISP use. Such contact has already shown evidence for improving adoption and outcomes of ISPs. Support from a human may help address many of the other ISP implementation strategy functions identified here [41,42]. We propose that a clinical intermediary, an individual different from the primary provider, can deliver this human contact. As suggested by interviewees in this study, such a clinical role could be performed by peer specialists, nurses, care managers, and other clinicians trained to facilitate ISP use by engaging in the important implementation strategy functions described in Table 2. Future research should test the use of a clinical intermediary and identify the content, timing, and other specifics of these strategy functions as well as the training needed for this role. In addition, the materials needed to train a clinical intermediary could be spread health system-wide as part of future ISP dissemination efforts.

    Study Limitations

    This study has several limitations. Interviews involved a relatively small group of individuals, and the frequency of ISP use within this group was low despite our objective to recruit ISP users. As such, these findings may represent only an initial indication of the important perceived determinants of practice for ISP use, and may be strengthened by a larger sample size or the inclusion of more VHA providers who regularly use ISP in care if they exist. In addition, the purposeful inclusion of individuals who have chosen not to use ISPs in care may reveal yet more critical determinants of practice. Moreover, it must also be noted that determinants were developed from the perspectives of VHA providers and administrators. We did not verify these determinants using quantifiable means. However, interviewees had a breadth of relevant backgrounds and adequately represented ISP stakeholders within VHA. They, in turn, discussed a range of determinants of practice that influence ISP use. In addition, interviews with health care consumers may reveal yet more determinants. Organizational-level determinants of practice that might suggest enterprise-wide dissemination and implementation strategies or policy changes were not discussed but will be examined in future work. Finally, the VHA is a unique, integrated, and nationwide health care system that provides care for US Veterans. Thus, our results may not be representative of providers or patients in other health care systems.

    Conclusions

    Developing and testing implementation strategies to facilitate the use of ISPs in primary care is a critical area in need of investigation. As an initial step, this study identified patient- and provider-level determinants of practice for the use of ISPs in the context of outpatient primary care settings in the VHA. An ISP implementation strategy that addresses the determinants identified here, including the use of a clinical intermediary to support both patients and providers, may be an effective way forward. These findings, as well as those from similar studies, can be used to inform the development of ISP implementation strategies, which can be further described and tested in future work.

    Acknowledgments

    The research outlined here was supported by the Department of Veterans affairs, Veterans Health Affairs, HSR&D Career Development Award (grant number 119234740 to EH), as well as the VA New England Mental Illness Research, Education, and Clinical Center and the VA Pain, Research, Informatics, Medical comorbidities, and Education Center. The funding sources had no role in the design, analysis, or interpretation of data or in the preparation of the report or decision to publish. All authors have reviewed and approved this manuscript for publication. The authors acknowledge and are grateful for the guidance of Mona Ritche, MSW, PhD, in the use of the Atlas/ti. qualitative data analysis and management software. Interviews were transcribed by the VA HSR&D Centralized Transcription Services Program located at the VA Salt Lake City Healthcare System.

    Conflicts of Interest

    None declared.

    1. Marks I, Cavanagh K, Gega L. Hands-on Help: Computer-aided Psychotherapy. UK. Psychology Press; 2007:1-267.
    2. Barak A, Klein B, Proudfoot JG. Defining internet-supported therapeutic interventions. Ann Behav Med. Aug 2009;38(1):4-17. [CrossRef] [Medline]
    3. Schueller SM, Tomasino KN, Mohr DC. Integrating human support into behavioral intervention technologies: the efficiency model of support. Clin Psychol Sci Pract. Mar 2017;24(1):27-45. [CrossRef]
    4. Rogers MA, Lemmen K, Kramer R, Mann J, Chopra V. Internet-delivered health interventions that work: systematic review of meta-analyses and evaluation of website availability. J Med Internet Res. Mar 24, 2017;19(3):e90. [FREE Full text] [CrossRef] [Medline]
    5. Sander L, Rausch L, Baumeister H. Effectiveness of internet-based interventions for the prevention of mental disorders: a systematic review and meta-analysis. JMIR Ment Health. Aug 2016;3(3):e38. [FREE Full text] [CrossRef] [Medline]
    6. Costello EJ, He JP, Sampson NA, Kessler RC, Merikangas KR. Services for adolescents with psychiatric disorders: 12-month data from the National Comorbidity Survey-Adolescent. Psychiatr Serv. Mar 01, 2014;65(3):359-366. [FREE Full text] [CrossRef] [Medline]
    7. US Congress. 2014. URL: https://www.congress.gov/bill/113th-congress/house-bill/3230 [accessed 2018-05-02] [WebCite Cache]
    8. Shortell SM, Gillies RR, Anderson DA, Erickson KM, Mitchell JB. Remaking health care in America. Hosp Health Netw. Mar 20, 1996;70(6):43-4, 46, 48. [Medline]
    9. Cucciare MA, Weingardt KR, Humphreys K. How Internet technology can improve the quality of care for substance use disorders. Curr Drug Abuse Rev. 2009;2(3):256-262. [CrossRef] [Medline]
    10. Lieberman DZ, Huang SW. A technological approach to reaching a hidden population of problem drinkers. Psychiatr Serv. Mar 2008;59(3):297-303. [CrossRef] [Medline]
    11. Warmerdam L, Smit F, Van Straten A, Riper H, Cuijpers P. Cost-utility and cost-effectiveness of internet-based treatment for adults with depressive symptoms: randomized trial. J Med Internet Res. 2010;12(5):e53. [FREE Full text] [CrossRef] [Medline]
    12. West A, Weeks WB. Physical and mental health and access to care among nonmetropolitan veterans health administration patients younger than 65 years. J Rural Health. 2006;22(1):9-16. [CrossRef] [Medline]
    13. Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med. Nov 2008;38(11):1521-1530. [CrossRef] [Medline]
    14. Beaulac J, Vincent N, Walsh K. Dissemination of an Internet-based treatment for chronic insomnia into primary care. Behav Sleep Med. 2014;13(2):124-139. [CrossRef] [Medline]
    15. Titov N, Dear BF, Staples LG, Bennett-Levy J, Klein B, Rapee RM, et al. MindSpot clinic: an accessible, efficient, and effective online treatment service for anxiety and depression. Psychiatr Serv. Oct 2015;66(10):1043-1050. [CrossRef] [Medline]
    16. National Institute for Health and Care Excellence. URL: https://www.nice.org.uk/guidance/cg90 [accessed 2018-05-02] [WebCite Cache]
    17. Topooco N, Riper H, Araya R, Berking M, Brunn M, Chevreul K, et al. Science Direct. 2017. URL: https://www.sciencedirect.com/science/article/pii/S2214782916300446 [accessed 2018-05-02] [WebCite Cache]
    18. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. Br Med J. 2015;351:h5627. [FREE Full text] [Medline]
    19. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. Apr 29, 2015:CD005470. [CrossRef] [Medline]
    20. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. Jan 2011;38(1):4-23. [FREE Full text] [CrossRef] [Medline]
    21. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Q. 2003;27(3). [CrossRef]
    22. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. Apr 2012;69(2):123-157. [FREE Full text] [CrossRef] [Medline]
    23. Veteran Training. URL: https://www.veterantraining.va.gov/apps/movingforward/index.html [accessed 2018-02-14] [WebCite Cache]
    24. US Department of Veteran Affairs. PTSD: National Center for PTSD. URL: https://www.ptsd.va.gov/public/materials/apps/PTSDCoach.asp [accessed 2018-02-14] [WebCite Cache]
    25. BeHealth Solutions, LLC. SHUTi. URL: http://www.myshuti.com/ [accessed 2018-02-14] [WebCite Cache]
    26. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101. [CrossRef]
    27. Brownson RC, Colditz GA, Proctor EK. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford. Oxford University Press; 2017.
    28. Department of Veterans Affairs. MyHealtheVet. URL: https://www.myhealth.va.gov/ [accessed 2017-12-07] [WebCite Cache]
    29. Kivi M, Eriksson M, Hange D, Petersson EL, Björkelund C, Johansson B. Experiences and attitudes of primary care therapists in the implementation and use of internet-based treatment in Swedish primary care settings. Internet Interv. 2015;2(3):248-256. [FREE Full text] [CrossRef]
    30. MacLeod M, Martinez R, Williams C. Cognitive behaviour therapy self-help: who does it help and what are its drawbacks? Behav Cogn Psychother. 2009;37(1):61-72. [CrossRef] [Medline]
    31. Hermes ED, Merrel J, Clayton A, Morris C, Rowe M. Computer-based self-help therapy: a qualitative analysis of attrition. Health Informatics J. 2016;1460458216683536. [CrossRef] [Medline]
    32. McInnes DK, Gifford AL, Kazis LE, Wagner TH. Disparities in health-related internet use by US veterans: results from a national survey. Inform Prim Care. 2010;18(1):59-68. [Medline]
    33. Houston TK, Volkman JE, Feng H, Nazi KM, Shimada SL, Fox S. Veteran internet use and engagement with health information online. Mil Med. Apr 2013;178(4):394-400. [CrossRef] [Medline]
    34. Alvarado MM, Kum HC, Gonzalez Coronado K, Foster MJ, Ortega P, Lawley MA. Barriers to remote health interventions for type 2 diabetes: a systematic review and proposed classification scheme. J Med Internet Res. Feb 2017;19(2):e28. [CrossRef] [Medline]
    35. Hermes ED, Tsai J, Rosenheck R. Technology use and interest in computerized psychotherapy: a survey of veterans in treatment for substance use disorders. Telemed J E Health. Sep 2015;21(9):721-728. [CrossRef] [Medline]
    36. Tsai J, Rosenheck RA. Use of the internet and an online personal health record system by US veterans: comparison of veterans affairs mental health service users and other veterans nationally. J Am Med Inform Assoc. 2012;19(6):1089-1094. [FREE Full text] [CrossRef] [Medline]
    37. Compen FR, Bisseling EM, Schellekens MP, Jansen ET, Van der Lee ML, Speckens AE. Mindfulness-based cognitive therapy for cancer patients delivered via Internet: qualitative study of patient and therapist barriers and facilitators. J Med Internet Res. 2017;19(12):e407. [FREE Full text] [CrossRef] [Medline]
    38. Ariens LF, Schussler-Raymakers FM, Frima C, Flinterman A, Hamminga E, Arents BW, et al. Barriers and facilitators to eHealth use in daily practice: perspectives of patients and professionals in dermatology. J Med Internet Res. Sep 05, 2017;19(9):e300. [FREE Full text] [CrossRef] [Medline]
    39. Hwang KO, Stuckey HL, Chen MC, Kraschnewski JL, Forjuoh SN, Poger JM, et al. Primary care providers' perspectives on online weight-loss programs: a big wish list. J Med Internet Res. Jan 19, 2012;14(1):e16. [FREE Full text] [CrossRef] [Medline]
    40. Tang PC, Smith MD. Democratization of health care. J Am Med Assoc. Oct 25, 2016;316(16):1663-1664. [CrossRef] [Medline]
    41. Drozd F, Vaskinn L, Bergsund HB, Haga SM, Slinning K, Bjørkli CA. The implementation of Internet interventions for depression: a scoping review. J Med Internet Res. Sep 08, 2016;18(9):e236. [FREE Full text] [CrossRef] [Medline]
    42. Andersson G. Using the Internet to provide cognitive behaviour therapy. Behav Res Ther. Mar 2009;47(3):175-180. [CrossRef] [Medline]

    ‎
    ISP: internet-based self-care program
    VHA: Veterans Health Administration

    Edited by P Batterham; submitted 07.12.17; peer-reviewed by M Subotic-Kerry, K Nazi, S Shimada; comments to author 15.01.18; revised version received 14.02.18; accepted 13.03.18; published 18.05.18.

    Copyright

    ©Eric Hermes, Laura Burrone, Elliottnell Perez, Steve Martino, Michael Rowe. Originally published in JMIR Mental Health (http://mental.jmir.org), 18.05.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.