JMIR Publications

JMIR Mental Health

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 23.03.15 in Vol 2, No 1 (2015): Jan-Mar

This paper is in the following e-collection/theme issue:

    Viewpoint

    Clinical Practice Models for the Use of E-Mental Health Resources in Primary Health Care by Health Professionals and Peer Workers: A Conceptual Framework

    1National Institute for Mental Health Research, Research School of Population Health, Australian National University, Canberra, Australia

    2Centre for Addiction and Mental Health, Toronto, ON, Canada

    Corresponding Author:

    Julia Reynolds, BA(Hons), MPsych(Clin)

    National Institute for Mental Health Research

    Research School of Population Health

    Australian National University

    Building 63

    Eggleston Road

    Canberra, 2601

    Australia

    Phone: 61 2 612 ext 57275

    Fax:61 6 612 50733

    Email:


    ABSTRACT

    Background: Research into e-mental health technologies has developed rapidly in the last 15 years. Applications such as Internet-delivered cognitive behavioral therapy interventions have accumulated considerable evidence of efficacy and some evidence of effectiveness. These programs have achieved similar outcomes to face-to-face therapy, while requiring much less clinician time. There is now burgeoning interest in integrating e-mental health resources with the broader mental health delivery system, particularly in primary care. The Australian government has supported the development and deployment of e-mental health resources, including websites that provide information, peer-to-peer support, automated self-help, and guided interventions. An ambitious national project has been commissioned to promote key resources to clinicians, to provide training in their use, and to evaluate the impact of promotion and training upon clinical practice. Previous initiatives have trained clinicians to use a single e-mental health program or a suite of related programs. In contrast, the current initiative will support community-based service providers to access a diverse array of resources developed and provided by many different groups.

    Objective: The objective of this paper was to develop a conceptual framework to support the use of e-mental health resources in routine primary health care. In particular, models of clinical practice are required to guide the use of the resources by diverse service providers and to inform professional training, promotional, and evaluation activities.

    Methods: Information about service providers’ use of e-mental health resources was synthesized from a nonsystematic overview of published literature and the authors’ experience of training primary care service providers.

    Results: Five emerging clinical practice models are proposed: (1) promotion; (2) case management; (3) coaching; (4) symptom-focused treatment; and (5) comprehensive therapy. We also consider the service provider skills required for each model and the ways that e-mental health resources might be used by general practice doctors and nurses, pharmacists, psychologists, social workers, occupational therapists, counselors, and peer workers

    Conclusions: The models proposed in the current paper provide a conceptual framework for policy-makers, researchers and clinicians interested in integrating e-mental health resources into primary care. Research is needed to establish the safety and effectiveness of the models in routine care and the best ways to support their implementation.

    JMIR Mental Health 2015;2(1):e6

    doi:10.2196/mental.4200

    KEYWORDS



    Introduction

    Background

    Over the past 15 years, there has been a rapidly burgeoning interest in the use of electronic communications to deliver psychological therapies. This has been driven by recognition of the high prevalence of mental health problems and the relatively low treatment rates. For example, up to half of those living in English-speaking developed countries will experience a mental disorder in their lifetime [1-3]. Accordingly, most will directly experience a mental disorder or be in a close relationship with a person who does. Unfortunately, only a minority of people with mental health problems receives treatment. In Australia, treatment rates improved following the introduction of subsidized psychological therapies but still remain substantially lower than those achieved for comparable physical conditions [4].

    E-mental health resources offer a potential means for addressing this treatment gap. Of the different resources, online symptom-focused programs have been the most thoroughly researched. These programs have typically drawn on cognitive behavior therapy (CBT) and involve online modules with interactive components such as symptom questionnaires, multimedia content that can be downloaded, and exercises in which service users practice skills. Programs that incorporate some human support and guidance are referred to as iCBT. Others are designed for use as self-guided, fully automated interventions.

    Numerous research trials have shown that Internet-delivered CBT programs can be efficacious and cost effective. A few such programs have accumulated enough evidence to be listed by national review bodies such as the National Institute for Health and Care Excellence (NICE) in the UK and the National Registry of Evidence-based Programs and Practices (NREPP) in the USA. For example, e-mental health programs listed by NREPP include Beating the Blues [5], MoodGYM [6], and the Co-ordinated Anxiety Learning and Management Tools (CALM) program [7].

    Further, a recent systematic review found that for three disorders (depression, social phobia, and panic disorder), the evidence for iCBT was sufficient to meet the American Psychological Association’s criteria for an “established” therapy [8]. iCBT interventions have achieved similar outcomes to individually administered face-to-face CBT while requiring much less therapist time [8,9].

    While the initial research effort has focused on programs that draw on standard forms of CBT, programs based on other types of therapy can also be efficacious. These include Internet-delivered programs drawing on interpersonal therapy [10], psychodynamic therapy [11], and structured writing techniques [12].

    Other technology-based interventions are also accumulating evidence of positive outcomes. Online psycho-education has been shown to improve attitudes to help seeking [13,14] and may be associated with symptom reduction [6]. Information websites can help address the large unmet needs for information, although quality continues to be variable [15,16]. Relapse prevention interventions are also showing promise across service users with diverse diagnostic [17] and/or treatment histories [18].

    Other applications include peer-to-peer support [19], game-based interventions [20], mobile apps [21], online chat [22], and video-conferenced therapy [23].

    Implementation in Routine Care

    Interest is now turning to the question of how e-mental health resources can be incorporated into traditional systems of care. Primary health care systems are of particular interest because they are typically the first point of contact and provide the bulk of mental health care to individuals, at least in developed countries. In Australia for example, the primary health care system is comprehensive and includes general medical practices, allied health, indigenous health workers, pharmacists, and other services [24]. Incorporating e-mental health services into primary health care could improve treatment rates for common mental disorders, especially for people who are unable or unwilling to access traditional services. It may also generate efficiencies and allow clinicians time to provide more intense care to those who require it [25].

    To this end some studies have explored implementation of iCBT in primary health care settings. More research is needed in this area, but positive outcomes have been found for programs designed to treat depression and a range of anxiety disorders [26-28].

    In an innovative initiative, the Australian Government has funded the e-Mental Health in Practice (eMHPrac) service to promote key e-mental health resources among primary health care service providers and to train them to use these resources in routine practice [25,29]. These providers include: general medical practitioners; primary care and mental health nurses; indigenous health workers; psychologists; social workers; occupational therapists; counselors; community health workers; and peer support workers.

    There are two routes by which e-mental health resources can be incorporated into primary health care (cf [30]).

    The best-researched route involves the delivery of iCBT to service users referred to centralized online clinics from community-based service providers. These clinics typically provide some in-house therapist support to service users as they work through programs. Online clinics of this type may receive public health funding to provide free treatment, such as those in Holland and Australia [31,32].

    The current paper will focus on the second route, in which community-based service providers incorporate a broad range of e-mental health resources into their practice. This requires clearly differentiated models of practice that are sufficiently flexible and diverse to be relevant to practitioners operating in different contexts and with varying levels of mental health training. Such models must take into account the perspectives and real-life challenges faced by service providers and users in community-based care settings.

    Although previous work has considered ways that community-based therapists can use e-mental health resources such as iCBT [33-36], there has been little systematic consideration of models by which diverse providers can incorporate a broad range of e-mental health resources into their practice.

    Accordingly, the current paper proposes practice models by which e-mental health resources can be comprehensively incorporated into primary health care systems. The proposed models draw on a nonsystematic overview of published literature and the authors’ experience of training primary care service providers to use e-mental health resources. Our aim in presenting these models is to stimulate discussion, to provide a conceptual framework for future research and to assist clinicians who wish to explore the use of e-mental health resources in their practice.


    Models of Delivery in Primary Health Care

    Broad Approaches

    We identify five broad approaches to the use of e-mental health resources, which require varying levels of service provider knowledge and engagement. The five models are: (1) promotion; (2) case management; (3) coaching; (4) symptom-focused treatment; and (5) comprehensive therapy.

    Table 1 describes these models, detailing the role of the provider, the role of the e-mental health resource and the contact between the service user and provider. Table 2 shows the workers for whom the models are most relevant and the knowledge required to implement each model.

    Table 1. Roles of service providers and e-MH resources in each practice model.
    View this table
    Table 2. Service providers most likely to use each practice model and required knowledge.
    View this table

    Referral Models

    Overview

    In the first three models, the e-mental health resource is the main intervention. The role of the community-based provider is to refer the service user to the resource and to provide varying degrees of support.

    Advantages for service providers using these models include access to free high quality psycho-educational resources and increased referral options. Use of these models may potentially reduce practice costs and increase effectiveness and efficiency.

    1) Promotion
    What is the Promotion Model?

    In this model, information is provided to service users to guide them towards high quality resources. At a minimum, providers can promote credible portal websites, which direct service users to relevant, evidence-based resources [37]. Providers can also become familiar with and promote the resources most relevant to their service - including information sites, symptom-focused treatment programs and online sources of emotional support.

    Which Providers Are Most Likely to Use the Promotion Model?

    Most primary health care providers can promote e-mental health resources. Those in non-clinical roles such as practice managers and administrators can arrange for promotional material to be displayed to service users in waiting rooms or on websites run by the practice. Providers in clinical roles may engage in more active forms of promotion such as briefly describing relevant resources to service users. This may occur in the course of general health consultations as well as in response to service users’ requests for mental health assistance.

    For example, general practice nurses have regular contact with people experiencing chronic physical conditions, to whom they provide information, monitoring and treatment. Given the high comorbidity of these conditions with mental health problems, practice nurses are well placed to promote awareness in this at risk group, to detect changes in life circumstances that may increase risk of mental health problems and to respond to the development of symptoms in these service users [38]. Similarly, service providers such as physiotherapists, exercise physiologists, and dietitians treat conditions that are highly comorbid with mental health concerns and can provide information about e-mental health services as appropriate.

    There is also enormous potential for promotion by those who provide ancillary services to people experiencing mental health concerns. Community pharmacists are highly accessible and have regular contact with people who are using psychotropic medications. Relationships with pharmacists may be particularly important for people who lack continuity of care in their relationships with other professionals. This may be critical where service users obtain repeat medication prescriptions from different doctors in walk-in clinics.

    There have been calls to extend pharmacists’ role in recovery-oriented mental health care [39] and the promotion of e-mental health resources through pharmacies could be one means of facilitating this development. Pharmacists can encourage service users to explore appropriate e-mental health resources in addition to using medication. People interested in self-help interventions may also seek information about supplements such as St John’s Wort. In addition to advice provided directly by the pharmacist, these customers can be encouraged to visit sites that provide evidence-based information about supplements and other interventions.

    What Knowledge is Required to Use the Promotional Model?

    Service providers working in clinical roles will have pre-existing knowledge about mental health, face-to-face care options, and referral processes. These providers would at least need to be familiar with key mental health portals and may also choose to learn about specific resources most relevant to their practice.

    Where workers in non-clinical roles engage in promotion activities, it is expected that the clinicians employing them would oversee the selection of resources and the manner in which they were promoted.

    Current Status of the Promotion Model

    This approach requires little specialist knowledge or change to existing practice. In Australia, there is anecdotal evidence that e-mental health resources are already being promoted by mental health service providers and by general health workers in clinical and non-clinical roles including general practice nurses, practice managers, community health outreach workers, physiotherapists, pharmacists, and osteopaths.

    2) Case Management
    What is the Case Management Model?

    In this model, the practitioner provides an initial assessment, referral to an e-mental health resource and a follow-up assessment. Referrals are most commonly made to iCBT or self-guided symptom-focused programs, but can also be made to e-mental health support groups and other resources.

    Prior to referral, an initial assessment is required to determine the most suitable intervention. E-mental health programs typically use automated screening tools to help guide the service user in their intervention choices. These vary in complexity, but are often focused on the symptoms treated by the particular program. In community-based practice where the service user has approached a service provider for assistance, more comprehensive assessments are required (eg, [40]). These require a relatively high level of skill and service providers engaging in assessment should be appropriately qualified. Assessment will generally occur during face-to-face or possibly video-conferenced consultations [41], although information from e-mental health tools may be incorporated into the assessment.

    Once the service user has engaged with the e-mental health program, the provider is not expected to assist them to work through it. However, the provider is expected to provide additional assessment, support, and alternative referrals if the service user does not find the e-mental health program to be helpful, and/or if their condition or situation deteriorates.

    When used as an intervention in the case management model, the e-mental health resource is the main psychological treatment for at least one area of concern. It may be provided as a stand-alone intervention used in addition to other interventions. For example, a general practitioner might prescribe psychotropic medication and refer the service user to an iCBT program. Similarly, a person experiencing pain and depression might be referred to a physiotherapist as well as to an e-mental health depression treatment program. E-mental health resources can also be combined with human delivered psychological treatments (see Models 4 and 5 in this paper).

    Which Providers Are Most Likely to Use the Case Management Model?

    This approach is particularly suitable for practitioners who provide non-specialist mental health assessment and referral such as general medical practitioners. It is also used by mental health professionals such as those working as injury / rehabilitation consultants and in community outreach services.

    In the case management approach, e-mental health services are considered as referral destinations where appropriate (eg, [40]). This can be particularly useful where specialist assistance is not available in a timely manner or not available at all. For example, we are aware of a public sector mental health team that encourages general medical practitioners to consider this approach for people who have undergone specialist assessment but whose symptoms are not sufficiently acute or severe to meet specialist service admission criteria.

    This approach is useful where service users have been accepted into a service but cannot immediately access face-to-face support or psychological therapy, for example where there are waiting lists (eg, [42]). This is highly relevant to services that have existing triage, internal referral, and case management processes such those catering for a particular demographic group (such as young people) or disorder (such as substance use). Workers who already provide assessment and case management in these systems such as youth workers, substance abuse counselors and community health workers, can incorporate e-mental health resources into their work with some additional training.

    The case management approach can also be used to introduce e-mental health resources as post-treatment booster or relapse prevention interventions. Anecdotal reports from service users indicate that although they may delay returning to face-to-face therapy until symptoms are relatively severe, they may access e-mental health resources when symptoms are still mild.

    What Knowledge is Required to Use the Case Management Model?

    Service providers using this model will already have clinical roles and relevant skills in assessment and referral. To refer to e-mental health resources, they would also need to be able to locate and evaluate relevant resources and have detailed knowledge of specific resources most relevant to their practice area.

    Engaging in active follow-up assessments is critical in this model to enhance safety by ensuring that services are matched to clinical need and by providing access to alternative care as required. It also helps providers to understand how service users in their particular setting respond to e-mental health resources. If outcomes and adherence are disappointing, this should be investigated so that local service users’ preferences can be understood and referral practices can be modified. However, it should be noted that people who do not complete online programs are not necessarily dissatisfied with the program (eg, [43-46]).

    Current Status of the Case Management Model

    This model is currently promoted and supported by a range of online services. For example, Australian online clinics support referrals and case management by offering clinicians’ resources such as an online referral facility [32] and online decision-making tools and guides [45,47,48].

    Some online providers have developed referral aids in the style of prescription or referral pads and distributed them to clinicians so they can direct service users to their particular programs. These aids have been distributed in printed format and in an electronic format that can be incorporated into medical practice software (eg, [47,48]).

    As part of the eMHPrac project in Australia [29], referral aids have been developed to raise practitioner awareness of multiple resources. These aids are currently incorporated into a software application embedded within clinical information software, and it is hoped in the future that they will be incorporated directly into the most frequently used clinical information software packages (Personal communication by J Proudfoot and J Tennant, 2014).

    3) Coaching
    What is the Coaching Model?

    In the coaching model, human support is added to a symptom-focused e-mental health program. The treatment team thus has three components—the service user, the human service provider and the e-mental health program. The online program is the primary intervention and provides the content and structure for the treatment. The coach supports the self-therapeutic activities undertaken by the service user and helps them engage with and complete the program [49].

    Coaching combines the benefits of human support and Internet-delivered content. Online formats can reliably deliver material with high fidelity and may enhance learning and retention of critical material compared to traditionally delivered CBT [50,51]. On the other hand, human coaches may improve the match between service user and intervention through assessment and tailoring; support users to complete the program and enhance users’ understanding and application of program material [41,52]. Coaching may be used to reduce the amount of clinician time required to deliver an intervention, improve fidelity and support non-expert workers to deliver basic interventions.

    Which Providers Are Most Likely to Use the Coaching Model?

    The provision of coaching by community-based providers raises critical questions about qualifications, training, and supervision. Requirements will depend upon the service users involved, the programs used, and the degree of therapeutic skill required by particular coaching protocols.

    In community-based primary health care settings, coaching is most likely to be delivered by providers with an existing clinical role. For example, in Australia, coaching is likely to be delivered by health professionals but in other countries such as the United Kingdom, coaching is likely to be delivered by para-professional workers such as the IAPT’s Psychological Wellbeing Practitioners [53].

    Effectiveness studies of individual programs have demonstrated that, with appropriate training and supervision, a wide range of people can provide coaching. In one study, coaches were drawn from a mixed group of clinicians (general medical practitioners, psychologists, medical specialists, nurses, and other allied health practitioners) [45]. Other studies have included coaches who were: general medical practitioners with additional mental health training [54,55]; primary care nurses and social workers [7]; psychologists working in private practice [44] and in a non-government mental health organization [56]; community-based psychologists and social workers [46]; paraprofessional graduate mental health workers [57]; peer workers [58], and administrative staff [59].

    There is great interest in incorporating the perspectives of people with lived experience of mental disorders into symptom-focused e-mental health programs. In addition to developing direct coaching roles, programs have included service user narratives in the automated content and/or hosted online peer forums within the program [59,60]. We are also aware of projects in which peer workers and advocates have developed their own coaching procedures for freely available Internet-delivered programs, in order to offer services to people living in areas that lack mental health treatment services.

    What Knowledge is Required to Use the Coaching Model?

    Providers already in clinical roles will have mental health knowledge and skills appropriate to their role and are able to provide mental health assessment, support, and referrals. Coaching that requires psychotherapeutic skills is likely to be delivered by providers who have therapy qualifications and already provide psychological services. In addition, they will need sufficient familiarity with e-mental health programs to match them with individual service users and skills relevant to providing coaching for the particular programs they use.

    Coaches will also need to use their existing clinical skills, their understanding of particular programs, and service user preferences to decide whether to offer a coached intervention to an individual service user. Many protocols have excluded people who are psychotic, acutely suicidal, engaging in substance abuse, or have other conditions that may impair learning. Other factors to be considered include service users’ Internet access; comfort and proficiency in using text-based material and computer programs, and their capacity to work independently [61-63].

    Current Status of the Coaching Model
    Overview

    Coaching protocols have been developed for specific programs and suites of related programs. They have largely been applied in relatively controlled settings such as research trials, centralized online clinics and university or hospital-based clinics (eg, [31,52,60,64]).

    Compared to providers in these settings, community-based providers have more diverse qualifications, work in more diverse service settings and provide services to more diverse users. Rather than using a single program or programs from a single provider, they may wish to access a broad range of programs from various online providers for service users with diverse concerns and demographics.

    Accordingly, examples of coaching protocols in the literature will often not, by themselves, be sufficient to guide providers in routine primary care settings. Rather, flexible protocols and clinical guides are required that can be tailored to the requirements of particular service users, the programs most relevant to those service users and the expertise of the service provider.

    Community-based service providers who wish to participate in coaching activities can currently access protocols in two ways. First, some online services actively support community-based providers to coach users through the particular programs offered by that service. Resources available vary across programs but may include coaching guidance or manuals (eg, [47,52,65]). Training may be available for service providers to help them become familiar with the program, its coaching protocol, online communication techniques, and relevant ethical, legal, and technical issues [52,63].

    Some e-mental health resources are explicitly designed to guide both the therapist and the service user to work on the material together during sessions. These do not necessarily reduce the amount of time required of service providers. Instead, they are designed to improve intervention quality and support a wider range of non-expert providers to deliver basic psychological interventions. For example, the Calm Tools for Living program guides the delivery of multi-diagnostic CBT treatment in primary care by health workers such as nurses and social workers [7]. Other examples include the StayStrong iPad app, which is designed for use with Australian Indigenous service users. It supports a wide range of workers to deliver a culturally appropriate structured mental health and substance abuse intervention [66].

    Second, existing protocols (for example, those used in research) can be adapted to suit community-based providers and the service users they work with most frequently. The following draws on the research literature to illustrate the ways in which coaching can be implemented.

    a) The Scope and Intensity of Coaching

    Basic coaching activities include monitoring/ supervision, emotional and technical support, and encouragement to complete the program. Other coaching activities require psychotherapeutic skills, although the online program remains the primary focus. These include tailoring the program in response to individual service users’ progress (for example by controlling the release of new content) and helping them to complete critical tasks such as exposure, which they might otherwise avoid [67-69]. In the CALM program, coaches also demonstrate skills such as controlled breathing [7].

    Further research is needed to identify which coaching activities are critical but some key considerations have been identified.

    Qualitative interview data has underlined the importance of interpersonal relatedness for maintaining engagement with iCBT [70]. With regards to the coach/ service user relationship, the Supportive Accountability model is an overall model for optimizing human support in e-mental health interventions. It was proposed to aid research, but is a useful conceptual aid for practitioners entering this area [71].

    Examples of coaching behaviors are available from analyses of emails sent by coaches to service users. In one study of an online program for Generalised Anxiety Disorder, the most frequently-coded therapist behaviors involved prompting and reinforcing task completion, and real-life application of program material. Reinforcement of task completion was significantly correlated with better outcomes. Poorer outcomes were associated with therapists being more flexible about program deadlines, but the direction of causality was not clear [72].

    Similar themes were found in an analysis of therapist communications in a case study of iCBT for depression. Additional disorder-specific therapist communications were found, such as email content intended to increase hope and normalize the client’s experience [62].

    b) The Amount of Time Required

    Program protocols usually indicate the coaching time required but this may be minimal once the service user has begun working through the program (eg, [59,73]). Indeed, many coaching functions can be at least partially automated, for example reminders to complete tasks, feedback about symptom quizzes, and prompts to seek professional advice [56,59,68,74-76].

    Providers should carefully monitor service users’ satisfaction with the time provided. Users may feel isolated and unmotivated if the coaching is too brief [70,77], although increasing amounts of therapist time may not necessarily improve outcomes [78]. Generally, however, more time will be required for coaching that involves therapeutic activity and coaching time in community settings will need to respond to the needs of individual service users [7,62,70].

    c) Contact Arrangements

    Service users can access online programs from locations of their own choice or from spaces provided by the coach (eg, [7,58]). Most symptom-focused programs require users to register an account. Some can be completed anonymously while others require users to provide personally identifying information.

    In many protocols, contact between coaches and service users is asynchronous and conducted via emails, but may also occur through face-to-face meetings and telephone calls (eg, [44-46, 52, 54-56,64,66,79]). The use of text-based media such as email and chat requires additional training to ensure that the coach is competent in the clinical use of the media as well as secure online spaces in which to communicate with service users [26,46,80]. Coaching protocols usually provide email templates that coaches can personalize for individual service users (eg, [26,63,80]).

    Most protocols specify deadlines for service users to complete tasks and schedules for contact between service users and providers, although service users may initiate additional contact [44-46]. Additional automated or human contact may be initiated at critical times, such as when users have not made contact within a specified time period, if coaches are concerned about users’ engagement with the program or their clinical status and when service users begin or complete key tasks [12,45-47,54,56].

    Most programs provide service users with feedback as they work through the intervention, for example via symptom quizzes and other interactive exercises. Service users can print out this feedback and bring it to sessions or email it to their coach. Some programs provide information about user progress and clinical status directly to coaches [45,47,56,60,61].

    Integrative Models

    Overview

    In the previous three referral models, the e-mental health program is the main intervention and the content and structure of the intervention is determined by the e-mental health resource. Models 4 and 5 are integrative models, where the therapist determines the content and structure of the assessment and overall intervention.

    e-mental health resources can be integrated into any part of the therapy process. During assessment, therapists may incorporate information from online assessments, which are available as stand-alone tools or embedded within information sites or symptom-focused programs [81,82]. These resources can also be used to help improve therapy readiness and engagement. Service users can be encouraged to access particular sites and to talk with the therapist about the sites they have visited. This may help normalize and destigmatize service users’ internal experiences, assist them to reflect upon and describe their internal experiences, help form optimistic and realistic expectations about therapy, and assist the transition into intervention. Anecdotal data indicates that the pre-therapy use of e-mental health resources may be particularly helpful with young people. Once a service user is engaged in treatment, providers may integrate e-mental health resources into their work to maximize the effectiveness of face-to-face sessions.

    Two integrative models are described below. The first relates to integration of e-mental health resources with symptom-focused treatment, and the second refers to complex interventions.

    4) Integrated into Symptom-Focused Treatment
    What is the Symptom-Focused Model?

    In this model, the therapist provides an individualized assessment, formulation, and intervention that incorporates traditional therapeutic activities as well as e-mental health resources.

    The relative contributions of electronic resources and traditional therapy will vary across therapists and service users. E-mental health resources can be used to assist with standard therapeutic tasks such as assessment, psycho-education, self-monitoring, cognitive restructuring and the development of skills such as relaxation and mindfulness. Online peer groups can provide opportunities for behavioral experiments and social skills practice exercises.

    Symptom focused programs can also be incorporated into therapy by presenting material in both electronic and face-to-face formats. In one study, participants received face-to-face CBT therapy in the first half of the sessions and worked independently through an online program in the second half. Outcomes for anxiety, distress, and depression in the combined (face-to-face and online) intervention were as good or better than outcomes in the traditionally delivered (face-to-face only) intervention [83]. The combined intervention thus reduced therapist time and maintained or improved outcomes.

    e-mental health and therapist-delivered material can also be delivered sequentially. For example, a service user could meet with a therapist for a series of face-to-face sessions and then complete a symptom-focused e-program before returning to the therapist for a follow-up assessment (cf [34]).

    There is also potential to develop symptom-focused group therapy interventions using e-mental health programs. This could occur in a similar manner to that reported for a primary-care mood-disorders group based on a computerized (but not Internet-delivered) CBT program [84]. This group was hosted by a general medical practice that provided a meeting room and a computer for each participant. The facilitators set a module for each session and in the first part of the session each participant used their computer to work through the module. After a tea break, facilitators led a group session on the topic of the module that had been completed by the group members.

    Which Providers Are Most Likely to Use the Symptom Focused Model?

    This model is likely to be used by therapists already providing discrete symptom-focused psychological interventions such as manual-based CBT to relieve specific symptoms and disorders. In the Australian primary health care system for example, this model is particularly relevant to providers delivering Focused Psychological Strategies interventions under the Medicare Better Access program [85]. These providers include psychologists, social workers, occupational therapists, and general medical practitioners with relevant therapy training. It would also be relevant to mental health nurses and other mental health workers delivering therapy in other primary care services.

    What Knowledge is Required to Use the Symptom-Focused Model?

    In order to use the symptom-focused model, providers require existing qualifications and experience in providing face-to-face symptom-focused therapies. In addition, they would need to be sufficiently familiar with specific e-mental health resources to choose relevant elements and integrate them into bespoke interventions for individual service users.

    Current Status of the Symptom-Focused Integration Model

    Although there are anecdotal reports of this model’s use and some promising research data, more ecologically valid research is needed to evaluate whether it delivers improved effectiveness and efficiency in routine care.

    5) Integrated Into Comprehensive Therapy
    What is the Comprehensive Therapy Model?

    In this practice model, e-mental health resources are used in a comprehensive intervention designed and structured by the human therapist. The e-mental health resource is one element of a multi-modal approach that uses a range of interventions to treat a presenting problem or simultaneously addresses a number of presenting problems.

    The therapist’s role will include both traditional therapeutic activities and activities related to the service users’ interaction with e-mental health resources. Therapists’ roles in relation to e-mental health will vary from case to case and may resemble activities in described Models 2, 3, or 4. Thus the therapist may encourage the service user to use an information site, online support group or symptom-focused program in a relatively independent manner and monitor the service users’ response to the resource. In other cases, the therapist will be more directly involved in supporting the service user’s interaction with the electronic resource. For example, they may provide some coaching for an online CBT program or integrate the program into a symptom-focused intervention.

    This integrated comprehensive model can be used where multiple therapeutic approaches are required to tackle one presenting problem, particularly where first line interventions have been unsuccessful or where some components of therapy require an emphasis on interpersonal processes occurring in the therapeutic relationship. For example, the early stages of treating complex trauma may require a strong focus on building the therapeutic relationship and engagement. Concurrent use of e-mental health programs at this stage could help lay the foundation for challenging activities such as exposure work. For example, working through a symptom-focused program could help provide some initial symptom relief, improve skills for managing arousal and enhance stability and resilience.

    This approach can also be helpful where there is more than one presenting problem, or where there are comorbidities and secondary disorders that require treatment. Given that the number of funded sessions is likely to be limited, augmenting human delivered therapy with e-mental health resources may reduce the risks associated with incomplete or inadequate treatment of complex issues.

    Which Providers Are Most Likely to Use This Model?

    This model will be most relevant to therapists who already provide comprehensive individualized therapies in their existing practice. In the Australian primary care system for example, this would include clinical psychologists providing psychological therapies under the Medicare Better Access program and other therapists whose work involves tackling complex problems. It would also be relevant to providers whose primary role is to treat very specific concerns and who may need an additional resource to tackle comorbid problems such as anxiety, depression, and substance abuse. For example, in relationship interventions where one or more of the participants is experiencing a high prevalence disorder such as depression, the therapist can focus on interpersonal processes during sessions and refer the symptomatic partner(s) to an online service that treats depression.

    What Knowledge is Required to Use the Comprehensive Model?

    Service providers who are engaging in comprehensive individualized assessment and therapy will already have extensive knowledge of psychological processes and diverse therapies as well as skills in developing multi-modal assessment and treatment plans. Those who provide therapy for specific issues such as relationship problems will have existing skills in screening for comorbid problems that require treatment. In addition, providers need to have familiarity with the e-mental health resources they intend to integrate into their therapies.

    Current Status of the Comprehensive Therapy Model

    At this stage, individual therapists are integrating e-mental health idiosyncratically into comprehensive therapies. Anecdotal evidence indicates that uptake of this model may be aided by its conceptual similarity to the use of face to face delivered CBT-based skills groups traditionally provided alongside individual specialist therapy in secondary care psychiatric services.


    Discussion

    Community-based service providers are developing innovative ways of incorporating e-mental health resources into their practice. The models described here are intended to help researchers, policy-makers, and service providers conceptualize the ways in which e-mental health resources can be incorporated into existing systems of health delivery.

    These practice models can also be integrated into innovative service delivery models that improve the delivery of mental health services in primary health care. For example, stepped care and stratified care models are distinguished from usual care partly by the systematic inclusion of less specialist and lower intensity psychological interventions [86-88]. The promotion, case management and coaching models can be used to deliver psycho-education, support, and symptom-focused interventions in low intensity interventions [89]. The integrative models described here can be incorporated into higher intensity interventions.

    Similarly, collaborative care models for severe depression typically include case management or care coordination roles and systematically support service user self- management. Access to psychological interventions within collaborative care models is associated with better outcome [90]. Clearly, the incorporation of e-mental health resources could support this work, for example through the promotion of high quality online self-help material and case management referrals to iCBT.

    The models described here are also compatible with emerging enhancements to service delivery systems. For example, virtual clinic environments are currently being developed that will offer a range of e-mental health resources within a defined online environment. They will provide a framework within which service providers and users can choose interventions of known quality and may allow the automation of some activities currently undertaken by humans, including aspects of promotion, case management, assessment, and coaching.

    e-mental health resources support self-management and service user expertise and are thus also compatible with emerging recovery-focused, person-centered approaches. One example is the support facilitation approach currently being developed in Australia for people with complex, severe and persistent mental illness [91]. Support facilitation workers assist service users to engage with a range of services and the referral models (promotion, case management, and coaching) described in the current paper are highly relevant to these roles.


    Future Research

    Outcome Research

    The current paper has demonstrated how e-mental health resources can be incorporated into routine care and some of the potential advantages of doing so. However, research into community providers’ use of e-mental health resources is at an early stage and it is not yet possible to determine the optimal ways for providers to interact with the resources. There is an urgent need for research to evaluate the effectiveness, safety, and efficiency of the clinical practice models described in this paper.

    As an initial step, a systematic synthesis of existing literature is needed to identify gaps in the literature and inform clinical practice. The current paper is based on a narrative review in which the most visible body of e-mental health outcome research involved coaching, but a more systematic approach may identify additional bodies of evidence that relate to other practice models.

    As outcome research develops in this area, it will be critical to use research methodologies that optimize ecological validity as well as scientific rigor. These should include practice-based methodologies that balance the need to retain intervention fidelity and the need for flexibility that allows interventions to be adapted for use in routine care settings [92]. It will also be important to actively involve practitioners in developing practice-based procedures to guide implementation and evaluation (eg, [56,93]).

    Research Into Providers’ Use of E-Mental Health Resources

    Anecdotal reports and existing literature suggests that practitioners most commonly interact with e-mental health resources by recommending websites to service users. For example, nearly 90% of an Australian sample of general medical practitioners reported that they recommended websites to service users, and a mental health information site was included amongst the three most frequently recommended sites [94].

    Other studies have investigated the uptake of iCBT amongst service providers and have reported low rates of use [61,95-97]. Providers seem to prefer the idea of adding iCBT to existing clinical practice rather than using it as a replacement. This preference has been reported in various provider groups including CBT therapists [97], youth workers [98], and rural health care practitioners [99].

    Taken together, these studies suggest that community-based practitioners may be most likely to use online resources in ways that are most consistent with the promotion, case management, and integrative models described in the current paper. Unfortunately, as noted above, there is less research on these models than on the coaching model.

    In order to assess the degree to which e-mental health resources are being integrated into health delivery systems, it will also be important to continue measuring uptake by provider groups. Some studies thus far have been limited by small sample sizes and a focus on measuring practitioner attitudes and intentions with less focus on quantifying actual behavior. Furthermore, the lack of clearly defined provider roles and behaviors in the use of e-mental health resources has meant that the meaning of “practitioner use” is at times unclear or variable across studies.

    Future studies can be improved by the use of adequate samples and using the models proposed in the current paper to more clearly conceptualize and define the outcomes of interest.

    Research Into Factors That Influence Provider Behavior

    Successful dissemination will require a good understanding of the factors that can hinder and facilitate provider engagement with online resources.

    Existing surveys on practitioner attitudes to iCBT and have found that they are generally open to the concept of using iCBT, particularly for people with less severe or complex symptoms and where some clinical support is provided (eg, [98-102]). Factors related to technological proficiency and access do not seem to be major obstacles to uptake in general but are perceived to affect the viability of using e-mental health resources with specific clients, therapists, and settings [97,100,101].

    Commonly cited barriers include providers’ lack of accurate knowledge about the resources, lack of training in their use, and concerns about their effectiveness, safety and clinical responsibility [97,98,100-105].

    The Promotion Model requires relatively little change to providers’ usual practice, particularly if they already provide psycho-educational materials such as printed handouts. Practitioners who wish to use the other models described in this paper will need to gain new knowledge, engage more deeply with e-mental health resources, and make some conceptual changes to their practice.

    Obstacles to acquiring relevant knowledge include having the time to review resources and lack of access to training [99]. Even where time and training is available, providers are unlikely to prioritize learning about e-mental health resources unless they perceive it to be beneficial. In the absence of direct financial incentives, potential benefits may include reduced expenses associated with preparing and printing psycho-educational material (promotion model), access to additional referral destinations (case management model), the possibility of achieving more within the same number of sessions (integrative models) and being able to meet the expectations of service users.

    In Australia, evaluation activities associated with the eMHPrac project will contribute to future research. Data will be collected on the ways that providers use e-mental health resources and on the proportion of e-mental health service users who have accessed key websites as result of a recommendation from a provider. Annual surveys and evaluation of eMHPrac provider training interventions will provide data on factors that influence practitioner engagement with and use of online resources.


    Conclusions

    e-mental health resources can be incorporated into the routine practice of diverse service providers and settings in primary health care. The current paper has described five clinical practice models to help researchers, policy-makers, and service providers conceptualize the ways in which e-mental health resources can be incorporated into existing and emerging systems of health delivery. Further research is urgently needed to identify the best procedures for implementing the various practice models, the best means of supporting practitioner uptake and to establish the effectiveness and safety of each practice model when applied in community-based routine care settings.

    Acknowledgments

    JR developed the practice models described in this paper and drafted the paper. KG assisted in developing the models and drafting the paper. JC, KB, and AB assisted in developing the models and editing the paper.  JR, KB, and AB are supported in part by grants from the Australian Government Department of Health including funding from the e-Mental Health in Practice (eMHPrac) project, which provides support for the use of online mental health resources by primary care clinicians. KG is supported by the Australian National Health and Medical Research Council (NHMRC) Research Fellowship 1059620.

    Conflicts of Interest

    The authors are responsible for delivering ANU’s e-hub Mental Health suite of online programs. These programs are promoted along with other services by the eMHPrac project.

    References

    1. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005 Jun;62(6):593-602. [CrossRef] [Medline]
    2. Slade T, Johnston A, Teesson M, Whiteford H, Burgess P, Pirkis J, et al. The mental health of Australians 2: report on the 2007 National Survey of Mental Health and Wellbeing. Canberra: Department of Health and Ageing; 2009 May.   URL: http://www.health.gov.au/internet/main/publishing.nsf/Content/mental-pubs-m-mhaust2 [accessed 2015-01-02] [WebCite Cache]
    3. Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980-2013. Int J Epidemiol 2014 Apr;43(2):476-493. [CrossRef] [Medline]
    4. Roxon N, Macklin J, Butler M. National mental health reform - ministerial statement. 2011.   URL: http://www.budget.gov.au/2011-12/content/ministerial_statements/health/download/ms_health.pdf [accessed 2014-12-31] [WebCite Cache]
    5. Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med 2003 Feb;33(2):217-227. [Medline]
    6. Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by using the internet: randomised controlled trial. BMJ 2004 Jan 31;328(7434):265 [FREE Full text] [CrossRef] [Medline]
    7. Craske MG, Rose RD, Lang A, Welch SS, Campbell-Sills L, Sullivan G, et al. Computer-assisted delivery of cognitive behavioral therapy for anxiety disorders in primary-care settings. Depress Anxiety 2009 Mar;26(3):235-242 [FREE Full text] [CrossRef] [Medline]
    8. Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res 2012 Dec;12(6):745-764. [CrossRef] [Medline]
    9. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
    10. Donker T, Bennett K, Bennett A, Mackinnon A, van SA, Cuijpers P, et al. Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: randomized controlled noninferiority trial. J Med Internet Res 2013;15(5):e82 [FREE Full text] [CrossRef] [Medline]
    11. Johansson R, Björklund M, Hornborg C, Karlsson S, Hesser H, Ljótsson B, et al. Affect-focused psychodynamic psychotherapy for depression and anxiety through the Internet: a randomized controlled trial. PeerJ 2013;1:e102 [FREE Full text] [CrossRef] [Medline]
    12. Lange A, Rietdijk D, Hudcovicova M, van de Ven J, Schrieken B, Emmelkamp PM. Interapy: a controlled randomized trial of the standardized treatment of posttraumatic stress through the internet. J Consult Clin Psychol 2003 Oct;71(5):901-909. [CrossRef] [Medline]
    13. Kravitz RL, Franks P, Feldman MD, Tancredi DJ, Slee CA, Epstein RM, et al. Patient engagement programs for recognition and initial treatment of depression in primary care: a randomized trial. JAMA 2013 Nov 6;310(17):1818-1828. [CrossRef] [Medline]
    14. Taylor-Rodgers E, Batterham PJ. Evaluation of an online psychoeducation intervention to promote mental health help seeking attitudes and intentions among young adults: randomised controlled trial. J Affect Disord 2014 Oct 15;168:65-71. [CrossRef] [Medline]
    15. Griffiths KM, Crisp DA. Unmet depression information needs in the community. J Affect Disord 2013 Apr 25;146(3):348-354. [CrossRef] [Medline]
    16. Grohol JM, Slimowicz J, Granda R. The quality of mental health information commonly searched for on the Internet. Cyberpsychol Behav Soc Netw 2014 Apr;17(4):216-221. [CrossRef] [Medline]
    17. Bauer S, Wolf M, Haug S, Kordy H. The effectiveness of internet chat groups in relapse prevention after inpatient psychotherapy. Psychother Res 2011 Mar;21(2):219-226. [CrossRef] [Medline]
    18. Holländare F, Johnsson S, Randestad M, Tillfors M, Carlbring P, Andersson G, et al. Randomized trial of Internet-based relapse prevention for partially remitted depression. Acta Psychiatr Scand 2011 Oct;124(4):285-294. [CrossRef] [Medline]
    19. Griffiths KM, Calear AL, Banfield M. Systematic review on Internet Support Groups (ISGs) and depression (1): Do ISGs reduce depressive symptoms? J Med Internet Res 2009;11(3):e40 [FREE Full text] [CrossRef] [Medline]
    20. Li J, Theng YL, Foo S. Game-based digital interventions for depression therapy: a systematic review and meta-analysis. Cyberpsychol Behav Soc Netw 2014 Aug;17(8):519-527. [CrossRef] [Medline]
    21. Donker T, Petrie K, Proudfoot J, Clarke J, Birch MR, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013;15(11):e247 [FREE Full text] [CrossRef] [Medline]
    22. Dowling M, Rickwood D. Online Counseling and Therapy for Mental Health Problems: A Systematic Review of Individual Synchronous Interventions Using Chat. Journal of Technology in Human Services 2013 Jan;31(1):1-21. [CrossRef]
    23. Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: a 2013 review. Telemed J E Health 2013 Jun;19(6):444-454 [FREE Full text] [CrossRef] [Medline]
    24. Australian Government Department of Health and Ageing. Canberra: Commonwealth of Australia; 2009. Primary health care reform in Australia: Report to support Australia’s first national primary health care strategy   URL: http://apo.org.au/files/Resource/nphc-supp.pdf [accessed 2014-12-31] [WebCite Cache]
    25. Australian Government Department of Health and Ageing 2012. e-Mental health strategy for Australia   URL: http:/​/www.​health.gov.au/​internet/​main/​publishing.nsf/​Content/​7C7B0BFEB985D0EBCA257BF0001BB0A6/​$File/​emstrat.​pdf [accessed 2014-12-31] [WebCite Cache]
    26. Andersson G, Hedman E. Effectiveness of guided internet-based cognitive behavior therapy in regular clinical settings. Verhaltenstherapie 2013;23(3):140-148. [CrossRef]
    27. Høifødt RS, Strøm C, Kolstrup N, Eisemann M, Waterloo K. Effectiveness of cognitive behavioural therapy in primary health care: a review. Fam Pract 2011 Oct;28(5):489-504 [FREE Full text] [CrossRef] [Medline]
    28. Twomey C, O'Reilly G, Byrne M. Effectiveness of cognitive behavioural therapy for anxiety and depression in primary care: a meta-analysis. Fam Pract 2015 Feb;32(1):3-15. [CrossRef] [Medline]
    29. Queensland University of Technology 2014. e-Mental Health in Practice (eMHPrac) project website   URL: http://www.emhprac.org.au/ [accessed 2014-12-31] [WebCite Cache]
    30. Williams AD, O'Moore K, Mason E, Andrews G. The effectiveness of internet cognitive behaviour therapy (iCBT) for social anxiety disorder across two routine practice pathways. Internet Interv 2014 Oct;1(4):225-229. [CrossRef]
    31. Ruwaard J, Lange A, Schrieken B, Dolan CV, Emmelkamp P. The effectiveness of online cognitive behavioral treatment in routine clinical practice. PLoS One 2012;7(7):e40089 [FREE Full text] [CrossRef] [Medline]
    32. Macquarie University 2014. Mindspot Clinic   URL: http://www.mindspot.org.au/ [accessed 2014-12-31] [WebCite Cache]
    33. Eells TD, Barrett MS, Wright JH, Thase M. Computer-assisted cognitive-behavior therapy for depression. Psychotherapy (Chic) 2014 Jun;51(2):191-197. [CrossRef] [Medline]
    34. Klein B. e-Interventions and psychology: time to log on. InPsych: Bulletin of the Australian Psychological Society 2010 Feb [FREE Full text]
    35. Kavanagh D. Six reasons to integrate e-health into psychology practice. InPsych: Bulletin of the Australian Psychological Society 2014 [FREE Full text]
    36. Kyrios M, Thomas N. Psychology and the internet: Where are we and where to from here? InPsych: Bulletin of the Australian Psychological Society 2014 [FREE Full text]
    37. Reynolds J. Online mental health resources: What's available and do they work? InPsych: Bulletin of the Australian Psychological Society 2014 Jun [FREE Full text]
    38. Crosland A, Kai J. 'They think they can talk to nurses': practice nurses' views of their roles in caring for mental health problems. Br J Gen Pract 1998 Jul;48(432):1383-1386 [FREE Full text] [Medline]
    39. Mey A, Fowler JL, Knox K, Shum D H K, Fejzic J, Hattingh L, et al. Review of community pharmacy staff educational needs for supporting mental health consumers and carers. Community Ment Health J 2014 Jan;50(1):59-67. [CrossRef] [Medline]
    40. Usherwood T, Harris A. Mood disorders in general practice. Med Today 2014;155(5):25-30.
    41. Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry 2014 Feb;13(1):4-11 [FREE Full text] [CrossRef] [Medline]
    42. Kenter R, Warmerdam L, Brouwer-Dudokdewit C, Cuijpers P, van SA. Guided online treatment in routine mental health care: an observational study on uptake, drop-out and effects. BMC Psychiatry 2013 Jan;13:43 [FREE Full text] [CrossRef] [Medline]
    43. Christensen H, Mackinnon A. The law of attrition revisited. J Med Internet Res 2006;8(3):e20; author reply e21 [FREE Full text] [CrossRef] [Medline]
    44. Jacmon J, Malouff J, Taylor N. Treatment of major depression: effectiveness of cognitive behavior therapy with an internet course as a central component. E J Appl Psychol 2009;5(2):1-8.
    45. Newby JM, Mackenzie A, Williams AD, McIntyre K, Watts S, Wong N, et al. Internet cognitive behavioural therapy for mixed anxiety and depression: a randomized controlled trial and evidence of effectiveness in primary care. Psychol Med 2013 Dec;43(12):2635-2648. [CrossRef] [Medline]
    46. Hadjistavropoulos HD, Pugh NE, Nugent MM, Hesser H, Andersson G, Ivanov M, et al. Therapist-assisted Internet-delivered cognitive behavior therapy for depression and anxiety: translating evidence into clinical practice. J Anxiety Disord 2014 Dec;28(8):884-893 [FREE Full text] [CrossRef] [Medline]
    47. St Vincent's Hospital. 2013. ThisWayUp Clinic clinician's resources   URL: https://thiswayup.org.au/clinicians/resources/ [accessed 2015-01-01] [WebCite Cache]
    48. Australian National University. 2014. e-hub assist: resources for referring someone   URL: http://www.ehub.anu.edu.au/assist/resources/referral/ [accessed 2014-12-31] [WebCite Cache]
    49. Lillevoll KR, Wilhelmsen M, Kolstrup N, Høifødt RS, Waterloo K, Eisemann M, et al. Patients' experiences of helpfulness in guided internet-based treatment for depression: qualitative study of integrated therapeutic dimensions. J Med Internet Res 2013;15(6):e126 [FREE Full text] [CrossRef] [Medline]
    50. Wagner B, Horn AB, Maercker A. Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. J Affect Disord 2014 Jan;152-154:113-121. [CrossRef] [Medline]
    51. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, et al. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry 2005 Jun;162(6):1158-1164. [CrossRef] [Medline]
    52. Abbott JM, Klein B, Ciechomski L. Best Practices in Online Therapy. Journal of Technology in Human Services 2008 Jul 03;26(2-4):360-375. [CrossRef]
    53. Richardson G, Richards D. Psychological wellbeing practitioners: best practice guide. 2010.   URL: http://www.iapt.nhs.uk/silo/files/psychological-wellbeing-practitioners--best-practice-guide.pdf [WebCite Cache]
    54. Shandley K, Austin DW, Klein B, Pier C, Schattner P, Pierce D, et al. Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists? J Med Internet Res 2008;10(2):e14 [FREE Full text] [CrossRef] [Medline]
    55. Hickie IB, Davenport TA, Luscombe GM, Moore M, Griffiths KM, Christensen H. Practitioner-supported delivery of internet-based cognitive behaviour therapy: evaluation of the feasibility of conducting a cluster randomised trial. Med J Aust 2010 Jun 7;192(11 Suppl):S31-S35. [Medline]
    56. Kirkpatrick T, Manoukian L, Dear BF, Johnston L, Titov N. A feasibility open trial of internet-delivered cognitive-behavioural therapy (iCBT) among consumers of a non-governmental mental health organisation with anxiety. PeerJ 2013;1:e210 [FREE Full text] [CrossRef] [Medline]
    57. Farrand P, Confue P, Byng R, Shaw S. Guided self-help supported by paraprofessional mental health workers: an uncontrolled before--after cohort study. Health Soc Care Community 2009 Feb;17(1):9-17. [CrossRef] [Medline]
    58. Cavanagh K, Seccombe N, Lidbetter N. The implementation of computerized cognitive behavioural therapies in a service user-led, third sector self help clinic. Behav Cogn Psychother 2011 Jul;39(4):427-442. [CrossRef] [Medline]
    59. Andrews G, Titov N. Is internet treatment for depressive and anxiety disorders ready for prime time? Med J Aust 2010 Jun 7;192(11 Suppl):S45-S47. [Medline]
    60. Hedman E, Ljótsson B, Rück C, Bergström J, Andersson G, Kaldo V, et al. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr Scand 2013 Dec;128(6):457-467. [CrossRef] [Medline]
    61. Andrews G, Williams AD. Up-scaling clinician assisted internet cognitive behavioural therapy (iCBT) for depression: A model for dissemination into primary care. Clin Psychol Rev 2014 Jun 11. [CrossRef] [Medline]
    62. Pugh NE, Hadjistavropoulos HD, Klein B, Austin DW. A case study illustrating therapist-assisted internet cognitive behavior therapy for depression. Cogn Behav Pract 2014;21(1):64-77. [CrossRef]
    63. Hadjistavropoulos HD, Thompson MJ, Klein B, Austin DW. Dissemination of therapist-assisted internet cognitive behaviour therapy: development and open pilot study of a workshop. Cogn Behav Ther 2012;41(3):230-240. [CrossRef] [Medline]
    64. Sharry J, Davidson R, McLoughlin O, Doherty G. A service-based evaluation of a therapist-supported online cognitive behavioral therapy program for depression. J Med Internet Res 2013;15(6):e121 [FREE Full text] [CrossRef] [Medline]
    65. Christensen H, Griffiths K, Groves C. MoodGYM training program: Clinican's manual. Centre for Mental Health Research, Canberra: Australian National Univeristy; 2004.
    66. Menzies School of Health Research. Stay Strong App   URL: http:/​/www.​menzies.edu.au/​page/​Research/​Projects/​Mental_Health_and_wellbeing/​Development_of_the_Stay_Strong_iPad_App/​ [accessed 2015-01-03] [WebCite Cache]
    67. Gega L, Smith J, Reynolds S. Cognitive behaviour therapy (CBT) for depression by computer vs. therapist: patient experiences and therapeutic processes. Psychother Res 2013;23(2):218-231. [CrossRef] [Medline]
    68. Spence SH, Donovan CL, March S, Gamble A, Anderson RE, Prosser S, et al. A randomized controlled trial of online versus clinic-based CBT for adolescent anxiety. J Consult Clin Psychol 2011 Oct;79(5):629-642. [CrossRef] [Medline]
    69. Stott R, Wild J, Grey N, Liness S, Warnock-Parkes E, Commins S, et al. Internet-delivered cognitive therapy for social anxiety disorder: a development pilot series. Behav Cogn Psychother 2013 Jul;41(4):383-397 [FREE Full text] [CrossRef] [Medline]
    70. Wilhelmsen M, Lillevoll K, Risør MB, Høifødt R, Johansen ML, Waterloo K, et al. Motivation to persist with internet-based cognitive behavioural treatment using blended care: a qualitative study. BMC Psychiatry 2013;13:296 [FREE Full text] [CrossRef] [Medline]
    71. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011;13(1):e30 [FREE Full text] [CrossRef] [Medline]
    72. Paxling B, Lundgren S, Norman A, Almlöv J, Carlbring P, Cuijpers P, et al. Therapist behaviours in internet-delivered cognitive behaviour therapy: analyses of e-mail correspondence in the treatment of generalized anxiety disorder. Behav Cogn Psychother 2013 May;41(3):280-289. [CrossRef] [Medline]
    73. Twomey CBM, O'Reilly G. Innovative ways to help increase access to CBT. Forum 2013;30(10):40-41.
    74. Bennett K, Reynolds J, Christensen H, Griffiths KM. e-hub: an online self-help mental health service in the community. Med J Aust 2010 Jun 7;192(11 Suppl):S48-S52. [Medline]
    75. Proudfoot J, Clarke J, Birch MR, Whitton AE, Parker G, Manicavasagar V, et al. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry 2013;13:312 [FREE Full text] [CrossRef] [Medline]
    76. Purves DG, Bennett M, Wellman N. An open trial in the NHS of Blues Begone: a new home based computerized CBT program. Behav Cogn Psychother 2009 Oct;37(5):541-551. [CrossRef] [Medline]
    77. Titov N. Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry 2007 Feb;41(2):95-114. [CrossRef] [Medline]
    78. Klein B, Austin D, Pier C, Kiropoulos L, Shandley K, Mitchell J, et al. Internet-based treatment for panic disorder: does frequency of therapist contact make a difference? Cogn Behav Ther 2009;38(2):100-113. [CrossRef] [Medline]
    79. Farrer L, Christensen H, Griffiths KM, Mackinnon A. Internet-based CBT for depression with and without telephone tracking in a national helpline: randomised controlled trial. PLoS One 2011;6(11):e28099 [FREE Full text] [CrossRef] [Medline]
    80. Bennett-Levy J, Richards D, Farrand P, Christensen H, Griffiths K, Kavanagh D, et al. Email in low intensity CBT interventions. In: Oxford guide to low intensity CBT interventions. Oxford: Oxford University Press; 2010.
    81. Wade AG. Use of the internet to assist in the treatment of depression and anxiety: a systematic review. Prim Care Companion J Clin Psychiatry 2010 Aug;12(4) [FREE Full text] [CrossRef] [Medline]
    82. Al-Asadi AM, Klein B, Meyer D. Comorbidity structure of psychological disorders in the online e-PASS data as predictors of psychosocial adjustment measures: psychological distress, adequate social support, self-confidence, quality of life, and suicidal ideation. J Med Internet Res 2014 Oct;16(10):e248 [FREE Full text] [CrossRef] [Medline]
    83. Sethi S, Campbell AJ, Ellis LA. The Use of Computerized Self-Help Packages to Treat Adolescent Depression and Anxiety. Journal of Technology in Human Services 2010 Aug 31;28(3):144-160. [CrossRef]
    84. Mitchell J, Howell C, Turnbull D, Murphy M. Computer-assisted group therapy for the treatment of depression and anxiety in general practice. Primary Care Mental Health 2005;3(1):27-40.
    85. Australian Government Department of Health. 2011. Allied Mental Health Services   URL: http://www.health.gov.au/internet/main/publishing.nsf/Content/health-pcd-programs-amhpm-pdf-QAs [accessed 2014-12-31] [WebCite Cache]
    86. Bower P, Gilbody S. Managing common mental health disorders in primary care: conceptual models and evidence base. BMJ 2005 Apr 9;330(7495):839-842 [FREE Full text] [CrossRef] [Medline]
    87. Franx G, Oud M, de LJ, Wensing M, Grol R. Implementing a stepped-care approach in primary care: results of a qualitative study. Implement Sci 2012;7:8 [FREE Full text] [CrossRef] [Medline]
    88. van SA, Seekles W, van't Veer-Tazelaar NJ, Beekman AT, Cuijpers P. Stepped care for depression in primary care: what should be offered and how? Med J Aust 2010 Jun 7;192(11 Suppl):S36-S39. [Medline]
    89. Green KE, Iverson KM. Computerized cognitive–behavioral therapy in a stepped care model of treatment. Prof Psychol Res Pr 2009;40(1):103. [CrossRef]
    90. Coventry PA, Hudson JL, Kontopantelis E, Archer J, Richards DA, Gilbody S, et al. Characteristics of effective collaborative care for treatment of depression: a systematic review and meta-regression of 74 randomised controlled trials. PLoS One 2014;9(9):e108114 [FREE Full text] [CrossRef] [Medline]
    91. Brophy L, Hodges C, Halloran K, Grigg M, Swift M. Impact of care coordination on Australia's mental health service delivery system. Aust Health Rev 2014 Sep;38(4):396-400. [CrossRef] [Medline]
    92. Cohen DJ, Crabtree BF, Etz RS, Balasubramanian BA, Donahue KE, Leviton LC, et al. Fidelity versus flexibility: translating evidence-based research into practice. Am J Prev Med 2008 Nov;35(5 Suppl):S381-S389. [CrossRef] [Medline]
    93. Kay-Lambkin FJ, Simpson AL, Bowman J, Childs S. Dissemination of a computer-based psychological treatment in a drug and alcohol clinical service: an observational study. Addict Sci Clin Pract 2014;9:15 [FREE Full text] [CrossRef] [Medline]
    94. Usher WT. Australian health professionals' health website recommendation trends. Health Promot J Austr 2011 Aug;22(2):134-141. [Medline]
    95. Hermens M L, Muntingh A, Franx G, van Splunteren P T, Nuyen J. Stepped care for depression is easy to recommend, but harder to implement: results of an explorative study within primary care in the Netherlands. BMC Fam Pract 2014;15:5 [FREE Full text] [CrossRef] [Medline]
    96. MacLeod M, Martinez R, Williams C. Cognitive Behaviour Therapy self-help: who does it help and what are its drawbacks? Behav Cogn Psychother 2009 Jan;37(1):61-72. [CrossRef] [Medline]
    97. Whitfield G, Williams C. If the evidence is so good – why doesn’t anyone use them? A national survey of the use of computerized cognitive behaviour therapy. Behav Cogn Psychother 2004;32(57):65. [CrossRef]
    98. Fleming T, Merry S. Youth work service providers' attitudes towards computerized CBT for adolescents. Behav Cogn Psychother 2013 May;41(3):265-279. [CrossRef] [Medline]
    99. Sinclair C, Holloway K, Riley G, Auret K. Online mental health resources in rural Australia: clinician perceptions of acceptability. J Med Internet Res 2013;15(9):e193 [FREE Full text] [CrossRef] [Medline]
    100. Becker EM, Jensen-Doss A. Computer-assisted therapies: examination of therapist-level barriers to their use. Behav Ther 2013 Dec;44(4):614-624. [CrossRef] [Medline]
    101. Stallard P, Richardson T, Velleman S. Clinicians' attitudes towards the use of computerized cognitive behaviour therapy (cCBT) with children and adolescents. Behav Cogn Psychother 2010 Oct;38(5):545-560. [CrossRef] [Medline]
    102. Vigerland S, Ljótsson B Bergdahl Gustafsson F, Hagert S, Thulina I, Andersson G. Attitudes towards the use of computerized cognitive behavior therapy (cCBT) with children and adolescents: A survey among Swedish mental health professionals. Internet Interv; , 111–117 2014 Jul [FREE Full text] [CrossRef]
    103. Carper MM, McHugh RK, Barlow DH. The dissemination of computer-based psychological treatment: a preliminary analysis of patient and clinician perceptions. Adm Policy Ment Health 2013 Mar;40(2):87-95. [CrossRef] [Medline]
    104. Christensen H, Reynolds J, Griffiths KM. The use of e-health applications for anxiety and depression in young people: challenges and solutions. Early Interv Psychiatry 2011 Feb;5 Suppl 1:58-62. [CrossRef] [Medline]
    105. Gun SY, Titov N, Andrews G. Acceptability of Internet treatment of anxiety and depression. Australas Psychiatry 2011 Jun;19(3):259-264. [CrossRef] [Medline]

    Edited by G Eysenbach; submitted 05.01.15; peer-reviewed by H Hadjistavropoulos, C Williams; comments to author 16.01.15; revised version received 10.02.15; accepted 12.02.15; published 23.03.15

    ©Julia Reynolds, Kathleen M Griffiths, John A Cunningham, Kylie Bennett, Anthony Bennett. Originally published in JMIR Mental Health (http://mental.jmir.org), 23.03.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.