Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 18.08.20 in Vol 7, No 8 (2020): August

Preprints (earlier versions) of this paper are available at, first published Jul 16, 2020.

This paper is in the following e-collection/theme issue:

    Original Paper

    Barriers and Facilitators for Referrals of Primary Care Patients to Blended Internet-Based Psychotherapy for Depression: Mixed Methods Study of General Practitioners’ Views

    1Department of Clinical Psychology and Psychotherapy, University of Erlangen-Nürnberg, Erlangen, Germany

    2Faculty of Behavioral and Movement Sciences, Section of Clinical Psychology, VU University Amsterdam, Amsterdam, Netherlands

    3Department of Research and Innovation, GGZinGeest, Amsterdam, Netherlands

    Corresponding Author:

    Ingrid Titzler, MSc

    Department of Clinical Psychology and Psychotherapy

    University of Erlangen-Nürnberg

    Nägelsbachstraße 25a

    Erlangen, 91052


    Phone: 49 91318567567



    Background: Major depressive disorder (MDD) is highly prevalent and often managed by general practitioners (GPs). GPs mostly prescribe medication and show low referral rates to psychotherapy. Many patients remain untreated. Blended psychotherapy (bPT) combines internet-based interventions with face-to-face psychotherapy and could increase treatment access and availability. Effectively implementing bPT in routine care requires an understanding of professional users’ perspectives and behavior.

    Objective: This study aims to identify barriers and facilitators perceived by GPs in referring patients to bPT. Explanations for variations in referral rates were examined.

    Methods: Semistructured interviews were conducted with 12 of 110 GPs participating in a German randomized controlled trial (RCT) to investigate barriers to and facilitators for referrals to bPT for MDD (10 web-based modules, app-based assessments, and 6 face-to-face sessions). The interview guide was based on the theoretical domains framework. The interviews were audio recorded and transcribed verbatim, and the qualitative content was analyzed by 2 independent coders (intercoder agreement, k=0.71). A follow-up survey with 12 interviewed GPs enabled the validation of emergent themes. The differences in the barriers and facilitators identified between groups with different characteristics (eg, GPs with high or low referral rates) were described. Correlations between referrals and characteristics, self-rated competences, and experiences managing depression of the RCT-GPs (n=76) were conducted.

    Results: GPs referred few patients to bPT, although varied in their referral rates, and interviewees referred more than twice as many patients as RCT-GPs (interview-GPs: mean 6.34, SD 9.42; RCT-GPs: mean 2.65, SD 3.92). A negative correlation was found between GPs’ referrals and their self-rated pharmacotherapeutic competence, r(73)=−0.31, P<.001. The qualitative findings revealed a total of 19 barriers (B) and 29 facilitators (F), at the levels of GP (B=4 and F=11), patient (B=11 and F=9), GP practice (B=1 and F=3), and sociopolitical circumstances (B=3 and F=6). Key barriers stated by all interviewed GPs included “little knowledge about internet-based interventions” and “patients’ lack of familiarity with technology/internet/media” (number of statements, each k=22). Key facilitators were “perceived patient suitability, e.g. well-educated, young” (k=22) and “no conflict with GP’s role” (k=16). The follow-up survey showed a very high agreement rate of at least 75% for 71% (34/48) of the identified themes. Descriptive findings indicated differences between GPs with low and high referral rates in terms of which and how many barriers (low: mean 9.75, SD 1.83; high: mean 10.50, SD 2.38) and facilitators (low: mean 18.25, SD 4.13; high: mean 21.00; SD 3.92) they mentioned.

    Conclusions: This study provides insights into factors influencing GPs’ referrals to bPT as gatekeepers to depression care. Barriers and facilitators should be considered when designing implementation strategies to enhance referral rates. The findings should be interpreted with care because of the small and self-selected sample and low response rates.

    JMIR Ment Health 2020;7(8):e18642





    Approximately 350 million people worldwide are affected by major depressive disorder (MDD) annually [1]. It is the most common mental health disorder, with an estimated lifetime prevalence of 12.8% [2]. Prevalence rates of MDD in primary care patients range from 10% to 14% [3-6].

    General practitioners (GPs) are considered most important in the management of depression because they are often the first point of contact in the pathways to care [7,8]. Accordingly, it is estimated that 60% to 71% of patients with MDD are treated by their GP [5,9-11]. However, a recent study [12] showed that 60% of patients with MDD under primary care did not receive guideline-oriented treatment; 54% were treated by their GPs with medication only (31%), counseling (45%), referral to specialized care (21%), and psychotherapy (10%). In general, referral rates of GPs to specialized care for depression vary between 16% and 58% [3,13-16]. In Germany, psychotherapy referral rates are low (8%-33%) [13,17,18], and only 17.3% of patients with mild or moderate depression and 3.6% with severe depression are treated with psychological psychotherapy in a 3-year period [10].

    There are several reasons for not adhering to clinical guidelines in primary care. Factors that may hinder GPs in adequately treating depression include low rates (47%) of correct recognition and diagnosis of depression [19], insufficient time for care provision [20], self-reported insufficient knowledge relating to diagnosis or treatment and low skills [16,20], and higher self-confidence in providing medication [4,16]. Studies have also identified factors influencing GPs’ referrals to mental health services [4,15,16,20-22]. These include characteristics of the disease (impairment, severity and duration of symptoms, and need for specialized treatment), patient (age, gender, and presentation of psychological complaints), GP (confidence in their abilities, insufficient time and insufficient skills to provide care, and need for clarification of diagnosis), and care system (lack of access to specialists, long distance or waiting time, personal communication, and medical exchange).

    Some of the abovementioned barriers may be overcome by providing digital technologies as an additional treatment option. Digital approaches may help to improve access to specialists, decrease long waiting times, and reduce the high number of untreated patients [23,24]. Internet- and mobile-based interventions (IMIs) have been shown to be effective in treating MDD [25] and to yield results comparable with face-to-face psychological short-term treatments, when delivered as guided internet-based treatment [26].

    Despite its potential, the uptake of evidence-based internet-delivered treatment remains limited [27,28], and not all patients are willing to use a digital stand-alone treatment [29,30]. For patients who are open to using technology-based treatments but also value personal contact, blended psychotherapy (bPT) might be a promising approach that combines IMIs with face-to-face treatment. This is a relatively new research field, and only a few studies have investigated the acceptance and effectiveness of blended treatments [31]. Previous studies indicate a potentially high acceptance and willingness to use bPT by both patients [32,33] and therapists [34-36]. Furthermore, randomized controlled trials (RCTs) on blended treatments indicated that the reduction of face-to-face psychotherapy sessions with equivalent replacement by IMIs did not lead to an inferior outcome when compared with traditional therapy [37-39]. Thus, implementing bPT could be an attractive alternative with the potential to reduce therapist time and increase the number of treated patients.

    Given GPs’ central role as gatekeepers in the management of depression, research on their perspectives as gatekeepers seems crucial. A recent study on the implementation of internet-based therapy services in routine care, in guided, unguided, or blended formats, suggested that GPs’ low referral rates and unstable patient intake could be explained by GP skepticism and reservations and called for research on referrers’ attitudes [40]. A study on Australian routine practice not only indicated a high GP satisfaction with web-based referrals and treatment services but also highlighted challenges in encouraging GPs’ uptake [41]. Studies on GPs’ attitudes toward depression IMIs identified benefits for implementation, such as a facilitated diagnosis, reduced workload, individualized treatment, shorter waiting time, and support for more patients [42,43]. Furthermore, GPs reported barriers to implementing IMIs, such as infrastructure requirements, privacy concerns, low awareness of electronic mental health (e–mental health) interventions, low confidence in prescribing IMIs, lack of training, uncertainty about the evidence base [42], and concerns that computer treatments are too impersonal and would not meet patient needs [44].


    The abovementioned insights refer to GPs’ perspective on IMIs and show that research on bPT is scarce. To the best of our knowledge, no study has yet investigated GPs’ perspectives on barriers to and facilitators for referrals to blended treatments. GPs may be more willing to refer to an IMI when it is combined with face-to-face psychotherapy, which is well known and trustworthy. At the same time, referrals might be facilitated by improved treatment availability and access to treatment options. However, given low GPs’ referral rates to psychotherapy and the novelty of blended treatments, an assessment of factors determining GPs’ referrals to bPT is needed. From this, implementation strategies can be designed to address these and maximize the uptake of bPT. This mixed methods study aimed to identify barriers and facilitators that influence the referral behavior of GPs to a bPT for MDD.


    Study Setting and Design of the RCT

    This study was part of a mixed methods German study arm of the European Project E-COMPARED, which was the first multicenter study that evaluated the clinical and cost-effectiveness of a blended depression treatment compared with treatment as usual across 8 countries. The objective of the German RCT was to evaluate the effectiveness of bPT based on cognitive behavioral therapy (CBT) for adults with a diagnosis of MDD compared with GP routine care. Further information on the RCT can be found in the study protocol [45].

    The trial was approved by the Ethics Committee of the German Society of Psychology and registered in the German Clinical Trials Register (DRKS00006866). All GPs provided informed consent to participate.

    Patients were recruited in GP offices in primary care and randomly assigned to the treatment conditions, bPT (n=86) and GP care (n=87). Professionals were not blinded. The bPT was conducted by psychologists in an outpatient university clinic. GP care consisted of (1) initial screening of MDD, informing about treatment options and referring depressed patients to the study; (2) conducting routine care for the control group; and (3) completing questionnaires about the diagnosis and treatment of patients at baseline and postassessment (13 weeks). All GPs received a print booklet with information about bPT and trial procedures as well as recruitment material (eg, flyers, displays, posters) to inform and motivate their patients. Their effort was reimbursed with EUR 100 (US $117.6) per partaking patient. Further motivating activities were a kickoff meeting, newsletter, phone calls, and onsite visits.

    Blended Internet-Based Psychotherapy

    bPT was delivered as a short-term treatment (13 weeks) for MDD. It combined 6 internet-based CBT lessons (psychoeducation, behavioral activation, cognitive restructuring, problem-solving, physical exercise, and preventing relapse), daily assessments of mood, and cognitive and behavior-related parameters on a mobile app (eg, sleep habits, worries), as well as 6 biweekly face-to-face sessions with a psychologist and CBT therapist in training (Multimedia Appendix 1). Patients had free and password-encrypted access to a website with 10 weekly CBT-based web-based modules (text, videos, and exercises), mood graphs, a calendar, and a messaging system and independently edited the web-based modules at home. They received automated tailored web-based reminders. Therapists referred back and forward to web- and mobile-based content within face-to-face sessions to structure the treatment, monitored patients’ treatment course on the internet, provided weekly web-based feedback on exercises and progress, or wrote reminder and motivation messages. A detailed description and a case report are published elsewhere [46].

    Recruitment and Referrals

    By the end of 2014, more than 1000 GPs were contacted through post, whose practices were located in the relevant catchment area for referrals to the university outpatient clinic (60 km). Overall, 137 GPs from 107 practices in Bavaria (Germany) agreed to refer to bPT. In the recruitment period from February 2015 to August 2016, 121 GPs referred to at least one patient, and 86 GPs had partaking patients in the RCT.

    Successful referrals were operationalized as the number of patients who completed screening for MDD after being informed of it in GP offices. Owing to the study design, it was not possible to document the intended referrals onsite. Higher referral rates can be assumed; since some patients may not have followed the recommendation of their GP, the GPs had reported a high number of patients treated daily (mean 49, SD 22.84), and not all patients could be assigned to a GP.

    The recruitment of GPs for this qualitative study took place shortly before the end of the study enrollment period (March to June 2016) by inviting 110 GPs from the trial population (those without study withdrawal) by email or telephone to participate in interviews. In total, 12 GPs (11%) took part in semistructured interviews. Of 110 GPs, 39 refused to participate; some were interested but had time constraints, and others could not be reached by telephone and emails with 3 reminders.

    Design of the Qualitative Study and Data Collection

    We used a qualitative method with a theory-based approach to gain insights into the experiences and perspectives of GPs regarding their referral behavior to bPT. This exploratory method is a recommended approach to identify barriers and facilitators for implementing interventions [47].

    The semistructured interview guide (Textbox 1) was based on the theoretical domains framework (TDF), which provides a comprehensive theoretical assessment of implementation problems and professional behavior [48]. The 14 domains (eg, knowledge and intention) represent potential determinants for the change of behavior of GPs during the implementation of bPT as a new referral option. It enables researchers to identify the hindering and facilitating factors to support the implementation of evidence-based interventions. Items were evaluated by 3 experts in clinical psychology, e–mental health, and qualitative research to enhance validity. When needed, guided prompts enabled gathering of information, and field notes complemented data collection. Pilot testing of the interview guide with the first interview (IT) did not require any adjustments.

    Textbox 1. Theoretical Domains Framework and exemplary questions in the interview guide.
    View this box

    The sample size was determined by the low response rates to interviews at the end of the RCT enrollment period. Hence, the sample size and composition could not be planned. In total, 3 interviewers conducted 4 interviews with GPs. IT trained 2 interviewers with a Bachelor of Science in Psychology in using the interview guide and gave feedback to their first interviews. The interviews were conducted through telephone between May 2016 and June 2016, whereas the GPs were at the workplace (n=9), at home (n=2), or in the car (n=1). The average duration of the interviews was 56 min (SD 15.26; minimum=36 and maximum=78). Interviews were audio recorded and transcribed verbatim, based on a transcription guide. Data were pseudonymized using code numbers.

    Data Analyses

    Qualitative Analysis

    A qualitative content analysis was conducted, drawing on an inductive-deductive approach and using standardized methodical steps in qualitative research [49]. Codes were developed from the raw data and referred to themes relevant to the research question. The items were based on the TDF.

    First, a list of codes was developed using 50% (n=6) of interview material, which involved identifying emerging themes, discussing the possible meanings of text excerpts, and arranging themes into categories within consensus meetings. Consensual coding was applied to enhance the quality of the coding process [50]; 2 independent coders developed and compared codes and discussed any differences to generate consensus on the codes.

    Second, a further text retrieval was done to (1) label text related to the first draft of the code list, (2) specify subthemes, (3) add new emerging themes to the code list, and (4) target a higher level of abstraction of the codes. A paragraph was coded if it contained themes from one or more categories. IT gave written feedback to the coders regarding all labeled text frames to enrich the data interpretation through clinical expertise. The list of codes was revised and completed with additional definitions and exemplary statements. A preliminary code system, based on 50% (6 interviews) of the material, was developed and discussed in a consensus meeting among IT, SS, and the coders. They reached final agreement on code definitions and excerpts, the structure of the code system, and coding rules. This iterative research process aimed to finalize the code list to fit the data and to optimize the content and number of identified categories.

    Third, all interview transcripts were independently coded by 2 coders in accordance with the code list. There was sufficiently moderate intercoder agreement with the coefficient kappa, k=0.71 [51]. Data saturation (a validity criterion of the codes list) was successfully reached, as each theme was mentioned by at least two GPs and no new themes emerged from the interview data. This indicated that no additional interviews should be conducted [52]. Key themes were defined as emerging themes mentioned by 100% of GPs.

    Finally, to ensure the validity of the identified themes, the 12 GPs were questioned as to whether they agreed with the resultant barriers and facilitators via a survey. The themes were presented as a list. GPs agreed, on average, with 37 of 48 of all themes identified (mean 78%, SD 14%; minimum=44% and maximum=98%). The mean agreement rate per theme (ie, for each barrier or facilitator) was 78% (SD 19%; minimum=8% and maximum=100%). Thus, the identified barriers and facilitators yielded very good validation results.

    Quantitative Analysis

    A comparison of all barriers and facilitators and the percentage of all 12 GPs who mentioned these in the interviews and in the follow-up validation survey was conducted. Agreement rates per GP were analyzed.

    Secondary quantitative analyses were conducted to evaluate whether there were any group differences in the identified barriers and facilitators. The interviewees were divided into 2 groups according to the following characteristics: “referral rates: high (n=4) versus low (n=8)” (cutoff by mean value=6), “experience years as licensed GP: high (n=6) versus low (n=6)” (cutoff by mean value=15), and “training in psychotherapy: yes (n=4) versus no (n=8).” A frequency table and descriptive statistics with 95% CIs were used to present differences in the barriers and facilitators between the abovementioned groups. Inferential statistics were not applied because of small samples and limited power. Power sensitivity analyses indicated that both equal (6/6) and unequal (8/4) small sample sizes would have the sensitivity to detect only a large proportional difference of 0.75 in a Fisher exact test and large effect sizes of d=1.96 (unequal) or 1.85 (equal) in a Mann-Whitney U test, each with a power of 80% and an alpha level of 5% (two tailed).

    Independence tests of the interviewed sample (n=12) and the remaining RCT sample (n=64) were conducted using the Fisher exact test for categorical variables (because of cell frequencies <5) and using the t test or Mann-Whitney U test for continuous variables. The results of the nonparametric Mann-Whitney U test were reported because of unequal sample sizes, if both assumptions (normality of distribution and equality of variance) were not met. The test was applied for verification if the data were nonnormally distributed. All analyses were two sided, with an alpha level of 5%. Sensitivity power analysis for t tests, targeting unequal sample sizes (n=64 and n=12), an alpha level of 5% (two tailed), and 80% power, had yielded the sensitivity to detect an effect size of d=0.89.

    Pearson correlations were conducted to determine the association between referrals and different continuous variables in the total RCT sample (n=76). The power analysis indicated that a sample size of 76 would have the sensitivity to detect an effect size of r=0.31, with a power of 80% (alpha level of 5%, two tailed) in a bivariate correlation.

    The tool MAXQDA 12 (VERBI software, 2015) was used for the qualitative analysis and SPSS 25 (IBM SPSS Statistics, 2017) for the quantitative analysis. Power analyses were conducted using G*Power, version [53]. A guideline for reporting qualitative studies (Consolidated Criteria for Reporting Qualitative Studies [COREQ] checklist) was applied (Multimedia Appendix 2) [54].


    Participant Characteristics and Referral Rates

    Interviewed GPs were mainly men (8/12, 67%), with an average of 50.67 (SD 11.88) years and reported an average of 15.75 (SD 10.42) years of practice experience. Regarding their qualifications, 4 GPs had a license for psychotherapy and 9 had further education in psychosomatic basic care in addition to their university degree in medicine. GPs worked in both metropolitan (7/12, 58%) and rural (5/12, 42%) areas. Sociodemographic data for the remaining RCT sample (available for n=64) showed similar values, such as a proportion of 64% (41/64) being men, an average age of 52.14 (SD 8.73) years with 16.65 (SD 8.34) working years as a licensed GP. There were no statistically significant group differences in characteristics between the interviewed sample (n=12) and the RCT sample (n=64), all P≥.08 (see descriptive statistics in Table 1).

    Table 1. Sample characteristics of participants.
    View this table

    The patients treated with bPT (n=86) were aged on average 43.22 years (SD 13.07; range 19-70), mostly women (61%), highly educated (56%), and employed (74%). Overall, 71% had no prior experience with psychotherapy.

    Of 698 successful referrals to bPT, 321 could be assigned to 121 GPs in the RCT. The majority of RCT-GPs (92/121, 76%) referred fewer than 3 patients to bPT during the 18-month study period, indicating a ceiling effect. However, there was a large variation between GPs in the number of referrals made (RCT sample: mean 2.65, SD 3.92; minimum=1, maximum=26; interview sample: mean 6.34, SD 9.42; minimum=0, maximum=26), and interviewed GPs had higher referral rates than those in the RCT. Table 2 shows the frequency distribution of GPs’ referrals to bPT in the interviewed sample (n=12) and the RCT sample (n=121).

    Table 2. Frequency distribution of GPs’ referral rates to blended psychotherapy.
    View this table

    A significant moderately negative correlation was found between the number of referrals made and self-rated competence in delivering pharmacotherapy (r(73)=−0.31; P<.001), that is, GPs with higher self-confidence in delivering pharmacotherapy were less likely to make referrals to bPT. Correlations between referrals and demographics, medical experience, competence ratings, and depression management statement agreements were otherwise nonsignificant (Multimedia Appendix 3).

    Qualitative Findings

    Altogether, 19 barriers and 29 facilitators were identified and categorized into 4 main areas: (1) the general practitioner, (2) the patient, (3) factors influencing the routine in the GP practice, and (4) factors relating to sociopolitical circumstances that influence the implementation of a bPT for MDD in Germany as well as the referral process. In total, 77% (37/48) of the identified themes were mentioned by at least five or more interviewed GPs (42%), whereas 44% (21/48) of the themes were mentioned by at least eight GPs (67%).

    Barriers to Referrals

    Most emerging barriers (11/19, 58%) were assigned to the patient level. Five barriers (5/19, 26%) were named by at least nine GPs (75% of interviewees), whereas 2 barriers (2/19, 11%) were mentioned by all 12 GPs. These latter 2 key barriers are presented in the following section with a quotation illustrating the physicians’ experiences, whereas all 19 barriers are described with a definition and supporting quotations in Table 3.

    Table 3. General practitioners’ perceived barriers for referrals to blended internet-based psychotherapy (bPT) for depression.
    View this table

    All interviewed GPs reported feeling poorly informed about the content and the procedure of bPT and having little knowledge about the scientific evidence of the treatment, as one GP outlined:

    I know nothing about the effectiveness of bPT. I have not read anything about it in GP journals.

    All respondents agreed that patients who are not familiar with computers or smartphones and have little experience with technology might struggle with the technology-based treatment:

    There are still people today who are not familiar with computers, especially in rural areas.
    Facilitators for Referrals

    The analysis revealed more facilitating than hindering factors on the levels of the GP as a person, GP practice, and sociopolitical circumstances. Of the identified 29 facilitators, 16 (55%) were mentioned by at least 75% (n=8) of GPs and 2 (7%) by all interviewees. Below, these 2 key facilitators are presented with an outlining statement. All 29 facilitators are described with a definition and supporting quotations in Table 4.

    Table 4. General practitioners’ perceived facilitators for referrals to blended internet-based psychotherapy for depression.
    View this table

    All interviewed GPs perceived no role conflict between the referral to bPT and their professional identity and standards:

    I see the referral to bPT as part of my role. I see it as a welcome opportunity to be able to offer the patient a quick and uncomplicated means of assistance here in the practice.

    Each GP identified patient groups for whom the bPT suits well. They characterized these patients as those who are young and highly educated, used to work with computers and smartphones, and ready to integrate them into their treatment:

    I think it is definitely suitable for younger patients and also for patients who are already at work and do not have that much time.

    Quantitative Findings

    Follow-Up Validation of Qualitative Findings

    The follow-up assessment of the identified themes within the qualitative interviews showed a very high agreement rate between all 12 GPs of at least 75% for 34 (71%) of the identified barriers and facilitators. In total, 44 of 48 (92%) themes yielded an agreement rate of at least 58%, and 3 (6%) themes were agreed by all GPs. Only 2 (4%) themes had an agreement rate less than or equal to 25% of GPs.

    Low awareness of bPT as a therapeutic method was the barrier, and integration within guidelines as evidence-based treatment and short waiting time for internet-based intervention were the facilitators, which resulted in the highest agreement rates. Patients’ fear of stigmatization was the barrier, and GPs’ expectation of social reinforcement was the facilitator with the lowest agreement rates.

    All barriers, and the percentage of all 12 GPs who mentioned these in the interviews and in the follow-up validation survey of qualitative findings, are listed in Figure 1. The average proportion of GPs who mentioned these 19 barriers in the interview was 53% (SD 25%; minimum=17% and maximum=100%), whereas the mean agreement rate for barriers in the quantitative follow-up survey was 69% (SD 24%; minimum=8% and maximum=100%). When these 19 identified barriers were presented to GPs as a list, 53% obtained higher agreement rates than in the interviews.

    Figure 1. Frequency graphs of barriers for referrals to blended psychotherapy (bPT) mentioned in the interviews and in the follow-up assessment. GP: general practitioner.
    View this figure

    All enabling factors and the percentage of all 12 GPs who mentioned the facilitator in the interview and the follow-up validation survey of qualitative findings are listed in Figure 2. The average proportion of GPs who mentioned these facilitators in the interview was 66% (SD 25%; minimum=17% and maximum=100%), whereas the mean agreement rate for facilitators in the quantitative follow-up survey was 83% (SD 14%; minimum=42% and maximum=100%). Overall, 66% of these 29 facilitators received higher agreement rates from GPs when presented as a list.

    Figure 2. Frequency graphs of facilitators for referrals to blended psychotherapy (bPT) mentioned in the interviews and in the follow-up assessment. GP: general practitioner.
    View this figure
    Differences in the Barriers and Facilitators Perceived by GPs With Different Characteristics

    Frequency tables indicated there were differences between GPs with different characteristics (low or high referrers, low or high GP experience, and no or yes psychotherapist) in terms of whether they mentioned particular barriers and facilitators. Descriptive statistics and 95% CIs can be found in Multimedia Appendix 4.

    Barriers (10/19) and facilitators (13/29) with a discrepancy of at least 25% among GPs with high versus low referral rates are shown in Figure 3. Those mentioned by more high referrers, with a difference of 50%, included bridging the waiting time for traditional psychotherapy, little room for individualized treatment or personal issues, and limited therapeutic relationship or personal contact. Low referrers more frequently mentioned with a difference of 50% low awareness of bPT as a therapeutic method, personal contact with the therapists and provider, and patients’ motivation and willingness for treatment. Such meaningful differences were not found when comparing GPs with high or low experience and training or no training in psychotherapy.

    Figure 3. Barriers and facilitators mentioned by high versus low referrers with group differences of ≥ 25%.
    View this figure

    The GPs with low referral rates mentioned fewer barriers (mean 9.75, SD 1.83) and facilitators (mean 18.25, SD 4.13) than those with high referral rates (B: mean 10.50, SD 2.38; F: mean 21.00, SD 3.92). Both GP groups mentioned, on average, more facilitators than barriers.


    Principal Findings and Comparison With Prior Work

    This mixed methods study investigated facilitators and barriers for referrals to bPT for depression from the perspective of GPs. The results should be interpreted in light of the fact that the sample was limited, small, self-selected, and unlikely to represent the full range of all GPs’ experiences (because of low response rates). Although referral rates were quite low, they differed between GPs. The lower the RCT-GPs rated their pharmacotherapeutic skills, the more referrals they made. Interviewees referred more than double the number of patients than the RCT-GPs.

    All 29 facilitating and 19 hindering factors were identified on the levels of GP, patient, GP practice, and sociopolitical circumstances. The most frequently named barriers by the interviewed GPs concerned the use of new technologies within blended treatments, as most assumed that some patients would not be familiar with internet and technology or would be skeptical and uncertain as to whether an internet-based intervention could help them. These findings are in line with earlier research on barriers to the use of bPT by psychotherapists [34,35,55,56] as well as IMIs by clinicians (psychiatrists or GPs or psychologists) [57] or by GPs [42]. Furthermore, GPs’ concerns regarding data safety were also mentioned in previous research on bPT by GPs [58] and psychotherapists [34,35,55] as well as on IMIs by GPs [42] and psychotherapists [59].

    The interviewed GPs reported not to be familiar with bPT, to have little knowledge of it, and to prefer more information and training. This is in accordance with the results of studies on bPT in GPs [58] and psychotherapists [56] and stand-alone IMIs in GPs [42,60] and health professionals [61]. GPs also asked for more feedback and professional exchange regarding the treatment, consistent with requests by professionals in previous research on IMIs [57] and with GPs’ reported facilitators for referrals to IMIs [41].

    GPs had positive beliefs about the treatment success of bPT as well as high self-efficacy levels for patient referrals, contrasting previous findings on IMIs for depression in GPs [42]. This might be because of a higher level of trust in familiar face-to-face treatments or to the scientific setting of the conducted bPT, which reduced uncertainties. The finding that GPs feel no conflict with their role was in contrast with psychotherapists’ perceptions [36] but could potentially be explained through their gatekeeper function [40]. Other research suggested that GPs valued the feeling of being more skillful and professional when they blended IMIs with their depression treatment [58].

    Further facilitators for referrals mentioned by the interviewed GPs relate to the health care system (eg, shorter waiting time, simple access, closing the treatment gap, feeling able to react to the high number of depressive patients in a better way) and are in line with research on IMIs with GPs [41,42] and bPT with GPs [58] and psychotherapists [34,35]. At the same time, GPs reported barriers such as little familiarity with treatment, missing reimbursement, and legal requirements, which correspond with research of IMIs in GPs [42] and of bPT in psychotherapists [34,55].

    GPs’ attitudes that patients are suitable for bPT if they are familiar with modern technologies or are affected by minor-to-moderate depression were in line with previous research on bPT with psychotherapists [34,35,56] and GPs [58] and on IMIs with professionals [57,61]. The latter corresponds with treatment guidelines for depression [7,8]. Nevertheless, there is evidence that patients with severe forms can benefit from IMIs compared with untreated controls [62]. Furthermore, GPs judged suicidality and psychotic symptoms as a contraindication for bPT, corresponding with the attitudes of psychotherapists [34,56], despite studies showing that suicidal thoughts [63,64] and positive psychotic symptoms [65] can be targeted successfully with IMIs.

    The finding that GPs perceived younger patients to be more suitable for bPT compared with older patients is in accordance with research on IMIs in clinicians [57] and GPs [58] and on bPT in psychotherapists [34,35,55,56]. However, a meta-analysis indicated that older patients significantly profited from a stand-alone IMI for depression and to an even greater extent compared with younger patients [62].

    The findings can be interpreted in the context of prior systematic reviews on barriers and facilitators and help to develop implementation strategies. First, they align with the 7 stages of the implementation process of an intervention (eg, physician, patient, and system barriers), which were identified using 256 publications to categorize barriers to optimal clinical practice in health care [66]. These stages could support the implementation of referrals to bPT in practice. Second, the findings fall under important groups of determinants relating to the use of e–mental health interventions [67] and suggest that these are considered valid by GPs in relation to bPT. The authors recommend that implementation practitioners consider such determinants to achieve better implementation results and use these to design and apply specific implementation activities. The research project ImpleMentAll aims to find evidence for such an intervention (the ItFits-toolkit) through the development, application, and evaluation of tailored implementation strategies in ongoing eHealth implementation initiatives [68].

    Limitations and Strengths

    This study has noteworthy limitations, such as low response rates of GPs to interview invitations (RCT-GPs: 12/110, 11%) and a limited sample size. This meant that further sampling to check the consistency of findings was not possible, and having a small sample size may have produced fewer themes and nuances in GPs’ views than the 20 to 30 interviews suggested by guidelines [69]. Yet, sufficient data saturation was reached with 12 interviews. All themes were mentioned by at least two GPs and 45 themes (94%) were mentioned by at least three GPs, which suggests that the findings covered important topics. This is in line with a study that reached saturation with the 12th interview of 60 interviews [70]. Nevertheless, the results of this study represent the perceptions of a small number of interviewees and, as such, may not be representative or generalizable for the whole RCT or primary care GPs. For example, interviewed GPs conducted twice as many referrals (and with 3 times the amount of variance) as the larger sample of RCT-GPs, indicating a self-selection bias of the interview participants toward the Central Research Question. Interviewees may have had a higher motivation and openness to referrals and possibly perceived different barriers and facilitators than the group of RCT-GPs. Concordantly, such differences were found within the interviewed GPs between those with high and low referral rates. However, the mean value of the interviewees was biased by 2 participants having the highest referral rates (each 26), and the average referrals of the interviewees as well as RCT-GPs were both rather low (difference in means was 3.69). The low referral rates resulted in a ceiling effect and limited the mean difference between the groups with low and high referrals. This meant that GP group members with high referral rates potentially experienced referrals to bPT similarly to the low referral group, negatively impacting the ability to make meaningful between-group comparisons. The interpretation of quantitative group comparisons should be considered with care because of the small and unequal distributed subsamples. To avoid this limitation, future studies should plan the composition and size of the sample a priori. Furthermore, as low referral rates indicate a low degree of practice and the referral behavior occurred within a standardized RCT setting with an unfamiliar intervention, the perceived barriers and facilitators may not be representative of routine practice. As the GPs participated in the RCT, they may have been biased toward more positive views regarding technology-based treatments and research, indicating a selection bias. GPs in routine care and/or with more experience with bPT might express other attitudes and views. On the other hand, when implementing bPT as a referral option in primary care, these early insights will be important for the outcome. Further limitations include the failure to register the intended referrals by GPs. Referral rates per GP relied on patient self-report screening data and may reflect the successful number of referrals rather than the actual intended referrals.

    The strengths of this work included the mixed methods approach, which enabled the research team to generate in-depth findings by using a theory-based interview guide, and validated qualitative findings with a survey. The consensus and iterative approach is used to develop codes, and independent coding with a moderate interrater agreement enhanced rigor in producing the results. Identifying GP subgroups and comparing differences between these provided an indication of whether barriers or facilitators differed in their importance to different GPs.

    Implications for Clinical Practice and Future Research

    Findings relating to barriers and facilitators could be used to design implementation strategies to support the integration of bPT as a referral option in clinical practice. The TDF is associated with the Behavior Change Wheel framework, which connects theoretical domains to 3 broad drivers of behavior: capability, opportunity, and motivation (COM-B model). It also connects these to specific intervention options [48,71], describes step-by-step how implementers can develop measures for behavioral change, and supports theory-based decision making [71]. Therefore, the study findings can be used to derive helpful, theory-based practical tips for developing effective intervention strategies for the implementation of bPT. Possible interventions include training and communication activities that can address the need for knowledge, attitudes, and misconceptions about bPT and expected patient reactions; legally required sophisticated IT solutions that reduce uncertainties regarding technology use and data safety; and stakeholder and policyholder involvement to drive the necessary adaptation of reimbursement or treatment guidelines.

    In addition, future studies might explore barriers and facilitators in larger samples, invite GPs that have referred to bPT in routine care, and consider different delivery options of digital interventions (eg, guided IMIs). Furthermore, the roles of GPs as referrers to or users of blended treatments for different mental health disorders should be evaluated.


    This study provides insights into barriers and facilitators determining GPs’ referral behavior in relation to bPT for depression. The results indicate that GPs perceive bPT as an additional and valuable treatment delivery format. Having a central position in depression treatment, they experience positive consequences for their own professional group and for patient care when they are able to use bPT as a referral option. Thus, GPs appear to be ready as stakeholders to integrate digital interventions blended with face-to-face psychotherapy in their depression management. However, they experienced considerable barriers to their referrals, which might have led to their low referral rates in this study. Understanding and addressing their perceived barriers and facilitators might enhance their uptake of bPT as a referral option and therefore improve patients’ access to specialized care.

    On the basis of these findings, the following issues should be taken into account when developing an implementation strategy: (1) address the organizational, legal, and reimbursement requirements; (2) consider GPs’ suggestions for implementation, such as the integration of bPT as an additional care pillar in treatment guidelines and the development of measures to increase familiarity with bPT and its advantages (eg, shorter waiting time, improved treatment availability); (3) save GPs’ resources (eg, digital referral receipts, automated feedback about treatment findings, patient leaflet); (4) ensure fit with GPs’ habits and routine activities (eg, interacting with information technology and reimbursement structures in health care); and (5) address GPs’ need for information and training on bPT as well as personal contact and feedback in communication with therapists.


    The authors would like to thank the participating GPs, the 2 coders, and Steffen Hartmann (student assistant) for their contributions to this work. The authors also thank Claire McCallum for proofreading. The study was conducted by the German trial partner of the E-COMPARED project, which was funded under the Seventh Framework Program (grant agreement 603098). The funder had no role in the research idea, study design, data collection, analysis and interpretation, decision to publish, or preparation of the paper. The authors acknowledge support from Deutsche Forschungsgemeinschaft and Friedrich-Alexander-Universität Erlangen-Nürnberg within the funding program, Open Access Publishing.

    Authors' Contributions

    IT designed the study, developed the interview guide, recruited participants, collected the data, designed or contributed to the qualitative data analysis and interpretation within consensus meetings, performed the quantitative analysis, and drafted the manuscript. SS contributed as therapist and independent researcher to the discussion of emerging themes within consensus meetings as well as critically revised the manuscript. DE supervised the manuscript writing. MB and HR critically revised the manuscript. All authors have read and approved the final manuscript.

    Conflicts of Interest

    DE has served as a consultant on scientific advisory boards in the eHealth setting. DE is a stakeholder of the GET.ON Institute, which aims to implement science-based digital health interventions in routine care. IT and SS have received fees and travel costs for lectures or workshops in the eHealth setting from congresses and psychotherapy training institutes. HR and MB declare that they have no competing interests.

    Multimedia Appendix 1

    Description of the blended internet-based psychotherapy as intervention and screenshots of the Moodbuster website and app.

    DOCX File , 1470 KB

    Multimedia Appendix 2

    Consolidated Criteria for Reporting Qualitative Studies checklist. This table shows the COREQ checklist, a guideline for reporting qualitative studies.

    DOCX File , 41 KB

    Multimedia Appendix 3

    Correlations, means, SDs, and minimum/maximum of general practitioners’ referrals and characteristics (randomized controlled trial; sample N=76). This table shows the correlations between the number of referrals and characteristics, self-ratings on competences, and agreements to statements about depression management.

    XLSX File (Microsoft Excel File), 14 KB

    Multimedia Appendix 4

    Descriptive statistics and 95% CI for different general practitioner groups and their mentioned barriers and facilitators (interviewed general practitioners N=12). The contingency tables show the frequency and total numbers of barriers and facilitators in each group (low/high referrers, low/high general practitioner experience, and psychotherapist yes/no).

    XLSX File (Microsoft Excel File), 30 KB


    1. World Health Organization. The Global Burden of Disease: Update 2004. 2008.   URL: [accessed 2019-12-11]
    2. Alonso J, Angermeyer MC, Bernert S, Bruffaerts R, Brugha TS, Bryson H, ESEMeD/MHEDEA 2000 Investigators‚ European Study of the Epidemiology of Mental Disorders (ESEMeD) Project. Prevalence of mental disorders in Europe: results from the European study of the epidemiology of mental disorders (ESEMeD) project. Acta Psychiatr Scand Suppl 2004(420):21-27. [CrossRef] [Medline]
    3. Wittchen H, Pittrow D. Prevalence, recognition and management of depression in primary care in Germany: the depression 2000 study. Hum Psychopharmacol 2002 Jun;17(Suppl 1):S1-11. [CrossRef] [Medline]
    4. Richards JC, Ryan P, McCabe MP, Groom G, Hickie IB. Barriers to the effective management of depression in general practice. Aust N Z J Psychiatry 2004 Oct;38(10):795-803. [CrossRef] [Medline]
    5. Craven MA, Bland R. Depression in primary care: current and future challenges. Can J Psychiatry 2013 Aug;58(8):442-448. [CrossRef] [Medline]
    6. Becker N, Abholz H. Prävalenz und Erkennen von depressiven Störungen in deutschen Allgemeinarztpraxen - eine systematische Literaturübersicht. Z Allg Med 2005;81(11):474-481. [CrossRef]
    7. National Collaborating Centre for Mental Health (UK). Depression: The Treatment and Management of Depression in Adults (Updated Edition) 2010. [Medline]
    8. DGPPN BÄK, KBV, AWMF, AkdÄ, BPtK, BApK, DAGSHG, DEGAM, DGPM, DGPs, DGRW. Unipolare Depression. NVL - Nationale VersorgungsLeitlinien 2015 DOI: 10.6101/AZQ/000266 [FREE Full text] [CrossRef]
    9. Wagner CJ, Metzger FG, Sievers C, Marschall U, L'hoest H, Stollenwerk B, et al. Depression-related treatment and costs in Germany: do they change with comorbidity? A claims data analysis. J Affect Disord 2016 Mar 15;193:257-266. [CrossRef] [Medline]
    10. Gaebel W, Kowitz S, Zielasek J. The DGPPN research project on mental healthcare utilization in Germany: inpatient and outpatient treatment of persons with depression by different disciplines. Eur Arch Psychiatry Clin Neurosci 2012 Nov;262(Suppl 2):S51-S55. [CrossRef] [Medline]
    11. Gerste B, Roick C. Prävalenz und Inzidenz sowie Versorgung depressiver Erkrankungen in Deutschland. In: Klauber J, Günster C, Gerste B, Robra BP, Schmacke N, editors. Versorgungs-Report 2013/2014. Germany: Schwerpkt Depress Stuttgart; 2014:21-54.
    12. Trautmann S, Beesdo-Baum K. The treatment of depression in primary care. Dtsch Arztebl Int 2017 Oct 27;114(43):721-728 [FREE Full text] [CrossRef] [Medline]
    13. Jacobi F, Höfler M, Meister W, Wittchen HU. [Prevalence, detection and prescribing behavior in depressive syndromes. A German federal family physician study]. Nervenarzt 2002 Jul;73(7):651-658. [CrossRef] [Medline]
    14. Piek E, van der Meer K, Penninx BW, Verhaak PF, Nolen WA. Referral of patients with depression to mental health care by Dutch general practitioners: an observational study. BMC Fam Pract 2011 May 26;12:41 [FREE Full text] [CrossRef] [Medline]
    15. Wang J, Langille DB, Patten SB. Mental health services received by depressed persons who visited general practitioners and family doctors. Psychiatr Serv 2003 Jun;54(6):878-883. [CrossRef] [Medline]
    16. Alexander C, Fraser J. General practitioners' management of patients with mental health conditions: the views of general practitioners working in rural north-western New South Wales. Aust J Rural Health 2008 Dec;16(6):363-369. [CrossRef] [Medline]
    17. Albani C, Blaser G, Geyer M, Schmutzer G, Brähler E. Ambulante psychotherapie in Deutschland aus Sicht der Patienten. Teil 1: Versorgungssituation. Psychotherapeut 2010;55(6):503-514. [CrossRef]
    18. Schneider F, Kratz S, Bermejo I, Menke R, Mulert C, Hegerl U, et al. Insufficient depression treatment in outpatient settings. Ger Med Sci 2004 Feb 26;2:Doc01 [FREE Full text] [Medline]
    19. Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet 2009 Aug 22;374(9690):609-619. [CrossRef] [Medline]
    20. Telford R, Hutchinson A, Jones R, Rix S, Howe A. Obstacles to effective treatment of depression: a general practice perspective. Fam Pract 2002 Feb;19(1):45-52. [CrossRef] [Medline]
    21. Gröber-Grätz D, Mosshammer D, Bölter R, Ose D, Joos S, Natanzon I. [Which criteria affect the cooperation between general practitioners and specialists in ambulatory care? A qualitative study about general practitioners' perception]. Z Evid Fortbild Qual Gesundhwes 2011;105(6):446-451. [CrossRef] [Medline]
    22. Kravitz RL, Franks P, Feldman M, Meredith LS, Hinton L, Franz C, et al. What drives referral from primary care physicians to mental health specialists? A randomized trial using actors portraying depressive symptoms. J Gen Intern Med 2006 Jun;21(6):584-589 [FREE Full text] [CrossRef] [Medline]
    23. Mack S, Jacobi F, Gerschler A, Strehle J, Höfler M, Busch MA, et al. Self-reported utilization of mental health services in the adult German population--evidence for unmet needs? Results of the DEGS1-mental health module (DEGS1-MH). Int J Methods Psychiatr Res 2014 Sep;23(3):289-303 [FREE Full text] [CrossRef] [Medline]
    24. Kessler RC, Berglund PA, Bruce ML, Koch JR, Laska EM, Leaf PJ, et al. The prevalence and correlates of untreated serious mental illness. Health Serv Res 2001 Dec;36(6 Pt 1):987-1007 [FREE Full text] [Medline]
    25. Königbauer J, Letsch J, Doebler P, Ebert D, Baumeister H. Internet- and Mobile-based Depression Interventions for People with Diagnosed Depression: A Systematic Review and Meta-analysis. J Affect Disord 2017 Dec 1;223:28-40. [CrossRef] [Medline]
    26. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther 2018 Jan;47(1):1-18. [CrossRef] [Medline]
    27. Hadjistavropoulos HD, Nugent MM, Dirkse D, Pugh N. Implementation of internet-delivered cognitive behavior therapy within community mental health clinics: a process evaluation using the consolidated framework for implementation research. BMC Psychiatry 2017 Sep 12;17(1):331 [FREE Full text] [CrossRef] [Medline]
    28. Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect Psychol Sci 2011 Jan;6(1):21-37. [CrossRef] [Medline]
    29. Apolinário-Hagen J, Vehreschild V, Alkoudmani RM. Current views and perspectives on e-mental health: an exploratory survey study for understanding public attitudes toward internet-based psychotherapy in Germany. JMIR Ment Health 2017 Feb 23;4(1):e8 [FREE Full text] [CrossRef] [Medline]
    30. Ebert DD, Berking M, Cuijpers P, Lehr D, Pörtner M, Baumeister H. Increasing the acceptance of internet-based mental health interventions in primary care patients with depressive symptoms. A randomized controlled trial. J Affect Disord 2015 May 1;176:9-17. [CrossRef] [Medline]
    31. Erbe D, Eichert H, Riper H, Ebert DD. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: systematic review. J Med Internet Res 2017 Sep 15;19(9):e306 [FREE Full text] [CrossRef] [Medline]
    32. Etzelmueller A, Radkovsky A, Hannig W, Berking M, Ebert DD. Patient's experience with blended video- and internet based cognitive behavioural therapy service in routine care. Internet Interv 2018 Jun;12:165-175 [FREE Full text] [CrossRef] [Medline]
    33. Schuster R, Sigl S, Berger T, Laireiter A. Patients' experiences of web- and mobile-assisted group therapy for depression and implications of the group setting: qualitative follow-up study. JMIR Ment Health 2018 Jul 11;5(3):e49 [FREE Full text] [CrossRef] [Medline]
    34. Titzler I, Saruhanjan K, Berking M, Riper H, Ebert DD. Barriers and facilitators for the implementation of blended psychotherapy for depression: a qualitative pilot study of therapists' perspective. Internet Interv 2018 Jun;12:150-164 [FREE Full text] [CrossRef] [Medline]
    35. Schuster R, Pokorny R, Berger T, Topooco N, Laireiter A. The advantages and disadvantages of online and blended therapy: survey study amongst licensed psychotherapists in Austria. J Med Internet Res 2018 Dec 18;20(12):e11007 [FREE Full text] [CrossRef] [Medline]
    36. Kivi M, Eriksson MC, Hange D, Petersson E, Björkelund C, Johansson B. Experiences and attitudes of primary care therapists in the implementation and use of internet-based treatment in Swedish primary care settings. Internet Interv 2015 Sep;2(3):248-256. [CrossRef]
    37. Sethi S, Campbell AJ, Ellis LA. The use of computerized self-help packages to treat adolescent depression and anxiety. J Technol Hum Serv 2010 Aug 31;28(3):144-160. [CrossRef]
    38. Ly KH, Topooco N, Cederlund H, Wallin A, Bergström J, Molander O, et al. Smartphone-supported versus full behavioural activation for depression: a randomised controlled trial. PLoS One 2015;10(5):e0126559 [FREE Full text] [CrossRef] [Medline]
    39. Kooistra LC, Wiersma JE, Ruwaard J, Neijenhuijs K, Lokkerbol J, van Oppen P, et al. Cost and effectiveness of blended versus standard cognitive behavioral therapy for outpatients with depression in routine specialized mental health care: pilot randomized controlled trial. J Med Internet Res 2019 Oct 29;21(10):e14261 [FREE Full text] [CrossRef] [Medline]
    40. Folker AP, Mathiasen K, Lauridsen SM, Stenderup E, Dozeman E, Folker MP. Implementing internet-delivered cognitive behavior therapy for common mental health disorders: A comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv 2018 Mar;11:60-70 [FREE Full text] [CrossRef] [Medline]
    41. Titov N, Rock D, Bezuidenhout G, Webb N, Kayrouz R, Nielssen O, et al. Evaluation of the practitioner online referral and treatment service (PORTS): the first 18 months of a state-wide digital service for adults with anxiety, depression, or substance use problems. Cogn Behav Ther 2020 Jul;49(4):307-326. [CrossRef] [Medline]
    42. Breedvelt JJ, Zamperoni V, Kessler D, Riper H, Kleiboer AM, Elliott I, et al. GPs' attitudes towards digital technologies for depression: an online survey in primary care. Br J Gen Pract 2019 Mar;69(680):e164-e170 [FREE Full text] [CrossRef] [Medline]
    43. Montero-Marín J, Prado-Abril J, Botella C, Mayoral-Cleries F, Baños R, Herrera-Mercadal P, et al. Expectations among patients and health professionals regarding web-based interventions for depression in primary care: a qualitative study. J Med Internet Res 2015 Mar 10;17(3):e67 [FREE Full text] [CrossRef] [Medline]
    44. Hetherton J, Matheson A, Robson M. Recruitment by GPs during consultations in a primary care randomized controlled trial comparing computerized psychological therapy with clinical psychology and routine GP care: problems and possible solutions. Prim Heal Care Res Dev 2004 Jan 1;5(1):5-10. [CrossRef]
    45. Kleiboer A, Smit J, Bosmans J, Ruwaard J, Andersson G, Topooco N, et al. European comparative effectiveness research on blended depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries. Trials 2016 Aug 3;17(1):387 [FREE Full text] [CrossRef] [Medline]
    46. Titzler I, Egle V, Berking M, Gumbmann C, Ebert D. Blended psychotherapy: treatment concept and case report for the integration of internet- and mobile-based interventions into brief psychotherapy of depressive disorders. Verhaltenstherapie 2019 Nov 20:1-15 [FREE Full text] [CrossRef]
    47. Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, et al. Designing and evaluating complex interventions to improve health care. Br Med J 2007 Mar 3;334(7591):455-459 [FREE Full text] [CrossRef] [Medline]
    48. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 2012 Apr 24;7:37 [FREE Full text] [CrossRef] [Medline]
    49. Mayring P. Qualitative Inhaltsanalyse: Grundlagen und Techniken. Eleventh Edition. Weinheim, Germany: Beltz; 2010.
    50. Kuckartz U. Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung Grundlagentexte Methoden. Germany: Beltz Juv; 2016.
    51. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22(3):276-282 [FREE Full text] [Medline]
    52. Corbin J, Strauss A. Basics of Qualitative Research. Third Edition. Thousand Oaks, CA: Sage Publications; 2008.
    53. Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007 May;39(2):175-191. [CrossRef] [Medline]
    54. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357. [CrossRef] [Medline]
    55. Paul J, Grünzig SD, Baumeister H, Bengel J, Krämer LV. [New media in psychotherapy: blended therapy from the perspective of outpatient psychotherapists]. Psychother Psychosom Med Psychol 2019 Oct;69(11):437-444. [CrossRef] [Medline]
    56. Mol M, van Genugten C, Dozeman E, van Schaik DJ, Draisma S, Riper H, et al. Why uptake of blended internet-based interventions for depression is challenging: a qualitative study on therapists' perspectives. J Clin Med 2019 Dec 30;9(1):91 [FREE Full text] [CrossRef] [Medline]
    57. Sinclair C, Holloway K, Riley G, Auret K. Online mental health resources in rural Australia: clinician perceptions of acceptability. J Med Internet Res 2013 Sep 5;15(9):e193 [FREE Full text] [CrossRef] [Medline]
    58. Wilhelmsen M, Høifødt RS, Kolstrup N, Waterloo K, Eisemann M, Chenhall R, et al. Norwegian general practitioners' perspectives on implementation of a guided web-based cognitive behavioral therapy for depression: a qualitative study. J Med Internet Res 2014 Sep 10;16(9):e208 [FREE Full text] [CrossRef] [Medline]
    59. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [CrossRef] [Medline]
    60. Beaulac J, Vincent N, Walsh K. Dissemination of an internet-based treatment for chronic insomnia into primary care. Behav Sleep Med 2015;13(2):124-139. [CrossRef] [Medline]
    61. Gun SY, Titov N, Andrews G. Acceptability of Internet treatment of anxiety and depression. Australas Psychiatry 2011 Jun;19(3):259-264. [CrossRef] [Medline]
    62. Karyotaki E, Ebert DD, Donkin L, Riper H, Twisk J, Burger S, et al. Do guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis. Clin Psychol Rev 2018 Jul;63:80-92. [CrossRef] [Medline]
    63. Christensen H, Farrer L, Batterham PJ, Mackinnon A, Griffiths KM, Donker T. The effect of a web-based depression intervention on suicide ideation: secondary outcome from a randomised controlled trial in a helpline. BMJ Open 2013 Jun 28;3(6):- [FREE Full text] [CrossRef] [Medline]
    64. Mewton L, Andrews G. Cognitive behaviour therapy via the internet for depression: a useful strategy to reduce suicidal ideation. J Affect Disord 2015 Jan 1;170:78-84. [CrossRef] [Medline]
    65. Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res 2014 Jun;156(1):96-106. [CrossRef] [Medline]
    66. Cochrane LJ, Olson CA, Murray S, Dupuis M, Tooman T, Hayes S. Gaps between knowing and doing: understanding and assessing the barriers to optimal health care. J Contin Educ Health Prof 2007;27(2):94-102. [CrossRef] [Medline]
    67. Vis C, Mol M, Kleiboer A, Bührmann L, Finch T, Smit J, et al. Improving implementation of emental health for mood disorders in routine practice: systematic review of barriers and facilitating factors. JMIR Ment Health 2018 Mar 16;5(1):e20 [FREE Full text] [CrossRef] [Medline]
    68. Bührmann, L, Schuurmans J, Ruwaard J, Fleuren M, Etzelmueller A, Piera-Jiménez J, et al. Tailored Implementation of Internet-based Cognitive Behavioural Therapy in the Multinational Context of the ImpleMentAll Project: A Study Protocol for a Stepped Wedge Cluster Randomized Trial. Trials (forthcoming).
    69. Creswell J. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. Thousand Oaks, CA: Sage Publications; 1998.
    70. Guest G, Bunce A, Johnson L. How many interviews are enough? Field Methods 2016 Jul 21;18(1):59-82. [CrossRef]
    71. Michie S, Atkins L, West R. The Behaviour Change Wheel: A Guide to Designing Interventions. Great Britain, UK: Silverback Publishing; 2014.


    bPT: blended psychotherapy
    CBT: cognitive behavioral therapy
    E-COMPARED: European Comparative Effectiveness Research on Internet-based Depression Treatment
    e–mental health: electronic mental health
    GP: general practitioner
    IMI: internet- and mobile-based intervention
    MDD: major depressive disorder
    RCT: randomized controlled trial
    TDF: theoretical domains framework

    Edited by J Torous, G Eysenbach; submitted 09.03.20; peer-reviewed by R Schuster, M Kivi, J D'Arcey; comments to author 31.03.20; revised version received 04.05.20; accepted 09.07.20; published 18.08.20

    ©Ingrid Titzler, Matthias Berking, Sandra Schlicker, Heleen Riper, David Daniel Ebert. Originally published in JMIR Mental Health (, 18.08.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.