This paper is in the following e-collection/theme issue:

Depression and Mood Disorders; Suicide Prevention (1392) Anxiety and Stress Disorders (1081) Telehealth and Telemonitoring (1713) Telepsychiatry (164)

Published on in Vol 8, No 3 (2021): March

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/22569, first published .
Mental Health Specialist Video Consultations Versus Treatment-as-Usual for Patients With Depression or Anxiety Disorders in Primary Care: Randomized Controlled Feasibility Trial

Mental Health Specialist Video Consultations Versus Treatment-as-Usual for Patients With Depression or Anxiety Disorders in Primary Care: Randomized Controlled Feasibility Trial

Mental Health Specialist Video Consultations Versus Treatment-as-Usual for Patients With Depression or Anxiety Disorders in Primary Care: Randomized Controlled Feasibility Trial

Authors of this article:

Justus Tönnies1 Author Orcid Image ;   Mechthild Hartmann1 Author Orcid Image ;   Michel Wensing2 Author Orcid Image ;   Joachim Szecsenyi2 Author Orcid Image ;   Frank Peters-Klimm2 Author Orcid Image ;   Regina Brinster3 Author Orcid Image ;   Dorothea Weber3 Author Orcid Image ;   Markus Vomhof4 Author Orcid Image ;   Andrea Icks4 Author Orcid Image ;   Hans-Christoph Friederich1 Author Orcid Image ;   Markus W Haun1 Author Orcid Image
Article Authors Cited by (20) Tweetations (4) Metrics

    Original Paper

    1Department of General Internal Medicine and Psychosomatics, Heidelberg University, Heidelberg, Germany

    2Department of General Practice and Health Services Research, Heidelberg University, Heidelberg, Germany

    3Institute of Medical Biometry and Informatics, Heidelberg University, Heidelberg, Germany

    4Institute for Health Services Research and Health Economics, Centre for Health and Society, Heinrich-Heine-University, Düsseldorf, Düsseldorf, Germany

    Corresponding Author:

    Markus W Haun, BSc, MSc, MD

    Department of General Internal Medicine and Psychosomatics

    Heidelberg University

    Im Neuenheimer Feld 410

    Heidelberg, D-69120

    Germany

    Phone: 49 622156 ext 8774

    Email: markus.haun@med.uni-heidelberg.de


    Background: Most people affected by depression or anxiety disorders are treated solely by their primary care physician. Access to specialized mental health care is impeded by patients’ comorbidity and immobility in aging societies and long waiting times at the providers’ end. Video-based integrated care models may leverage limited resources more efficiently and provide timely specialized care in primary care settings.

    Objective: The study aims to evaluate the feasibility of mental health specialist video consultations with primary care patients with depression or anxiety disorders.

    Methods: Participants were recruited by their primary care physicians during regular practice visits. Patients who had experienced at least moderate symptoms of depression and/or anxiety disorders were considered eligible for the study. Patients were randomized into 2 groups receiving either treatment-as-usual as provided by their general practitioner or up to 5 video consultations conducted by a mental health specialist. Video consultations focused on systematic diagnosis and proactive monitoring using validated clinical rating scales, the establishment of an effective working alliance, and a stepped-care algorithm within integrated care adjusting treatments based on clinical outcomes. Feasibility outcomes were recruitment, rate of loss to follow-up, acceptability of treatment, and attendance at sessions. Effectiveness outcomes included depression (Patient Health Questionnaire-9), anxiety (Generalized Anxiety Disorder-7), burden of specific somatic complaints (Somatic Symptom Disorder-B Criteria Scale-12), recovery (Recovery Assessment Scale-German [RAS-G]), and perception of chronic illness care (Patient Assessment of Chronic Illness Care), which were measured at baseline and 16 weeks postallocation by assessors blinded to the group allocation.

    Results: A total of 50 patients with depression and/or anxiety disorders were randomized, 23 in the intervention group and 27 in the treatment-as-usual group. The recruitment yield (number randomized per number screened) and the consent rate (number randomized per number eligible) were 69% (50/73) and 86% (50/58), respectively. Regarding acceptability, 87% (20/23) of the participants in the intervention group completed the intervention. Of the 108 planned video consultations, 102 (94.4%) were delivered. Follow-up rates were 96% (22/23) and 85% (23/27) for the intervention and control groups, respectively. The change from baseline scores at postmeasurement for the No Domination by Symptoms domain of recovery (RAS-G) was somewhat higher in the intervention group than in the control group (Mann-Whitney U test: rank-biserial r=0.19; 95% CI −0.09 to 0.46; P=.18). We did not detect any notable differences between the intervention and control groups in terms of other effectiveness outcomes. We did not observe any serious adverse events related to the trial.

    Conclusions: The intervention and study procedures were found to be feasible for patients, primary care practice staff, and mental health specialists. A sufficiently powered pragmatic trial on mental health specialist video consultations should be conducted to investigate their effectiveness in routine care.

    Trial Registration: German Clinical Trials Register DRKS00015812; https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00015812.

    JMIR Ment Health 2021;8(3):e22569

    doi:10.2196/22569

    Keywords

    primary care; integrated care; telepsychiatry; videoconferencing; depression; anxiety; recovery; randomized controlled trial 


    Primary Care Mental Health

    Depression and anxiety disorders are two of the three most prevalent mental disorders and cause substantial global and individual disease burden [1]. Patients with depression or anxiety disorders are often treated exclusively in primary care, which brings the primary care physician in a crucial position for mental health care [2-4]. Most primary care physicians provide comprehensive care to their patients. However, a substantial proportion of patients with severe conditions and somatic comorbidities are not adequately treated. They need more specialized care; however, their access is often impeded by (1) long waiting times at the provider’s end, (2) older patients’ immobility because of increasing multimorbidity in an aging society, and (3) an emphasis on assessment and treatment of somatic symptoms because of guideline recommendations [5,6]. To resolve these challenges, it is essential to develop health care models that combine the easily accessible environment of primary care and the expertise in timely diagnostics and therapy of a mental health specialist.

    Health care models, which may provide a tailored treatment for patients initially presenting to their primary care physician, have been developed. In some of these models (eg, collaborative care), the primary care physician is supported by a care manager, who tracks patients per telephone, conducts psychological assessments, and presents the data to a mental health specialist, often a psychiatrist [7,8]. The mental health specialist monitors the patients by scanning the case reports and can intervene, if necessary, by prescribing drugs or scheduling face-to-face consultations. In other models, the primary care team and mental health specialist are colocated [9-11]. The mental health specialist provides team-based specialized treatment as a routine part of primary care, such as goal setting together with patients, patient activation, and psychosocial care. In the practice, direct cooperation allows patients to be referred by warm handoffs instead of conventional referral forms. Regardless of whether a mental health specialist is locally present, these health care models provide more direct access to specialized care for mental health patients and foster cooperation between primary and specialized mental health care.

    These integrated care models are promising and have been successfully implemented, particularly in the US health care system. However, small and remote primary care practices struggle with the implementation of these care models. In European countries, such as the United Kingdom, France, and Germany, where the mean number of physicians per practice is lower than that in the United States, practices with 1 or 2 doctors often do not have the financial resources to employ an additional mental health specialist [12]. Especially in the German health care system, integrated mental health care models have rarely been implemented so far. Therefore, it is essential to develop and evaluate innovative modes to put these integrated care approaches into practice.

    Mental Health Specialist Video Consultations

    Real-time video consultations conducted by mental health specialists have been shown to be a promising approach to integrated care. This technology-supported mode of delivery is increasingly considered as an alternative to face-to-face settings. We conducted a thorough literature review and identified 315 records. A total of 11 records were relevant, and among these, 6 systematic reviews show that, in general, telemedicine interventions for mental health conditions seem to be effective [13-18]. Concerning the integration of telepsychiatry services in primary care, several observational and interventional studies have demonstrated that mental health specialist video consultations contribute to overcoming geographical barriers and treating the increasing number of multimorbid patients often cut off from specialized care [19-24]. Randomized trials evaluating video consultations have been conducted either in the unique setting of the US Veterans Health Care Administration in Rural Federally Qualified Health Centers [19,20] or included patients from inpatient health care settings [23,24]. The implementation of telemedical approaches within mental health care has generally been promoted more in the United States than in Europe through passing guidelines by the American Telemedicine Association [25]. In particular, for European primary health care settings, the results of those settings can therefore only be generalized to a limited extent.

    Purpose of the Study

    Consequently, the aim of this study is to evaluate if and how mental health specialist video consultations and primary care can be integrated into a European health care system. Therefore, we conducted a randomized controlled feasibility trial in Germany by implementing mental health specialist video consultations in 5 primary care practices. If the intervention proves to be feasible, the results of this trial will inform the planning and setup of a subsequent larger randomized controlled prospective trial to evaluate efficacy.


    Trial Design and Participants

    We conducted an assessor-blinded, randomized, prospective, parallel group feasibility PROVIDE-B (improving cross-sectoral collaboration between primary and psychosocial care: an implementation study on video consultations-B) trial between March 1 and October 7, 2019, in 5 primary care practices in the State of Baden-Wuerttemberg in Southern Germany [26]. Primary care physicians were either recruited during a preceding qualitative preimplementation study [27] or through a network of collaborating academic research practices affiliated with the Department of General Practice and Health Services Research at Heidelberg University. We sent an invitation letter and visited interested practices to inform the practice teams about the study, including the concomitant process evaluation and the assessments involved. We also tested the quality of the internet connection to evaluate eligibility. We recruited 4 mental health specialists at the Institute for Psychotherapy, Heidelberg, which is a state-approved psychotherapeutic training facility at Heidelberg University. Mental health specialists were clinical psychologists with a diploma or master’s degree in psychotherapy training or resident doctor training for board certification in psychosomatic medicine and psychotherapy, which is an independent specialty in Germany. All participating specialists had at least 2 years of training. Although specialists were not allowed to prescribe medication because of regulatory reasons, they had the possibility to suggest starting the patient on medication or changing their medication.

    Eligible patients (1) exceeded cutoffs of 9 points for the Patient Health Questionnaire-9 (PHQ-9) and/or for the Generalized Anxiety Disorder-7 (GAD-7), respectively [28], which represents at least moderately severe symptom burden by either disorder; (2) did not yet have mental health treatment or, until the date of commencement of the study, insufficient treatment (psychotherapy, psychopharmacotherapy, or both) or difficulty with adherence to treatment; (3) agreed to participate in the study by written informed consent; (4) were capable of giving consent; and (5) were aged 18 years or older. Exclusion criteria for patients were (1) substance abuse/dependence that is likely to compromise intervention adherence; (2) risk of endangerment to others and/or risk of self-endangerment; (3) need for emergency medical treatment, for example, admission; (4) acute psychotic symptoms, for example, persecutory delusions and/or thought insertion; (5) severe cognitive impairment or dementia; (6) significant hearing and/or visual impairment; (7) pregnancy in the second trimester or later; and (8) insufficient German language proficiency. To ensure maximum generalizability, general practitioners as experts for their patients decided whether treatment was insufficient or whether there were difficulties with adherence. All other inclusion and exclusion criteria were assessed through standardized computer-assisted telephone interviews conducted by a study team member. The PROVIDE-B trial protocol was approved by the Medical Faculty of the University of Heidelberg Ethics Committee (S-634/2018) and was subsequently published [26].

    Randomization and Masking

    The participants were recruited via their primary care physicians during regular visits in the practice. On the basis of their clinical judgment, GPs prospectively selected individuals suspected to be affected by depression or anxiety and presented the study to them by offering information material. After providing written informed consent, eligible participants were randomly assigned (1:1) to the video consultation model versus treatment-as-usual via a secure, web-based randomization system (Randomizer V.2.0.2) operated by a data manager at the Institute of Medical Biometry and Informatics, Heidelberg University. We used block randomization stratified by primary care practice, with a block size of 4. Randomization at the individual level was independent and concealed. Allocation was subsequently made known to the principal investigator (M Haun), trial coordinator (JT), and mental health specialists. Participants, mental health specialists, and primary care practice staff were informed of the allocation by phone or email. Telephone interviews were used to assess the baseline data before randomization. Two research assistants, masked to group allocation, conducted the postmeasurement in telephone interviews with the participants.

    Procedures

    Participants allocated to the mental health specialist video consultations were offered up to 5 sessions with a mental health specialist during a 3-month treatment window. If patients and mental health specialists agreed that no further treatment was required, they were allowed to end the consultations as early as after the third session. The intervention featured web-based, real-time video consultations involving a 2-way interactive video to a primary care practice between mental health specialists and patients. Apart from that, the intervention was fairly similar to conventional consultation-liaison models in mental health primary care [29] and the collaborative care model [30,31], which both constitute a trade-off between increasing involvement of the primary care clinician on the one hand and increasing involvement of the mental health specialist on the other hand [32]. Both models, such as the PROVIDE-B intervention, target well-defined disorders that are associated with some degree of disability but for which effective treatments are available. Nevertheless, in contrast to consultation-liaison and collaborative care services where mental health specialists act as advisors to primary care physicians (eg, care managers in collaborative care), our intervention included more therapeutic aspects. Specifically, the intervention included 3 core intervention elements (active ingredients) for effective primary care–based mental health care, namely (1) systematic diagnosis plus proactive monitoring using validated clinical rating scales, (2) the establishment of an effective working alliance, and (3) a stepped-care algorithm within integrated care adjusting treatments based on clinical outcomes. If indicated, the intervention also included brief psychological therapy that worked with interpersonal dynamics, which has been shown to confer additional benefits [8]. When the patient had a more chronic condition that demanded long-term treatment, the mental health specialists and the patients mutually developed a care plan that, if indicated, included transition to secondary specialist care. Furthermore, the mental health specialist discussed cases with primary care physicians. The intervention followed a transdiagnostic treatment approach for emotional disorders (depression and anxiety), for which various meta-analyses have shown efficacy compared with control conditions on measures of overall anxiety, disorder-specific anxiety, and depression [33,34]. In addition, the intervention entailed elements from problem-solving therapy, which has been shown to yield moderate effects in alleviating depression and anxiety in primary care [35]. Psychodynamic elements following a relationship focus and interpersonal understanding were added to foster the working alliance, which has been promoted as a crucial element of manuals achieving high acceptability in both patients and clinicians. At the end of the consultations, the mental health specialist proceeded as laid out in the care plan, providing a treatment summary and tailored recommendations to both the patient and the primary care physician. The intervention was conducted in line with the Best Practices in Videoconferencing-Based Telemental Health issued by the American Psychiatric Association and the American Telemedicine Association [25]. In line with the stage model of psychotherapy manual development, we compiled a stage I intervention manual delineating treatment techniques, goals, and format (the manual is available in the study by Tönnies et al [26]). For the description of the intervention, we followed the template for intervention description and replication guidance [26,36]. A structured description of the intervention is presented in Multimedia Appendix 1.

    Patients received their first video consultation shortly after randomization and were scheduled for up to 5 sessions, lasting 50 minutes each, at biweekly intervals. The video consultations were conducted on a secure (ie, encrypted), web-based videoconferencing platform on a subscription basis (arztkonsultation ak GmbH;) at the fixed time slots set by the primary care practice staff. The patients were in a designated room in the general practice and the mental health specialists in either their office/private practice or another suitable, designated room at home. For every video consultation, patients received a transaction authentication number to log on to the encrypted, web-based videoconferencing platform for clinical video consultations. As the platform was easy to access, patients who had different levels of experience with videoconferencing had no major difficulties with logging in. Each mental health specialist was permanently assigned to one primary care practice. After the third session, we conducted an interim evaluation of the symptoms (using the PHQ-9 and GAD-7) and sent the results to the mental health specialist to tailor the treatment accordingly. After the last consultation with the patient, the mental health specialist sent a written case summary to the primary care practice, which was then attached to the medical record and on which, if needed, further decisions on follow-up procedures were based. Parallel to the study, mental health specialists received weekly group supervision led by a senior consultant in psychiatry and psychosomatic medicine from the Department of General Practice and Psychosomatics, Heidelberg University. Patients allocated to the control group were informed that they would receive the usual care provided by their primary care physicians. This might or might not have included a referral to a mental health specialist or other psychosocial treatment outside the study. The respective primary care physician was also informed about the group to which the patient was allocated. There were no restrictions on the usual treatment by primary care physicians.

    Outcomes

    The main outcome was the feasibility of a mental health care model integrating mental health specialist video consultations and primary care, which we operationalized by applying early stage implementation outcomes [37]:

    • Recruitment strategy and recruitment rate (efficiency of recruitment strategies).
    • Intervention acceptability in patients (attendance of sessions for the intervention arm).
    • Acceptability of outcome measurements (rate of loss to follow-up and feedback after assessments).
    • Intervention safety in patients (Inventory for the Assessment of Negative Effects of Psychotherapy [INEP]).
    • Feasibility of study procedures, including the intervention (qualitative process evaluation will be published elsewhere).

    In addition to feasibility, the measurements of effectiveness were also included. Effectiveness outcomes were depressive (PHQ-9) and anxiety (GAD-7) symptom severity, burden of specific somatic complaints (Somatic Symptom Disorder-B Criteria Scale [SSD-12]) [38], and recovery (Recovery Assessment Scale [RAS-G]), defined as “the personal process of adaptation and development through which the individual overcomes the negative personal and social consequences of [a] mental disorder and regains a self-determined and meaningful life” [39] consisting of 5 subdomains (more details on the domains are given in Multimedia Appendix 2 [40]) and “the quality and patient-centeredness of chronic illness care” (Patient Assessment of Chronic Illness Care [PACIC]) [41]. Health-related quality of life was measured using the European Quality of Life 5 Dimensions [42]. This also included a visual analog scale ranging from 0 to 100, on which the patients rated their quality of life with 0 for the lowest quality and 100 for the highest quality. Intervention-related costs and health care usage, including use of service and medication prescribing, were measured using the Questionnaire for the Assessment of Medical and Nonmedical Resource Utilization in Mental Disorders [43]. All these outcomes were assessed at baseline and 16 weeks postallocation. By choosing this period, we sought to (1) ensure that the intervention was completed despite possible delays (eg, because of time-consuming appointment management, patients’ and providers’ vacations) and (2) be able to assess not only immediate but also long-term effects. For the intervention group, intervention safety (unintended consequences and adverse effects) was assessed during close-out measurement by applying INEP [44]. INEP comprises 21 items asking the participant how they assess the effects of a psychosocial intervention.

    Statistical Analysis

    We based the sample size on recommendations for obtaining reliable sample size estimates in feasibility studies, which indicated that 50 patients would be needed (ie, 25 in each group) [45]. The primary analysis followed the intention-to-treat principle.

    First, as part of the data preparation, we applied an available-item strategy to calculate the total scale scores [46]. In this feasibility trial, we used pairwise deletion as a missing data strategy and did not adjust for multiple testing in the analyses. Second, we computed descriptive statistics for the feasibility outcomes, summarizing results for discrete variables in absolute and relative frequencies, and for continuous variables in means, SDs, medians, and IQRs. Third, we conducted assumption checks (screening for normality and equality of variances) for all variables of effectiveness outcomes. To investigate differences in effectiveness outcomes, we compared the change between baseline assessment and postassessment of PHQ-9, GAD-7, RAS-G, SSD-12, and PACIC in both groups using Mann-Whitney U tests [47,48]. We applied the screening values for computing the PHQ-9 and GAD-7 change scores if the baseline assessment had been performed no later than 28 days after screening. To increase interpretability, change scores were calculated by taking the difference between baseline assessment and postassessment scores or between postassessment and baseline assessment scores, depending on the respective scale. Therefore, a positive change indicates an improvement between the baseline assessment and postassessment. For the effect size r (rank-biserial correlation coefficient), the following interpretation applies: if r>0 in the baseline or follow-up scores, the health status in the intervention group was better than that in the control group. If r>0 for the change score, the improvement in the intervention group was larger than that in the control group.

    We used R (version 4.0.2), JASP (version 0.12.2) [49,50], and Stata (version 15.1) for all analyses. This trial was prospectively registered with the German Clinical Trials Register (registration no. DRKS00015812). We did not implement any changes to the methods after trial commencement. We have reported this trial in accordance with the CONSORT (Consolidated Standards of Reporting Trials) extension for randomized pilot and feasibility trials (see the checklist given in Multimedia Appendices 3 and 4) [51].


    Sample Description

    Of the 70 approached primary care practices, 12 were interested in participation. This relatively low rate may be explained by the fact that the provision of a designated room for a fixed time slot of 4 hours per week and a stable internet connection were mentioned as obligatory inclusion criteria. Outside the fixed time slot, the practice could use the room for routine clinical care. Some practices might not have been able to meet these requirements; therefore, they did not reply in the first place. Supporting this assumption, a preimplementation survey among primary care practitioners showed that more than half of them had no designated room available for video consultations [52]. After screening, we included 5 practices. Reasons for exclusion were a lack of designated rooms and/or internet connectivity. We recruited 50 participants—23 were randomized to mental health specialist video consultations and 27 to treatment-as-usual (Figure 1; Table 1). A total of 96% (48/50) participants had at least moderate levels of both depressive (PHQ-9≥10) and anxiety (GAD-7≥10) symptom severity, whereas 4% (2/50) participants were affected by moderate levels of depressive symptom severity only.

    ‎
    Figure 1. CONSORT (Consolidated Standards of Reporting Trials) flow diagram.
    View this figure
    Table 1. Baseline characteristics (N=50).
    VariableIntervention group (n=23)Control group (n=27)Overall (N=50)
    Age (years)

    		Mean (SD)45.9 (15.86)51.2 (15.46)48.8 (15.71)

    		Median (range)48 (22-72)56 (18-72)54 (18-72)
    Gender, n (%)

    		Female16 (69.6)19 (70.4)35 (70)

    		Male7 (30.4)8 (29.6)15 (30)
    Marital status, n (%)

    		Single5 (21.7)5 (18.5)10 (20)

    		In partnership18 (78.3)22 (81.5)40 (80)
    Education level, n (%)

    		9 years or less6 (26.1)14 (51.9)20 (40)

    		More than 9 years15 (65.2)13 (48.1)28 (56)

    		Missing2 (8.7)0 (0)2 (4)
    Employment status, n (%)

    		Employed13 (56.5)9 (33.3)22 (44)

    		On sick leave3 (13)5 (18.5)8 (16)

    		Retired4 (17.4)5 (18.5)9 (18)

    		Unemployed2 (8.7)2 (7.4)4 (8)

    		Missing1 (4.3)6 (22.2)7 (14)
    Number of chronic diseases

    		Mean (SD)0.9 (1.06)1.4 (1.33)1.1 (1.22)

    		Median (range)1 (0-3)1 (0-4)1 (0-4)

    		Missing0 (0)1 (3.7)1 (2)
    Current psychiatric treatment or psychotherapy, n (%)

    		No20 (87)21 (77.8)41 (82)

    		Yes3 (13)6 (22.2)9 (18)
    Past psychiatric treatment or psychotherapy, n (%)

    		No7 (30.4)9 (33.3)16 (32)

    		Yes13 (56.5)12 (44.4)25 (50)

    		Declined to answer2 (8.7)3 (11.1)5 (10)

    		Missing1 (4.3)3 (11.1)4 (8)
    Current psychopharmacological treatment, n (%)

    		No15 (65.2)14 (51.9)29 (58)

    		Yes8 (34.8)12 (44.4)20 (40)

    		Missing0 (0)1 (3.7)1 (2)
    Past psychopharmacological treatment, n (%)

    		No9 (39.1)12 (44.4)21 (42)

    		Yes5 (21.7)4 (14.8)9 (18)

    		Declined to answer5 (21.7)5 (18.5)10 (20)

    		Missing4 (17.4)6 (22.2)10 (20)
    Willingness to accept psychotherapy, n (%)

    		Disagree1 (4.3)1 (3.7)2 (4)

    		Agree4 (17.4)3 (11.1)7 (14)

    		Strongly agree17 (73.9)20 (74.1)37 (74)

    		Missing1 (4.3)3 (11.1)4 (8)
    Willingness to accept psychopharmacological treatment, n (%)

    		Strongly disagree5 (21.7)4 (14.8)9 (18)

    		Disagree6 (26.1)5 (18.5)11 (22)

    		Agree5 (21.7)3 (11.1)8 (16)

    		Strongly agree6 (26.1)9 (33.3)15 (30)

    		Missing1 (4.3)6 (22.2)7 (14)
    Level of depressive symptoms (PHQ-9a), n (%)

    		Blank1 (4.3)0 (0)1 (2)

    		Mild1 (4.3)4 (14.8)5 (10)

    		Moderate17 (73.9)10 (37)27 (54)

    		Severe3 (13)11 (40.7)14 (28)

    		Highly severe1 (4.3)2 (7.4)3 (6)
    Level of anxiety (GAD-7b), n (%)

    		Blank2 (8.7)2 (7.4)4 (8)

    		Mild8 (34.8)5 (18.5)13 (26)

    		Moderate10 (43.5)12 (44.4)22 (44)

    		Severe3 (13)8 (29.6)11 (22)

    aPHQ-9: Patient Health Questionnaire-9.

    bGAD-7: Generalized Anxiety Disorder-7.

    Recruitment, Rate of Loss to Follow-Up, and Success of Blinded Assessment

    The overall recruitment yield (number randomized per number screened) was 69% (50/73), the recruitment rate (number recruited and randomized per primary care practice per month) was 50/(4×7+1×5)=1.52. The consent rate (number randomized per number eligible) was 86% (50/58). We did not have to employ any additional recruitment routes in addition to direct recruitment by primary care physicians. With 1 dropout in the intervention group (1/23, 4% could not be reached/reason unknown) and 4 dropouts in the control group (4/27, 15%; reasons: 3 lost interest and 1 could not be reached/reasons unknown), follow-up rates were 95.7% and 85.2% for the intervention and control groups, respectively. The overall follow-up rate was 90%. Unintentional unblinding of the actual randomly assigned group during postmeasurement occurred in 13% (4/45) of retained cases. Of these, 5 blind breaks were in the video consultation group and 1 was in the control group. The period of recruitment and intervention was between March 1 and October 7, 2019. The last follow-up measurement was conducted on February 10, 2020.

    Acceptability of Treatment, Attendance at Sessions, and Reasons for Dropout

    Retention in the video consultation group was reasonable, with 87.0% (20/23) of the participants completing the intervention as planned (regardless of availability of follow-up data). In total, 8.7% (2/23) participants attended only the first 2 sessions (1 experienced persistent connectivity failures; 1 expected long-term therapy and was dissatisfied with the length of the intervention). Of the 23 participants, 1 (4.3%) stopped after the third session for unknown reasons. Participants who were allocated to the 50-minute video consultation received an average of 4.4 sessions (SD 0.9; range 2-5 consultations). Of the 108 planned video consultations, 102 (94.4%) were successfully delivered. For completers, the median interval between the initial and final video consultation amounted to 49.5 days (range 21-70 days). In the intervention group, 35% (8/23) of the patients received some form of specialist mental health care (defined as at least one visit to a psychiatrist or psychotherapist as measured on the Questionnaire for the Assessment of Medical and Nonmedical Resource Utilization in Mental Disorders) outside the study. Of the 27 participants in the control group, 12 (44%) received some form of specialist mental health care. The patients had no major difficulties in responding to the questionnaires applied as part of the outcome measurement. They evaluated the assessments as feasible and appropriate. Considering all data for the change from baseline scores at follow-up, the highest fraction of missing information was found for the No Domination by Symptoms domain of recovery (RAS-G), amounting to 7% (3/45) of cases.

    Effectiveness and Health Economic Outcomes

    The findings for the effectiveness outcomes and health economics are presented in Multimedia Appendices 5 and 6 [43,53], respectively. Change from baseline scores for the “No Domination by Symptoms” domain of recovery (RAS-G) were somewhat higher at postmeasurement for the video consultation group (mean change score 1.8, SD 2.56) compared with the control group (mean change score: 0.9, SD: 2.30; Mann-Whitney U test: rank-biserial r=0.19; 95% CI −0.09 to 0.46, 75% CI 0.02-0.35, P=.18). We did not detect any notable differences between the intervention group and the control group for the other effectiveness outcomes. Regarding the use of services outside the trial, the number of psychiatric outpatient clinic contacts seems to be larger at follow-up than at baseline for both groups. However, only 3 individuals in the intervention group had 21 contacts. The 7 contacts in the control group were induced by 2 individuals. The sum of provided specialist mental health care by psychotherapists, specialists in psychosomatic medicine, and psychiatrists is larger in the control group (baseline: 29; follow-up: 61) than in the intervention group (baseline: 7; follow-up: 38), which is again driven by few individuals (individuals of the control group with at least one specialist mental health care contact at baseline [n=6] and at follow-up [n=12]; individuals of the intervention group with at least one specialist mental health care contact at baseline [n=6] and at follow-up [n=8]).

    Unintended Consequences and Adverse Effects

    Self-report data for unintended consequences and adverse effects, as measured on the INEP at 16 weeks postallocation, were available for 96% (22/23) of the participants assigned to video consultations. Considering all 21 INEP items, 18% (4/22) of these participants reported at least one unintended consequence or adverse effect attributed to the intervention instead of their life circumstances (average number of adverse effects per patient 0.3, SD 0.7). One participant reported that she or he “feels worse” at the end of the intervention and that they were depending too much on their mental health specialist. A second participant stated that they “felt hurt” by the mental health specialist’s statements and that they experienced longer periods of feeling down during or after the intervention. A third participant indicated that they feared that colleagues could find out about them being in treatment and that they experienced longer periods of feeling down during or after the intervention. A fourth participant reported being affected “from events in her/his past more than in the time before the intervention.” Of the 22 participants in the intervention group for whom data were available, 18 (81%) did not report any unintended consequences or adverse effects attributed to the intervention. Notably, we did not observe any serious adverse events (ie, sexual harassment by mental health specialists, self-endangerment, and/or endangerment to others).


    Principal Findings

    In this assessor-blinded, randomized controlled feasibility trial, we found that a study comparing mental health specialist video consultations and treatment-as-usual by primary care physicians is feasible in people presenting with depression and/or anxiety in primary care. The feasibility of a subsequent definitive randomized controlled trial providing robust information on effectiveness is underscored by a reasonable recruitment yield, the high level of consent among eligible patients, and most importantly high levels of intervention acceptability and a low rate of loss to follow-up, which was slightly more pronounced in the control group. We attribute this to the integration of mental health specialists and primary care physicians, which accounted for seamless referrals from primary care to specialized care. Mental health specialist video consultations were generally safe and well accepted by both patients and health professionals. Although this feasibility trial was not formally powered to assess the evidence of a clinical response, the preliminary outcome data point to the benefits of being empowered to cope with symptoms. For the remaining outcomes, we did not find notable differences between the intervention and control groups, which may be explained by the fact that more patients in the control group had already been receiving psychiatric treatment, psychotherapy, and/or psychopharmacological treatment at enrollment compared with the intervention group. Even if we have found significant differences in our feasibility trial, it would have been inappropriate to interpret them as such because of the small and not formally calculated sample size in pilot or feasibility trials [54]. As our feasibility study covered several aspects of a full-scale randomized controlled trial and we present different outcomes, it is similar to a pilot trial. However, because our main objective was to test the feasibility of a mental health care model and to find aspects that may improve the implementation in the upcoming main trial, this study meets the characteristics of a feasibility trial. In addition, the fact that we conducted a parallel qualitative process evaluation indicates a feasibility trial [55].

    Limitations

    This feasibility trial had several limitations. First, with respect to generalizability, we had to draw on a nonprobability sample for all participants, including practices and mental health specialists. In this regard, we cannot fully rule out volunteer bias, that is, participating stakeholders exhibiting a higher openness toward web-based delivery of care compared with the respective underlying population. However, at this stage, our main goal is to evaluate feasibility, which usually builds on the motivation and engagement of innovators who are less reluctant to depart from the conventional paradigm of face-to-face clinical encounters. Some authors have explicitly encouraged trialists to focus on innovators as opposed to losing time on so-called laggards in the pilot phase of telepsychiatry programs [56]. Second, we did not systemically observe or measure fidelity to the intervention as laid out in the intervention manual to prevent implementation failure because we regarded video and/or audio recording of the sessions as too disruptive for the therapeutic process [57,58]. Although we cannot fully rule out inadequate implementation, together with the supervisor, the principal investigator (M Haun) did assess the content of the sessions in the weekly supervision. However, in the sufficiently powered effectiveness trial, we will implement a systematic self-report fidelity assessment for mental health specialists at the end of each video consultation, enabling us to determine the extent to which the results will be because of the study intervention and to further increase statistical power [59]. Most importantly, we will monitor and foster continuous adherence to the fidelity plan throughout the trial. Third, patients’ intervention acceptability was measured by the number of sessions attended. However, patients might have attended mental health video consultations despite finding them not useful. As this was the only measure of patients’ acceptability, more detailed statements on how the patients evaluate the consultations will not be available until the results of the qualitative process evaluation are published. Fourth, it is not clear to which degree participants in the intervention group received more attention than those in the control group. A potential clinical improvement in the intervention group therefore does not necessarily have to be caused by the intervention but may be because of greater attention. However, following the recommendation for pragmatic trials in mental health services research, the definition of the control condition as treatment-as-usual was deliberately broad and was supposed to be as equal as possible to routine care [60]. Therefore, we tried not to interact with the patients at all and did not assess more information about the potential attention they might have received, for example, by using other health services during the trial. Nevertheless, in the sufficiently powered main trial, we will include health care service use in subgroup analyses with respect to attention received by control group subjects. Fifth, during the trial, some patients in both groups received other psychosocial care. As the intervention aimed to provide triage and, if indicated, facilitate the transition to specialist mental health care and the control condition was defined as treatment-as-usual, the use of treatment outside the trial was not excluded. As described, we did not collect data on the use of services in great detail and therefore did not include those in the analysis of clinical outcomes. However, we will include these data in subgroup analyses in a sufficiently powered main trial to investigate the potential clinical impact of psychosocial services use.

    Comparison With Previous Work

    The findings of this trial concur with results on feasibility from previous trials and synthesized findings from reviews. A large systematic review on telehealth interventions in mental health analyzed 5 full-scale randomized controlled trials using video consultations for various mental health conditions in settings other than primary care. In all of these trials, video consultations were reported as well accepted by different populations and under different conditions [61-64]. The few trials that specifically integrated mental health specialist video consultations in primary care also yielded substantial acceptance of and satisfaction with this new form of technology-based care [19,20,65,66]. Nevertheless, all these trials were conducted in the United States and/or drew on samples from specific, in part, high-structured contexts (eg, military, including veterans). It is very likely that the patient population in primary care and the contextual factors of how primary care is organized (eg, single-practitioner models in Europe) differ in many other Western countries [12,67]. Specifically, neither collaborative care nor integrated care models are commonly used in Germany or other European health care systems. Thus, our intervention comprising the integration of specialized and primary care combined with the video-based mode of delivery can be considered relatively innovative. In this regard, the findings of our trial, that is, high retention, no major adverse effects, and high satisfaction, show that video-based integrated care models are feasible more broadly, even in health care systems with a low level of experience in integrated care. Against the background of the debate on which patient populations video consultations might be suitable for, particularly pertaining to older aged and/or severely burdened patients [68], our sample proved to be quite heterogeneous (eg, in terms of age and socioeconomic status). Overall, the findings of this feasibility trial indicate that even severely burdened patients can be reached through mental health specialist video consultations in primary care. In this regard, our intervention involved patients from difficult-to-reach populations who might have never been engaged in specialized treatment following conventional care pathways [69,70]. Indeed, 1 in 3 of the participants in our study had never sought specialized mental health treatment before enrollment in our trial. An additional strength of this feasibility trial was the innovative, systematic assessment of adverse and negative effects and harms and their potential attribution to the intervention itself using a validated self-report instrument. Although calls for such an assessment in clinical trials are continuously put forward, there is some evidence that in the field of psychotherapy, only a small proportion of studies actually report unintended consequences or adverse effects [71]. We found that 18.2% of all participants (average number of adverse effects per patient 0.3, SD 0.7) reported at least one unintended consequence or adverse effect attributed to the intervention, which is (1) much less than the prevalence of 70.5% for INEP in a clinical sample (average number of adverse effects per patient 2.1, SD 2.2) [72] and (2) well within the range of 0% to 25% reported for intervention groups in psychotherapy trials [71]. However, in digital health interventions, the impact of the patient-clinician relationship has scarcely been investigated [73]. Therefore, it is not clear whether unintended consequences or adverse effects are caused by the mode of delivery through videoconferencing or by failed rapport between the patient and clinician. At any rate, technology-supported interventions are challenging for the patient-clinician relationship, and this requires investigation regarding the negative or adverse effects of psychotherapy. Notwithstanding, interpreting unintended consequences or adverse effects remains to be a unique challenge in psychotherapy interventions, where the sound delivery of treatment may nevertheless be linked to patients reporting such effects [74].

    Conclusions

    A study comparing mental health specialist video consultations and treatment-as-usual by primary care physicians in patients with depressive and anxiety disorders is feasible. The main implication of this trial is that a sufficiently powered effectiveness trial is needed to provide evidence about the relative efficacy of mental health video consultations in primary care. In our trial, the intervention proved to be unobtrusive and compatible with normal practice. Participants from various socioeconomic and cultural backgrounds could be enrolled so that a definitive trial should aim more broadly at the primary care patient population by applying pragmatic eligibility criteria. Indeed, we have embarked on a full-scale effectiveness trial in which 320 patients will be enrolled (NCT04316572), which will also include a health economic evaluation. Having applied a conservative sample size calculation, we accounted for loss to follow-up by inflating the recruitment by 20%. From a clinical perspective, at present, it seems reasonable and safe to offer video consultations to patients who cannot assess specialist services using conventional pathways.

    Acknowledgments

    This research was supported by grants from the Federal Ministry of Education and Research (Bundesministerium fuer Bildung und Forschung, BMBF; grant no. 01GY1612). The funder was not actively involved in this study. The authors JT, RB, DW, and the corresponding author had full access to all the data in this study and take complete responsibility for the integrity of the data and the accuracy of the data analysis. The corresponding author had the final responsibility for the decision to submit for publication. This work was supported by the German Federal Ministry of Education and Research (BMBF; grant no. 01GY16129). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

    Authors' Contributions

    All authors contributed substantially to the conception, design, and analyses of this study. RB, DW, JT, MW, MH, AI, MV, and M Haun contributed to the data analysis plan. JT and M Haun collected and prepared data for analysis. RB, DW, and MV conducted all data analyses regarding feasibility and clinical and health economic outcomes. JT, RB, DW, MV, MH, and M Haun drafted the manuscript. All authors critically revised the manuscript for important intellectual content. All authors approved the version of the manuscript to be published and agreed to be accountable for all aspects of the work.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Structured description of the intervention.

    DOCX File , 19 KB
    ‎
    Multimedia Appendix 2

    Structured description of the domains of the Recovery Assessment Scale, German Version.

    DOCX File , 29 KB
    ‎
    Multimedia Appendix 3

    CONSORT (Consolidated Standards of Reporting Trials) checklist, an extension for randomized pilot and feasibility trials.

    DOCX File , 32 KB
    ‎
    Multimedia Appendix 4

    CONSORT (Consolidated Standards of Reporting Trials) eHealth checklist (V.1.6.1).

    PDF File (Adobe PDF File), 10077 KB
    ‎
    Multimedia Appendix 5

    Results on effectiveness outcomes.

    DOCX File , 36 KB
    ‎
    Multimedia Appendix 6

    Detailed health economic results (European Quality of Life 5 Dimensions and the Questionnaire for the Assessment of Medical and Nonmedical Resource Utilization in Mental Disorders).

    DOCX File , 47 KB
    1. Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013 Nov;382(9904):1575-1586. [CrossRef]
    2. Wang PS, Demler O, Olfson M, Pincus HA, Wells KB, Kessler RC. Changing profiles of service sectors used for mental health care in the United States. AJP 2006 Jul;163(7):1187-1198. [CrossRef]
    3. Gaebel W, Kowitz S, Fritze J, Zielasek J. Use of health care services by people with mental illness: secondary data from three statutory health insurers and the German Statutory Pension Insurance Scheme. Dtsch Arztebl Int 2013 Nov 22;110(47):799-808 [FREE Full text] [CrossRef] [Medline]
    4. Linde K, Sigterman K, Kriston L, Rücker G, Jamil S, Meissner K, et al. Effectiveness of psychological treatments for depressive disorders in primary care: systematic review and meta-analysis. Ann Fam Med 2015;13(1):56-68 [FREE Full text] [CrossRef] [Medline]
    5. Cassano P, Fava M. Depression and public health: an overview. J Psychosom Res 2002 Oct 1;53(4):849-857 [FREE Full text] [CrossRef] [Medline]
    6. Cunningham PJ. Beyond parity: primary care physicians' perspectives on access to mental health care. Health Aff (Millwood) 2009 May;28(3):w490-w501. [CrossRef] [Medline]
    7. Unützer J, Harbin H, Schoenbaum M, Druss B. The Collaborative Care Model: An approach for integrating physical and mental health care in medicaid home. Hamilton, New Jersey: Center for Health Care Strategies; 2013.
    8. Coventry PA, Hudson JL, Kontopantelis E, Archer J, Richards DA, Gilbody S, et al. Characteristics of effective collaborative care for treatment of depression: a systematic review and meta-regression of 74 randomised controlled trials. PLoS One 2014 Sep 29;9(9):e108114 [FREE Full text] [CrossRef] [Medline]
    9. Crowley RA, Kirschner N, HealthPublic Policy Committee of the American College of Physicians. The integration of care for mental health, substance abuse, and other behavioral health conditions into primary care: executive summary of an American College of Physicians position paper. Ann Intern Med 2015 Aug 18;163(4):298-299. [CrossRef] [Medline]
    10. Kanapaux W. The road to integrated care: commitment is the key; Tennessee CMHC demonstrates promise of co-located behavioral and primary care. Behav Healthc Tomorrow 2004;13(2):15.
    11. Reiter JT, Dobmeyer AC, Hunter CL. The Primary Care Behavioral Health (PCBH) Model: An Overview and Operational Definition. J Clin Psychol Med Settings 2018 Jun 26;25(2):109-126. [CrossRef] [Medline]
    12. Schoen C, Osborn R, Doty MM, Squires D, Peugh J, Applebaum S. A survey of primary care physicians in eleven countries, 2009: perspectives on care, costs, and experiences. Health Aff (Millwood) 2009 Jan;28(6):w1171-w1183. [CrossRef] [Medline]
    13. Hilty DM, Rabinowitz T, McCarron RM, Katzelnick DJ, Chang T, Bauer AM, et al. An update on telepsychiatry and how it can leverage collaborative, stepped, and integrated services to primary care. Psychosomatics 2018 May;59(3):227-250. [CrossRef] [Medline]
    14. Shore JH. Telepsychiatry: videoconferencing in the delivery of psychiatric care. Am J Psychiatry 2013 Mar;170(3):256-262. [CrossRef] [Medline]
    15. Berryhill MB, Culmer N, Williams N, Halli-Tierney A, Betancourt A, Roberts H, et al. Videoconferencing psychotherapy and depression: a systematic review. Telemed J E Health 2019 Jun;25(6):435-446. [CrossRef] [Medline]
    16. Berryhill MB, Halli-Tierney A, Culmer N, Williams N, Betancourt A, King M, et al. Videoconferencing psychological therapy and anxiety: a systematic review. Fam Pract 2019 Jan 25;36(1):53-63. [CrossRef] [Medline]
    17. Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: a 2013 review. Telemed J E Health 2013 Jun;19(6):444-454 [FREE Full text] [CrossRef] [Medline]
    18. Bashshur RL, Shannon GW, Bashshur N, Yellowlees PM. The empirical evidence for telemedicine interventions in mental disorders. Telemed J E Health 2016 Feb;22(2):87-113 [FREE Full text] [CrossRef] [Medline]
    19. Fortney JC, Pyne JM, Edlund MJ, Williams DK, Robinson DE, Mittal D, et al. A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med 2007 Aug;22(8):1086-1093 [FREE Full text] [CrossRef] [Medline]
    20. Fortney JC, Pyne JM, Mouden SB, Mittal D, Hudson TJ, Schroeder GW, et al. Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: a pragmatic randomized comparative effectiveness trial. Am J Psychiatry 2013 Apr;170(4):414-425 [FREE Full text] [CrossRef] [Medline]
    21. Fortney JC, Pyne JM, Turner EE, Farris KM, Normoyle TM, Avery MD, et al. Telepsychiatry integration of mental health services into rural primary care settings. Int Rev Psychiatry 2015 Dec;27(6):525-539. [CrossRef] [Medline]
    22. Shore J. The evolution and history of telepsychiatry and its impact on psychiatric care: Current implications for psychiatrists and psychiatric organizations. Int Rev Psychiatry 2015 Sep 23;27(6):469-475. [CrossRef] [Medline]
    23. Morriss R, Patel S, Malins S, Guo B, Higton F, James M, et al. Clinical and economic outcomes of remotely delivered cognitive behaviour therapy versus treatment as usual for repeat unscheduled care users with severe health anxiety: a multicentre randomised controlled trial. BMC Med 2019 Jan 23;17(1):16 [FREE Full text] [CrossRef] [Medline]
    24. Scogin F, Lichstein K, DiNapoli EA, Woosley J, Thomas SJ, LaRocca MA, et al. Effects of integrated telehealth-delivered cognitive-behavioral therapy for depression and insomnia in rural older adults. J Psychother Integr 2018 Sep;28(3):292-309 [FREE Full text] [CrossRef] [Medline]
    25. Shore JH, Yellowlees P, Caudill R, Johnston B, Turvey C, Mishkind M, et al. Best practices in videoconferencing-based telemental health april 2018. Telemed J E Health 2018 Nov;24(11):827-832. [CrossRef] [Medline]
    26. Tönnies J, Hartmann M, Wensing M, Szecsenyi J, Icks A, Friederich H, et al. Mental health specialist video consultations for patients with depression or anxiety disorders in primary care: protocol for a randomised controlled feasibility trial. BMJ Open 2019 Sep 4;9(9):e030003 [FREE Full text] [CrossRef] [Medline]
    27. Hoffmann M, Hartmann M, Wensing M, Friederich H, Haun MW. Potential for integrating mental health specialist video consultations in office-based routine primary care: cross-sectional qualitative study among family physicians. J Med Internet Res 2019 Aug 19;21(8):e13382 [FREE Full text] [CrossRef] [Medline]
    28. Chilcot J, Hudson JL, Moss-Morris R, Carroll A, Game D, Simpson A, et al. Screening for psychological distress using the patient health questionnaire anxiety and depression scale (phq-ads): initial validation of structural validity in dialysis patients. Gen Hosp Psychiatry 2018 Jan;50:15-19. [CrossRef] [Medline]
    29. Bower P, Gask L. The changing nature of consultation-liaison in primary care: bridging the gap between research and practice. General Hospital Psychiatry 2002 Mar;24(2):63-70. [CrossRef]
    30. Archer J, Bower P, Gilbody S, Lovell K, Richards D, Gask L, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev 2012 Oct 17;10:CD006525. [CrossRef] [Medline]
    31. Gunn J, Diggens J, Hegarty K, Blashki G. A systematic review of complex system interventions designed to increase recovery from depression in primary care. BMC Health Serv Res 2006 Jul 16;6(1):88 [FREE Full text] [CrossRef] [Medline]
    32. Bower P, Gilbody S. Managing common mental health disorders in primary care: conceptual models and evidence base. Br Med J 2005 Apr 9;330(7495):839-842 [FREE Full text] [CrossRef] [Medline]
    33. Craske MG. Transdiagnostic treatment for anxiety and depression. Depress Anxiety 2012 Sep;29(9):749-753. [CrossRef] [Medline]
    34. Newby J, McKinnon A, Kuyken W, Gilbody S, Dalgleish T. Systematic review and meta-analysis of transdiagnostic psychological treatments for anxiety and depressive disorders in adulthood. Clin Psychol Rev 2015 Aug 1;40:91-110 [FREE Full text] [CrossRef]
    35. Zhang A, Franklin C, Jing S, Bornheimer L, Hai A, Himle J, et al. The effectiveness of four empirically supported psychotherapies for primary care depression and anxiety: A systematic review and meta-analysis. J Affect Disord 2019 Feb 15;245:1168-1186 [FREE Full text] [CrossRef]
    36. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J 2014 Mar 7;348(mar07 3):g1687. [CrossRef] [Medline]
    37. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar 19;38(2):65-76 [FREE Full text] [CrossRef] [Medline]
    38. Toussaint A, Murray A, Voigt K, Herzog A, Gierk B, Kroenke K, et al. Development and Validation of the Somatic Symptom Disorder-B Criteria Scale (SSD-12). Psychosom Med 2016;78(1):5-12. [CrossRef]
    39. Cavelti M, Wirtz M, Corrigan P, Vauth R. Recovery assessment scale: Examining the factor structure of the German version (RAS-G) in people with schizophrenia spectrum disorders. Eur Psychiatr 2016 Dec 31;41(1):60-67. [CrossRef]
    40. Corrigan PW, Salzer M, Ralph RO, Sangster Y, Keck L. Examining the factor structure of the recovery assessment scale. Schizophr Bull 2004;30(4):1035-1041. [CrossRef] [Medline]
    41. Goetz K, Freund T, Gensichen J, Miksch A, Szecsenyi J, Steinhaeuser J. Adaptation and psychometric properties of the PACIC short form. Am J Manag Care 2012 Feb 01;18(2):e55-e60. [Medline]
    42. Greiner W, Claes C, Busschbach JJ, von der Schulenburg JM. Validating the EQ-5D with time trade off for the German population. Eur J Health Econ 2005 Jun;6(2):124-130. [CrossRef] [Medline]
    43. Grupp H, König HH, Riedel-Heller S, Konnopka A. [FIMPsy - Questionnaire for the Assessment of Medical and non Medical Resource Utilisation in Mental Disorders: Development and Application]. Psychiatr Prax 2018 Mar 26;45(2):87-94. [CrossRef] [Medline]
    44. Ladwig I, Rief W, Nestoriuc Y. Welche Risiken und Nebenwirkungen hat Psychotherapie? - Entwicklung des Inventars zur Erfassung Negativer Effekte von Psychotherapie (INEP). Verhaltenstherapie 2014 Oct 30;24(4):252-263. [CrossRef]
    45. Justifying Sample Size for a Feasibility Study. National Insitute for Health Research. 2019 Feb 22.   URL: https://www.rds-london.nihr.ac.uk/resources/justify-sample-size-for-a-feasibility-study/ [accessed 2021-02-26]
    46. Parent MC. Handling Item-Level Missing Data. The Counseling Psychologist 2012 May 10;41(4):568-600. [CrossRef]
    47. Lee EC, Whitehead AL, Jacques RM, Julious SA. The statistical interpretation of pilot trials: should significance thresholds be reconsidered? BMC Med Res Methodol 2014 Mar 20;14:41 [FREE Full text] [CrossRef] [Medline]
    48. Vickers AJ, Altman DG. Statistics notes: analysing controlled trials with baseline and follow up measurements. Br Med J 2001 Nov 10;323(7321):1123-1124 [FREE Full text] [CrossRef] [Medline]
    49. JASP Team. JASP 0.12. 2021.   URL: https://jasp-stats.org [accessed 2021-02-26]
    50. R: A Language and Environment for Statistical Computing Internet. Vienna, Austria: R Foundation for Statistical Computing; 2021.   URL: https://www.R-project.org/ [accessed 2021-02-26]
    51. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ 2016 Oct 24;355:i5239 [FREE Full text] [CrossRef] [Medline]
    52. Haun MW, Stephan I, Wensing M, Hartmann M, Hoffmann M, Friederich H. Intent to Adopt Video-Based Integrated Mental Health Care and the Characteristics of its Supporters: Mixed Methods Study Among General Practitioners Applying Diffusion of Innovations Theory. JMIR Ment Health 2020 Oct 15;7(10):e23660 [FREE Full text] [CrossRef] [Medline]
    53. Ludwig K, Graf von der Schulenburg JM, Greiner W. German value set for the EQ-5D-5L. Pharmacoeconomics 2018 Jun;36(6):663-674 [FREE Full text] [CrossRef] [Medline]
    54. Sim J. Should treatment effects be estimated in pilot and feasibility studies? Pilot Feasibility Stud 2019 Aug 28;5(1):107 [FREE Full text] [CrossRef] [Medline]
    55. Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One 2016 Mar 15;11(3):e0150205 [FREE Full text] [CrossRef] [Medline]
    56. Yellowlees P, Shore J. Telepsychiatry and Health Technologies. Arlington, Virginia: American Psychiatric Association; 2018.
    57. Perepletchikova F, Kazdin A. Treatment integrity and therapeutic change: issues and research recommendations. Clin Psychol Sci Pract 2006;12(4):365-383. [CrossRef]
    58. Allen J, Shelton R, Emmons K, Linnan L. Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination. In: Browson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. New York, NY: Oxford University Press; 2018:267-284.
    59. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol 2004 Sep;23(5):443-451. [CrossRef] [Medline]
    60. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. Br Med J 2015 May 8;350:h2147. [Medline]
    61. Bashshur RL, Howell JD, Krupinski EA, Harms KM, Bashshur N, Doarn CR. The Empirical Foundations of Telemedicine Interventions in Primary Care. Telemed J E Health 2016 May;22(5):342-375 [FREE Full text] [CrossRef] [Medline]
    62. Trief PM, Morin PC, Izquierdo R, Teresi JA, Eimicke JP, Goland R, et al. Depression and glycemic control in elderly ethnically diverse patients with diabetes: the IDEATel project. Diabetes Care 2006 Apr 27;29(4):830-835. [CrossRef] [Medline]
    63. Rabinowitz T, Murphy KM, Amour JL, Ricci MA, Caputo MP, Newhouse PA. Benefits of a telepsychiatry consultation service for rural nursing home residents. Telemed J E Health 2010 Feb;16(1):34-40. [CrossRef] [Medline]
    64. De Las Cuevas C, Arredondo MT, Cabrera MF, Sulzenbacher H, Meise U. Randomized clinical trial of telepsychiatry through videoconference versus face-to-face conventional psychiatric treatment. Telemed J E Health 2006 Jun;12(3):341-350. [CrossRef] [Medline]
    65. Myers K, Vander Stoep A, Zhou C, McCarty CA, Katon W. Effectiveness of a telehealth service delivery model for treating attention-deficit/hyperactivity disorder: a community-based randomized controlled trial. J Am Acad Child Adolesc Psychiatry 2015 Apr;54(4):263-274 [FREE Full text] [CrossRef] [Medline]
    66. Fortney JC, Pyne JM, Kimbrell TA, Hudson TJ, Robinson DE, Schneider R, et al. Telemedicine-based collaborative care for posttraumatic stress disorder: a randomized clinical trial. JAMA Psychiatry 2015 Jan 01;72(1):58-67. [CrossRef] [Medline]
    67. Kringos DS, Boerma WGW, Hutchinson A, Saltman RB. Building primary care in a changing Europe case studies. 2015.   URL: http:/​/www.​euro.who.int/​en/​about-us/​partners/​observatory/​publications/​studies/​building-primary-care-in-a-changing-europe-case-studiesISBN:978-92-890-5033-3 [accessed 2021-02-26]
    68. Kocsis BJ, Yellowlees P. Telepsychotherapy and the Therapeutic Relationship: Principles, Advantages, and Case Examples. Telemed J E Health 2018 May;24(5):329-334. [CrossRef] [Medline]
    69. Ramos-Ríos R, Mateos R, Lojo D, Conn DK, Patterson T. Telepsychogeriatrics: a new horizon in the care of mental health problems in the elderly. Int Psychogeriatr 2012 Jun 12;24(11):1708-1724. [CrossRef]
    70. Bleyel C, Hoffmann M, Wensing M, Hartmann M, Friederich H, Haun MW. Patients' Perspective on Mental Health Specialist Video Consultations in Primary Care: Qualitative Preimplementation Study of Anticipated Benefits and Barriers. J Med Internet Res 2020 Apr 20;22(4):e17330 [FREE Full text] [CrossRef] [Medline]
    71. Cuijpers P, Reijnders M, Karyotaki E, de Wit L, Ebert DD. Negative effects of psychotherapies for adult depression: A meta-analysis of deterioration rates. J Affect Disord 2018 Oct 15;239:138-145. [CrossRef] [Medline]
    72. Abeling B, Müller A, Stephan M, Pollmann I, de Zwaan M. [Negative Effects of Psychotherapy: Prevalence and Correlates in a Clinical Sample]. Psychother Psychosom Med Psychol 2018 Sep 11;68(9-10):428-436. [CrossRef] [Medline]
    73. Parish MB, Fazio S, Chan S, Yellowlees PM. Managing Psychiatrist-Patient Relationships in the Digital Age: a Summary Review of the Impact of Technology-enabled Care on Clinical Processes and Rapport. Curr Psychiatry Rep 2017 Oct 27;19(11):90. [CrossRef] [Medline]
    74. Bloch-Elkouby S, Eubanks CF, Knopf L, Gorman BS, Muran JC. The Difficult Task of Assessing and Interpreting Treatment Deterioration: An Evidence-Based Case Study. Front Psychol 2019 Jul 3;10:1180 [FREE Full text] [CrossRef] [Medline]

    ‎
    GAD-7: Generalized Anxiety Disorder-7
    INEP: Inventory for the Assessment of Negative Effects of Psychotherapy
    PACIC: Patient Assessment of Chronic Illness Care
    PHQ-9: Patient Health Questionnaire-9
    PROVIDE-B: improving cross-sectoral collaboration between primary and psychosocial care: an implementation study on video consultations-B
    RAS-G: Recovery Assessment Scale-German
    SSD-12: Somatic Symptom Disorder-B Criteria Scale

    Edited by J Torous; submitted 16.07.20; peer-reviewed by P Yellowlees, J Li, R Krukowski, S Steingrimsson, N Kaur, C Jones; comments to author 07.10.20; revised version received 13.11.20; accepted 29.01.21; published 12.03.21

    Copyright

    ©Justus Tönnies, Mechthild Hartmann, Michel Wensing, Joachim Szecsenyi, Frank Peters-Klimm, Regina Brinster, Dorothea Weber, Markus Vomhof, Andrea Icks, Hans-Christoph Friederich, Markus W Haun. Originally published in JMIR Mental Health (http://mental.jmir.org), 12.03.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.