This paper is in the following e-collection/theme issue:

JMH Theme Issue: COVID-19 and Mental Health: Impact and Interventions (383) JMIR Theme Issue: COVID-19 Special Issue (2452) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1514) Outbreak and Pandemic Preparedness and Management (1727) Users' and Patients' Needs for Mental Health Services (403) Depression and Mood Disorders; Suicide Prevention (1363) Health Services Research (482) Telepsychiatry (159) Telehealth and Telemonitoring (1671)

Published on in Vol 8, No 7 (2021): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/26187, first published .
Tele–Mental Health for Reaching Out to Patients in a Time of Pandemic: Provider Survey and Meta-analysis of Patient Satisfaction

Tele–Mental Health for Reaching Out to Patients in a Time of Pandemic: Provider Survey and Meta-analysis of Patient Satisfaction

Tele–Mental Health for Reaching Out to Patients in a Time of Pandemic: Provider Survey and Meta-analysis of Patient Satisfaction

Authors of this article:

Raffaele Mazziotti1, 2 Author Orcid Image ;   Grazia Rutigliano3 Author Orcid Image
Article Authors Cited by (29) Tweetations (8) Metrics

    Original Paper

    1Institute of Neuroscience, National Research Council (CNR), Pisa, Italy

    2Department of Neuroscience, Psychology, Drug Research and Child Health (NEUROFARBA), University of Florence, Florence, Italy

    3Department of Pathology, University of Pisa, Pisa, Italy

    Corresponding Author:

    Grazia Rutigliano, MD, PhD

    Department of Pathology

    University of Pisa

    via Savi, 10

    Pisa, 56126

    Italy

    Phone: 39 3496117744

    Email: grazia.rutigliano.gr@gmail.com


    Background: The COVID-19 pandemic threatened to impact mental health by disrupting access to care due to physical distance measures and the unexpected pressure on public health services. Tele–mental health was rapidly implemented to deliver health care services.

    Objective: The aims of this study were (1) to present state-of-the-art tele–mental health research, (2) to survey mental health providers about care delivery during the pandemic, and (3) to assess patient satisfaction with tele–mental health.

    Methods: Document clustering was applied to map research topics within tele–mental health research. A survey was circulated among mental health providers. Patient satisfaction was investigated through a meta-analysis of studies that compared satisfaction scores between tele–mental health and face-to-face interventions for mental health disorders, retrieved from Web of Knowledge and Scopus. Hedges g was used as the effect size measure, and effect sizes were pooled using a random-effect model. Sources of heterogeneity and bias were examined.

    Results: Evidence on tele–mental health has been accumulating since 2000, especially regarding service implementation, depressive or anxiety disorders, posttraumatic stress disorder, and special populations. Research was concentrated in a few countries. The survey (n=174 respondents from Italy, n=120 international) confirmed that, after the onset of COVID-19 outbreak, there was a massive shift from face-to-face to tele–mental health delivery of care. However, respondents held skeptical views about tele–mental health and did not feel sufficiently trained and satisfied. Meta-analysis of 29 studies (n=2143) showed that patients would be equally satisfied with tele–mental health as they are with face-to-face interventions (Hedges g=−0.001, 95% CI −0.116 to 0.114, P=.98, Q=43.83, I2=36%, P=.03) if technology-related issues were minimized.

    Conclusions: Mental health services equipped with tele–mental health will be better able to cope with public health crises. Both providers and patients need to be actively engaged in digitization, to reshape their reciprocal trust around technological innovations.

    JMIR Ment Health 2021;8(7):e26187

    doi:10.2196/26187

    Keywords

    telepsychiatry; telepsychology; e-mental health; document clustering; survey; COVID-19; access to care; patient satisfaction; mental health; tele–mental health; review; telemedicine; satisfaction; access 


    In March 2020, a COVID-19 outbreak spread throughout the globe reaching the size of a pandemic. Most governments responded with physical distancing measures. In this science fiction–like context, mental health is expected to pay a heavy toll [1]. Paradoxically, in a time of increased mental health vulnerability, access to care had to be restricted by pausing nonurgent outpatient services, closing day centers, and reducing home visits [2-6]. There was one recourse to address reduced access to care—tele–mental health was used to reach out and support patients [7-9]. In most health care organizations, personnel had limited previous experience, and there were little or no regulations in place [10].

    The term tele–mental health refers to the remote delivery of mental health care using telecommunications, such as telephone, email, interactive video, digital imaging, and health care monitoring devices [10-12]. The evolution of tele–mental health can be modeled with the double-peak effect Gartner Hype Cycle [13], which describes the course of new technological discoveries integrating special or unusual circumstances (ie, the COVID-19 pandemic) (Figure 1). The Gartner Hype Cycle has served as a useful descriptive model in other medical fields, such as the ultra-high risk for psychosis paradigm [14,15]. According to the Gartner Hype Cycle, new technologies trigger inflated expectations in the short term, and in the long term, expectations are largely underestimated. For tele–mental health, the innovation trigger (stage 1) was the set-up, in 1959, of the first television links between the Nebraska Psychiatric Institute and the Norfolk State Hospital for providing therapy, consultation-liaison psychiatry, and medical student training. Over subsequent years, tele–mental health became increasingly common (stage 2, inflated expectations), expanding in scope to several diagnostic and therapeutic applications, and geographically, from the United States to other countries, in particular, to Australia and Canada. Much enthusiasm developed around tele–mental health’s ability to reach remote rural areas, which suffer from systemic mental health care shortages. By the 2000s, evidence on the use of tele–mental health had accumulated, demonstrating its (1) validity and efficacy in several mental disorders, (2) applicability to different patient populations (eg, war veterans, comorbid medical conditions) and age groups, (3) versatility (diverse cultures), and (4) ability to increase access to care [16]. Despite encouraging evidence and endorsement in clinical guidelines, the adoption of tele–mental health has been slow and scattered, owing to several barriers from clinicians’ perspectives, such as concerns regarding ability to establish a good doctor–patient rapport, confidentiality and data protection, safety, technology-related factors, and financial and legal aspects (stage 3, trough of disillusionment) [17]. The COVID-19 crisis has boosted the attention paid to tele–mental health. In an incredibly short time, a broad array of educational resources, toolkits, and guidelines have been made available. Mental health professionals from around the globe have joined forces and shared their experiences in an effort to provide the best care to patients during this terrible time. The digitization of the field of medicine has become a matter of public interest (stage 4, slope of enlightenment). We will find out, in the years to come, whether this unexpected massive public effort will crystallize into mental health service organization and resource allocation (stage 5, plateau of applicability).

    ‎
    Figure 1. The double-peak effect Gartner Hype Cycle describes the course of tele–mental health, integrating the COVID-19 pandemic as a special or unusual circumstance.
    View this figure

    In this study, we provide an analysis of state-of-the-art scientific publications on tele–mental health by applying document clustering to map prominent research topics in the field. We surveyed mental health professionals about their experiences of care delivery during the pandemic, especially regarding their use of and attitudes toward tele–mental health. Patients’ perspectives on tele–mental health were assessed through a systematic review and meta-analysis of satisfaction with tele–mental health compared to face-to-face interventions.


    Analysis of Scientific Publications

    Data Collection

    The publication search was performed using Scopus advanced search [18], with the following search formula:

    TITLE-ABS-KEY ( telepsychiatry ) OR TITLE-ABS-KEY ( telepsychology ) OR TITLE-ABS-KEY ( telepsychotherapy ) AND ( LIMIT-TO ( PUBSTAGE , “final” ) ) AND ( LIMIT-TO ( DOCTYPE , “ar” ) ) AND ( LIMIT-TO ( SRCTYPE , “j” ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) )”.

    We restricted the search to articles describing original research performed in the field of tele–mental health, while excluding review papers. Results (653 articles on June 4, 2020) were exported in .csv format, with as much information as possible.

    Data Analysis

    After excluding 212 articles for having no relation with tele–mental health (manual filtration) or no abstract, the remaining corpus of 441 articles was imported to Python (version 3; Pandas package, version 1.2.3). Abstracts and titles were concatenated and tokenized (NLTK package, version 3.5). After part-of-speech tagging (filtering only nouns, adjectives and verbs) and lemmatization, common stop words were removed, and stemming was performed. We calculated bigrams (gensim, version 3.8.1) and subsequently removed an array of stop words with broad meanings, such as “paper,” “method,” “analyze,” and other terms that appear in almost every paper. Each tokenized abstract was transformed into a numerical multidimensional representation (TfidfVectorizer, version 0.22.1), which transforms the tokens into an array of term frequency–inverse document frequency values. The similarity between documents was computed with cosine distance between term frequency–inverse document frequency vectors and visualized with t-distributed stochastic neighbor embedding, which was used to perform hierarchical density-based spatial clustering of applications with noise [19]. The code is freely available [20].

    Provider Survey

    We developed an Italian-language web-based survey targeting mental health providers to map (an English-language version was circulated in an international network of mental health providers.): (1) COVID-19–related disruptions in care provision; (2) tele–mental health use during and prior to the COVID-19 pandemic; (3) tele–mental health intention-to-use; and (4) attitude toward tele–mental health. Sociodemographic (age and gender), employment role, setting, and geographic area information was collected. The survey, designed to be completed within 10 to 15 minutes, consisted of 6 sections, with 21 multiple- or forced-choice questions and 15 Likert-scale questions. The survey was shared through email invitations and social media. The survey remained open for 20 days (from May 30, 2020 to June 20, 2020). All respondents provided informed consent.

    Meta-analysis of Patients’ Satisfaction With Tele–Mental Health Interventions

    Search Strategy and Selection Criteria

    A systematic review and meta-analysis were conducted based on the Population, Intervention, Comparisons, Outcomes and Study Design (PICOS [21]) strategy. We used a 2-step search strategy. First, we searched the Web of Knowledge (Thomson Reuters) and Scopus databases, using the following terms:

    (telepsychiatry OR telepsychiatric OR telepsychology OR teletherapy OR tele–mental health OR e-mental) AND (satisfaction).

    The search was extended until June 10, 2020. Second, we implemented an electronic manual search of the reference lists of the retrieved articles. Duplicate references were manually removed. Articles were screened by title and abstract, and the full-texts of remaining articles were further inspected for eligibility against a priori defined inclusion and exclusion criteria.

    We included original articles written in English that included patients with a diagnosis of any mental disorders and whose study design included both tele–mental health and face-to-face groups that reported satisfaction scores for both groups. Articles were excluded if they only reported data on service acceptability, credibility, and working alliance; failed to report enough data for meta-analysis (authors were contacted to obtain missing data); or presented data from overlapping data sets (in which case, we selected the largest one).

    Literature search, study selection, and data extraction were performed by both authors independently. Disagreement was resolved by discussion. The study followed PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses [21]) guidelines (Table S1 in Multimedia Appendix 1). The protocol was registered in PROSPERO (CRD42020192299).

    Data Extraction

    We extracted author, publication year, setting (country, underserved area), mental disorder diagnosis, population type, study design, intervention type, intervention duration, intervention modality, satisfaction scale, number of participants in the tele–mental health and face-to-face group, age, and gender. As a measure of satisfaction, we extracted mean satisfaction score, standard deviation, or standard error of the mean, t test statistic, or P value of the t test, if the normality assumption was met in the original paper (Methods S1 in Multimedia Appendix 1).

    Data Analysis

    The meta-analysis was performed using R (version 4.0.0, The R Project; meta [22], metaphor [23], dmetar [24] packages). We calculated Hedges g and relative standard error. Since high heterogeneity was expected, we pooled effect sizes using a random-effect model [25]. We assessed between-study heterogeneity using the Q statistic and quantified total variability using the I2 index [26]. To assess the robustness of results, we performed influence analyses with graphical display of heterogeneity plots [27], by sequentially fitting our meta-analysis model to all 2k−1 possible combinations of the studies. We applied 3 clustering algorithms—k-means, density-based spatial clustering of applications with noise, and the Gaussian mixture model—to detect studies with heavy influences on the overall effect size estimate. Sensitivity analyses were conducted by removing these heavy-influence studies and re-running the meta-analysis (Methods S2 in Multimedia Appendix 1). We performed subgroup analyses with mixed-effect models to determine the influence of predefined categorical moderators: mental disorder diagnosis, population type, underserved area, study design, intervention type, and satisfaction scale. Meta-regression models were fit to investigate the influence of predefined continuous predictors: publication year, mean age, proportion of females, intervention duration, and sample size. We assessed publication bias with the Egger test [28], and risk of bias was examined with the revised Cochrane tool for randomized trials [29].


    Analysis of State-of-the-Art Scientific Publications

    The field was pioneered in 1973, by a paper published in the American Journal of Psychiatry, which described an interactive television system that connected Massachusetts General Hospital and a medical station in Boston (United States) [30]. In 1986, a group from McGill University (Canada) [31] published the first case-control study, which found no substantial difference between tele–mental health and face-to-face in terms of satisfaction among patients and providers. It was only in 1995 that a systematic interest developed, and the annual number of articles began to grow steadily, to reach 39 records in 2019. We expect a further surge in 2020, from a renewed interest for tele–mental health caused by the COVID-19 pandemic. Annual number of articles and sum of citations followed the same pattern until 2010. Then, the sum of citation declined, because more time is required for newly published articles to accumulate citations (Figure 2A).

    Only 10 articles had more than 100 citations each (range 120-244; Table S2 in Multimedia Appendix 1). Of these, 1 was the above-mentioned study published in 1986 [31], 8 were published between 2000 and 2010 [32-39], and 1 was published in 2013 [40]. All top-cited articles, except one [37], were controlled trials. The 10 top articles came from only 4 countries—United States, Canada, Australia, and United Kingdom—which are the countries contributing the most to the whole article data set (N=363 articles, 82%; Figure S1A in Multimedia Appendix 1). In the remaining countries, including Italy, that delivered 5 articles or less, tele–mental health research might be at an early stage, corresponding to scarce, if not absent, applications. In terms of international cooperation, the main hub countries are United States, United Kingdom, Australia, and Canada. Other cooperation patterns are more scattered, possibly being more occasional (Figure S1B in Multimedia Appendix 1).

    Document clustering identified 36 topics (Figure 2B). The top 10 topics (Table S3 in Multimedia Appendix 1) encompass 34% proportion of the data set. Of these, 6 are specific subjects concerning issues related to the implementation of tele–mental health services. Two topics regard tele–mental health interventions for depressive or anxiety disorders and posttraumatic stress disorder. Finally, 2 topics are focused on the use of tele–mental health in peculiar populations, such as children and adolescents, and patients with neurocognitive deficits (Figure 2B, Figure S1C, and Table S3 in Multimedia Appendix 1).

    ‎
    Figure 2. State-of-the-art of tele–mental health scientific publications: (A) Number of articles by year (blue bars) and the sum of citations for annual articles (green dotted line) and (B) Document clustering (total identified topics: 36) showing emerging COVID-19–related topics (pink dots, arrow) in relation to the top 10 topics (legend). PTSD: posttraumatic stress disorder; t-SNE: t-distributed stochastic neighbor embedding.
    View this figure

    Providers’ Responses to the Survey

    The survey was completed by 174 Italian mental health care providers, 112 (64.4%) of whom were female. Most respondents (75/174, 43.1%) were between 30 and 40 years old. The most represented region was Tuscany (n=42), followed by Lombardy (n=37) and Apulia (n=28) (Figure S2A in Multimedia Appendix 1), and 67.8% of respondents (n=118) were employed in the public sector, including inpatient or outpatient clinics, hospitals, psychiatric residential facilities, residences for the implementation of safety measures, and addiction treatment services, while 56 (32%) worked in the private sector, either in solo or group (n=4) settings. Our sample consisted of 83 (47.7%) physicians, 63 psychologists (36.2%), and 28 (16.1%) other mental health workers, that is, specialized nurses and professionals providing rehabilitative and educational interventions. The majority of the sample (134/174, 77.0%) reported that COVID-19 disrupted their normal service provision. The main reason (55%) was a reduction (or block) in nonurgent services, sometimes accompanied by conversion of structures to COVID clinics. Lockdown was the culprit in 44% of cases, while disruption was directly caused by the virus, that is, being infected or quarantined following contact with someone infected, in 2 cases. The median of COVID-19-related disruption was 7/10 (IQR 5-8) (Figure 3A).

    ‎
    Figure 3. Italian providers’ responses to the survey on the use of tele–mental health during the COVID-19 pandemic: (A) COVID-19–related disruption in mental health service provision; (B) number of physicians, psychologists, and other mental health professionals offering services exclusively face-to-face, mostly face-to-face, mostly by tele–mental health, or exclusively by tele–mental health during and prior to the pandemic; (C) tele–mental health tools used during and prior to the pandemic; and (D) providers’ attitude toward tele–mental health. EHR: electronic health record; IM: instant messaging; STM: supported telemedicine systems; VTC: video-based teleconferencing.
    View this figure

    During the pandemic, the rate of respondents providing any services through tele–mental health doubled, passing from 47.7% (83/174) to 92.5% (161/174); 68% respondents reported using mostly or exclusively tele–mental health (vs 1/174, or 0.6%, prior to the pandemic) (Figure 3B). Psychologists reported the highest rate of tele–mental health almost exclusive use (56/63, 89%), compared with psychiatrists (45/83, 54%) and other mental health workers (17/28, 61%) (χ62=45.97, P<.001) (Figure 3B). Respondents provided a variable amount of care provisions through tele–mental health, in contrast to their previous practice, where tele–mental health was used for less than 25% of care provisions in 82% of cases (Figure S2B in Multimedia Appendix 1). The use of telephone, instant messaging tools, emails remained stable, while we observed a 3.7-fold increase in the use of video-based teleconferencing (Figure 3C). Even if most respondents (132/174, 75.8%) found tele–mental health much or very much useful during the COVID-19 crisis (Figure S2C in Multimedia Appendix 1), 82% (143/174) envisaged to reduce tele–mental health use after the pandemic was over. Half of respondents had to resort to personal telecommunications at their own initiative, as only a small fraction of work settings were adequately equipped (31/118, 26.2% and 19/56, 33.9% in the public and private sector, respectively); however, an effort was made by the Italian National Health System to strengthen tele–mental health during the crisis, as reported by 28% of those in public work settings.

    Depression and anxiety disorders were deemed amenable to tele–mental health interventions by a large portion (70/174, 40.2%) of our sample. On the contrary, schizophrenia and other psychotic disorders, substance use disorders and major neurocognitive disorders ranked low (Figure S2D in Multimedia Appendix 1). One-third of respondents (50/174, 28.7%) considered tele–mental health particularly useful for underserved populations, but 25% (44/174) would offer tele–mental health to any population group (Figure S2E in Multimedia Appendix 1).

    Providers’ global attitude was skeptical. Only 21.3% of respondents (37/174) thought that tele–mental health was as valid, accurate and effective as face-to-face; 66.1% (115/174) were not positive about the ability to establish a good doctor–patient relationship. Most (132/174, 75.9%) did not believe that tele–mental health could reduce the barrier of stigma. Less than half of respondents felt somewhat or very much trained and satisfied with tele–mental health (Figure 3D).

    We received 120 responses from mental health providers from the 5 continents, which replicated Italian data, except for a more positive attitude toward tele–mental health interventions, in terms of: ability to reduce stigma (70/120, 58.3%); feeling prepared and satisfied with tele–mental health care (91/120, 75.8% and 80/120, 66.7%, respectively) (Results S1, Figures S3 and S4 in Multimedia Appendix 1).

    Meta-analysis of Patient Satisfaction With Tele–Mental Health Interventions

    Eligibility screening of 247 articles yielded 41 articles (Figure S5 in Multimedia Appendix 1). Of these, 12 could not be included in the meta-analysis because they did not report enough data for computation (Table S4 in Multimedia Appendix 1). The final sample included 29 studies (Table 1), contributing data from 2143 patients (tele–mental health: n=1039; face-to-face: n=1104; 34% female), with mean age of 39.4 years (SD 14.3). The average sample size was 74 (range 12 to 254). The majority (n=19) of studies were conducted in the United States. Approximately half (n=13) of studies reported on tele–mental health in remote geographic areas, such as Thunder Bay in Canada [33,41], the Hawaiian Islands [42-44] and Pacific northwest of the United States [45], rural Australia [46], or targeted underserved communities, such as Native American communities [47], Hispanic communities [48], low-income patients with HIV [49], and inmates of correctional institutions [50-52]. The most represented diagnosis was depression or anxiety disorders (n=11), followed by posttraumatic stress disorder (n=6), alcohol or substance use disorders (n=3), and attention deficit hyperactivity disorder or disruptive disorders (n=2); 7 studies included individuals with any mental disorder. Thirteen studies recruited only adult individuals, while 3 recruited children or adolescents and their caregivers. Most studies offered services to special populations, such as military personnel or veterans (n=10) and individuals in correctional settings (n=3). Eleven studies used telepsychiatry (8 providing consultations and 3 assessment), 17 studies used telepsychology or counseling, and 1 study used both; most studies (n=24) were randomized controlled trials. Mean follow-up was 227 days (range 90 to 540 days) for telepsychiatry and 70 days for telepsychology. The preferred modality was video-based teleconferencing (n=24), the rest were telephone- or web-based interventions. Patient satisfaction was assessed with standardized validated scales in 21 studies; 8 studies used custom tools.

    Table 1. Studies included in the meta-analysis.
    Diagnosis, study typePopulationAgeaF (%)bInterventioncMeasureTele–mental health, n/N (%)Face-to-face, n/N (%)Hedges g (95% CI)
    Attention-deficit hyperactivity disorder

    		RCTd [45]Children and caregivers9.330Caregiver behavior training; 25 weeksCSQe-ADHD—f /12 (—)—/25 (—)−0.817 (−1.531 to −0.102)

    		RCT [53]Children and caregivers10.432Group parent training; 10 weeksCustom8/9 (88.9)11/13 (84.6)−0.005 (−0.916 to 0.906)
    Alcohol use disorder

    		Pilot [54]Volunteers36.6432 motivational interviews (phone also)Custom20100.005 (−0.754 to 0.764)
    Any mental disorder

    		RCT [55]Patients (severe)46.348Telepsychiatry; 18-month follow-upGGZg33/47 (70.2)38/46 (82.6)0.495 (0.022 to 0.969)

    		RCT, eq [33]Outpatients—63Telepsychiatry; 4-month follow-upCSQ-8125/241 (51.9)129/254 (50.8)−0.051 (−0.297 to 0.195)

    		Comph, pilot [46]Children and adolescents12.328TelepsychiatrySurvey13340.004 (−0.635 to 0.643)

    		Test–retest [47]American Indian veterans540TelepsychiatryCustom5353−0.112 (−0.493 to 0.269)

    		Comp [50]Forensic psychiatric patient inmates34.2—InterviewSurveyi23200.000 (−0.599 to 0.599)

    		Comp [52]Correctional institution inmates31.80Telepsychiatry or telepsychologyCSQ-886100−0.081 (−0.369 to 0.207)

    		RCT [51]Forensic psychiatric patient inmatesj4243Competency testkCustom1110−0.173 (−1.031; 0.685)
    Major depressive disorder

    		RCT [34]Veterans49.712Telepsychiatry; 6-month follow-upCustom43/59 (72.9)42/60 (70)0.280 (−0.148 to 0.707)

    		RCT, pilot [41]Outpatients—62Telepsychiatry; 4-month follow-upCSQ-88/12 (66.7)9/12 (75)−1.142 (−2.177 to −0.108)

    		RCT [48]Low-income Hispanic patients4388Telepsychiatry; 6-month follow-upVSQl-969/80 (86.3)78/87 (89.7)0.166 (−0.159 to 0.490)

    		RCT, pilot [49]Low-income patients with HIV45.174CBTm (phone only); 14 weeksSIMHn14/16 (87.5)17/18 (94.4)−0.354 (−1.067 to 0.359)

    		RCT, prag [56]Outpatients43.166Psychotherapy and web-based; 12 weeksCSQ-837/51 (72.5)32/47 (68.1)0.071 (−0.402 to 0.544)

    		RCT [57]General35.671Telepsychiatry; 12-month follow-upCSQ-845/53 (84.9)40/54 (74.1)−0.443 (−0.874 to −0.012)

    		RCT, NI [58]Older adult veterans63.92.5Behavior activation; 8 weeksCPOSS-VAo100/120 (83.3)104/121 (86)−0.256 (−0.531 to 0.020)

    		RCT, NI [59]pMilitary and veterans—18Behavior activation; 8 weeksCSQ-845/62 (72.6)42/59 (71.2)−0.142 (−0.563 to 0.279)
    Mood or anxiety disorders

    		RCT [60]Outpatients3058CBT; 12 weeksCSQ13/14 (65)11/12 (91.7)0.115 (−0.689 to 0.919)
    Obsessive compulsive disorder

    		RCT, NI [61]Outpatients31.960CBT (phone only); 10 weeksCSQ34/36 (94.4)32/36 (88.9)−0.331 (−0.818 to 0.155)
    Opioid use disorder

    		RCT, pilot [62]In treatment (methadone)40.662Web-basedq counseling; 6 weeksSurvey20/33 (60.6)17/17 (100)0.004 (−0.642 to 0.651)

    		RCT [63]In treatment (drug abstinence)4156Web-basedq counseling; 12 weeksCSQ-822/50 (44)35/35 (100)0.437 (−0.103 to 0.976)
    Posttraumatic stress disorder

    		RCT, NI [38]Veterans55.50Groupr CBT; 14 weeksCPOSS-VA9/17 (52.9)12/21 (57.1)0.247 (−0.621 to 1.115)

    		RCT, NI [42]Veterans55.10Anger management group; 12 weeksCPOSS-VA55/61 (90.2)57/64 (89.1)−0.015 (−0.386 to 0.355)

    		RCT, pilot [43]Veterans—0Coping skills group; 8 weeksCustom8/9 (88.9)4/8 (50)0.405 (−0.808 to 1.618)

    		RCT, NI [44]Veterans55.30Cognitive processing therapyCPOSS-VA46/61 (75.4)50/64 (78.1)0.249 (−0.153 to 0.651)

    		RCT [64]Veterans50.00Telepsychiatrys; 3-month follow-upCustom30/37 (81.1)30/34 (88.2)0.888 (0.357 to 1.419)

    		RCT, NI [65]Veterans446Prolonged exposure; 12 weeksCPOSS-VA27/75 (36)40/75 (53.3)−0.056 (−0.544 to 0.432)
    Social phobia

    		RCT [66]Volunteers24.4—Web-basedt; 2 monthsCustom—/30 (—)—/22 (—) 0.104 (−0.446; 0.655)

    aMean, in years.

    bPercentage of female individuals included in each study.

    cThe intervention used video-based teleconferencing, unless otherwise indicated.

    dRCT: randomized controlled trial—eq indicates equivalence, NI indicates noninferiority, and prag indicates pragmatic.

    eCSQ: Client Satisfaction Questionnaire—8, 9, and ADHD indicate the 8-item, 9-item, and attention-deficit hyperactivity disorder versions, respectively.

    fData were not provided.

    gGGZ Thermometer.

    hComp: comparative study.

    iGroup Health Association of America Consumer Satisfaction Survey.

    jPatients with Schizophrenia spectrum disorders and mental retardation.

    kGeorgia Court Competency Test Mississippi State Hospital revision.

    lVSQ: Visit-specific Satisfaction Questionnaire.

    mCBT: cognitive behavioral therapy.

    nSIMH: Satisfaction Index Mental Health.

    oCPOSS-VA: Charleston psychiatric outpatient satisfaction scale: Veteran Affairs version.

    pMinor depressive disorder was also included.

    qGetgoing program.

    rSocial and emotional rehabilitation.

    sImo voice calls, text messaging, Telegram, and Skype.

    tTalk to me, a self-administered program.

    Our meta-analysis revealed no difference in patient satisfaction between tele–mental health and face-to-face interventions (Hedges g=−0.001, 95% CI −0.116 to 0.114, P=.985; Figure S6 in Multimedia Appendix 1). There was moderate between-study heterogeneity (Q=43,83, I2=36%, P=.03).

    The graphical display of heterogeneity plot formed a symmetric distribution, around g=0, slightly deviating toward a pattern with positive effect sizes and moderate heterogeneity (peak around 50%) (Figure S7 in Multimedia Appendix 1). Clustering algorithms detected one study [64] that explained the shift toward higher heterogeneity estimates (Figures S7 and S8 in Multimedia Appendix 1). After removing this study, heterogeneity became nonsignificant (Q=32.51, I2=17%, P=.214); however, the overall effect size, though slightly more negative (ie, favoring face-to-face over tele–mental health), was not impacted (Hedges g=−0.032, 95% CI −0.132 to 0.068, P=.531; Figure S11 in Multimedia Appendix 1). These findings were corroborated by other influence diagnostics (Results S2, Figures S9 and S10 in Multimedia Appendix 1).

    Subgroup analyses, performed after removing [64], did not show any significant differences between mental disorder diagnoses (P=.341), population types (P=.813), served vs underserved area (P=.683), study designs (P=.392), and satisfaction scales (P=.407) (Figures S12-16 in Multimedia Appendix 1). No significant difference emerged between telepsychiatry vs telepsychology studies, excluding studies providing assessment (total between group heterogeneity Q=0.176, df=1, P=.674). While there was virtually no heterogeneity among telepsychology studies (n=17, Q=15,66, I2=0%, P=.468), we found moderate to substantial heterogeneity among telepsychiatry studies (n=7, Q=15.78, I2=62%, P<.05) (Figure 4). None of the meta-regression models yielded significant results (publication year: P=.417; age: P=.207; gender: P=.433; intervention duration: P=.531; sample size: P=.588) (Figures S17-21 in Multimedia Appendix 1).

    ‎
    Figure 4. Forest plot of satisfaction scores for tele–mental health vs face-to-face interventions according by intervention type. Positive values favor tele–mental health, while negative values favor face-to-face.
    View this figure

    No publication bias was detected (t=0.17, P=.867; Figure S22 in Multimedia Appendix 1). There was a high risk of bias for 9 studies, some concerns for 14 studies, and low risk for 6 studies (Figure S24 in Multimedia Appendix 1). The main weakness was due to missing outcome data, as satisfaction scores were generally available for a fraction of randomized participants, which ranged from 36% [65] to 100% [62,63] and varied between intervention arms in the same study (eg, 44% vs 100% in the tele–mental health and face-to-face arms, respectively [63]) (Figures S23 and S24 in Multimedia Appendix 1).


    The field of tele–mental health has been continuously evolving since 2000. Such progress was limited to a few countries, namely United States, United Kingdom, Australia and Canada. This might be related to uneven incomes and scientific or technological levels among countries. In addition, tele–mental health may represent a valuable approach to overcome “the tyranny of distance” [46] in countries where substantial portions of the population live in remote rural areas and have unequal access to care. We found that a large amount of tele–mental health literature evaluates (1) service-centered parameters, such as feasibility, acceptability, and sustainability, and (2) care-centered parameters, such as therapeutic alliance, treatment outcome, and patient satisfaction. Depression and posttraumatic stress disorder stand out among the top 10 research topics. Evidence supporting the efficacy of tele–mental health interventions for depression, anxiety, and posttraumatic stress disorders is abundant and robust [67-69]. Another prominent research topic is children and adolescent, a population considered more suited to tele–mental health since they are perceived as digital natives. Two meta-reviews showed that tele–mental health is a valid option for youth with depression and anxiety, while its clinical benefits for autism spectrum disorder, attention deficit hyperactivity disorder, psychosis, and eating disorders remain questionable [70,71]. Tele–mental health has been proven to alleviate pressure on emergency departments (third research topic) [72,73].

    Research in tele-mental health did not translate into policy changes and resource allocation. The World Psychiatric Association–Lancet Psychiatry Commission on the Future of Psychiatry has defined 6 core considerations to be met for technological innovations to transform health care: (1) patient and clinician engagement; (2) clinical evidence and standards; (3) clinical systems integration; (4) digital trust, ethics, and transparency; (5) interoperability and scalability; (6) data science and methods [74]. At present, tele–mental health has partially met these targets, due to factors related to both clinicians and patients. Clinicians are often reluctant to adopt tele–mental health because of concerns about the ability to establish a satisfying doctor–patient relationship and lack of knowledge of relevant privacy, transparency, and confidentiality issues [75,76]. A digital divide exists among patients, which excludes a large share of them from tele-mental health interventions [74,77].

    When COVID-19 started its inexorable march over the planet, very few countries were sufficiently equipped with tele–mental health technologies, trained clinicians, and guidelines [10,78]. Italy was no exception. Less than one-third of respondents deemed their tele–mental health service to be adequate prior to COVID-19. Care provision was massively disrupted. Qualitative reports have been published, that mostly refer to the situation of the Italian National Health System in Lombardy, which was the most affected Italian region [2-6]. All reports agree that COVID-19 initiated an abrupt transition to tele–mental health delivery of care [2,4-6]. We observed that this change was more evident for psychologists, compared with physicians, probably because physicians carry out part of their clinical activity in in-patient units (COVID-positive patients with serious psychiatric conditions were still admitted to hospital wards) [5] and need to perform physical examinations, while psychotherapy may be more easily conducted remotely. We are aware that the relatively small number of respondents (n=174) hampers generalizability of our observations. Nevertheless, our findings were fully replicated by responses of international mental health professionals (Figure S3 in Multimedia Appendix 1). Also, similar trends have been reported by others in several countries. In China, when the novel human coronavirus (SARS-CoV-2) emerged in December 2019, there was rapid implementation of mental health hotlines and hospital tele–mental health services. In some cases, as the West China Hospital of Sichuan University, tele–mental health services collaborated with courier services to deliver medication to patients’ homes [8]. In most European countries there was a boost in tele–mental health use and value, and regulatory barriers were substantially lifted [79,80]. An analysis of electronic health record data showed a substantial shift from face-to-face to tele–mental health contacts in London, United Kingdom after lockdown measures [81]. These findings are paralleled by those in Australia and Africa [82,83]. In May 2020, the American Psychiatric Association surveyed its members on the matter, and responses from 500 American psychiatrists grossly replicated our findings of a major transition to tele–mental health use—in a couple of months the percentage of respondents seeing more than 75% of their patient caseload via tele–mental health increased from 2.1% to 84.7% [84]. In addition, respondents reported that satisfaction was high or very high among patients first assessed via tele–mental health [84], and Sammons et al [85] reported a similar adaptation to COVID-19 among psychologists in the Trust and National Register (United States).

    Half of respondents used personal tools on their own initiative. If, from one side, these spontaneous efforts are to be commended because they allow the system to rapidly adjust to unexpected stressors, then from the other, uncoordinated and uneducated use of such tools might increase the risk of breaches in consent processes, privacy, and data protection or may lack appropriate emergency management plans. We advocate that mental health departments be digitized in order to improve their resilience in face of public health emergencies [2]. Such technological leaps will only be successful if complemented with proper training and supported by policy changes. Conveniently, open-access resources have been flourishing during the last months; we recommend the practical guidance developed by the Oxford Precision Psychiatry Lab [12], and the American Psychiatric Association Toolkit [11]. A better understanding of best practices could modify skeptical views such as those we recorded in our survey. It has been reported that clinicians had a gatekeeper role against tele–mental health, and their concerns were mainly related to the ability to build a meaningful doctor–patient relationship [17]. The integration of tele–mental health in mental health care implies a transition from the current centralized model of care to a more distributed model, in order to re-define the equilibrium between clinicians and patients’ responsibilities. Many issues regarding standard of care, access to data, clinicians’ compensation remain open, thus it is no surprise that clinicians may feel uncomfortable. Research that explicitly addresses these issues will be needed. It will be fundamental to ensure that clinicians’ needs and desires are heard effectively. Formal education about the role of technology in care provision will have to be implemented starting from medical school, without neglecting colleagues who may be less familiar with technology because they have practiced for years before the advent of smartphones and their application in health care. Furthermore, the field will need to meet some key requirements to support the transformation. Borrowing from the recommendations issued by the World Psychiatric Association–Lancet Psychiatry Commission [74] and the American Psychiatric Association [84], we maintain that (1) both psychiatrists and patients should be engaged in all phases of tele–mental health development and implementation and not only as final users; (2) patients’ routine screening should include an assessment of digital access, literacy, and comfort, and specialized education, technical support, and internet (or device) access programs should be offered to improve treatment delivery (especially to vulnerable populations, eg, older people, homeless people, asylum seekers); (3) a tele–mental health ethical code will be needed, to reassure patients about confidentiality and safety issues and help them making informed decisions; (4) tele–mental health sustainability and scalability should be promoted, to avoid care fragmentation and abate costs; (5) current regulations should be reviewed regarding remote drug prescription, use of audio-only communications for patients’ assessment and management, and service frequency in in-patient settings and nursing facilities; (6) careful considerations should be made about compensations and national health insurance programs. These considerations apply, not only to tele–mental health, but also, to digital psychiatry. The last decade has witnessed an expansion in smartphone apps, wearable sensors and other technologies for digital phenotyping of patients suffering from mental disorders, which has been accompanied by the growing use of artificial intelligence in health care. This expansion has mainly been driven by the opportunities offered by technological advancement, which often lack adequate scientific and clinical roots. Research, funded by government programs, will be needed [86]: for instance, digitization is one of the pillars of the €750-billion Next Generation European Union plan (equivalent to approximately US $891 billion), which aims to support recovery from the COVID-19 crisis and also to invest in the future and resilience of our society.

    To focus on service users, we investigated how tele–mental health compares to face-to-face interventions in terms of patient satisfaction, because this is a crucial influence on treatment outcome, particularly in mental health [87,88]. We performed a systematic review and meta-analysis, which did not detect a significant difference in satisfaction between tele–mental health and face-to-face (P=.985). Because studies were moderately heterogeneous, we applied 2 methods to explore heterogeneity. Both methods showed that one study—Haghnia et al 2019 [64], alone—explained much of the heterogeneity. This study was conducted in Iran, whose conditions might be different from high-income countries. The economic impact, difficulties of travelling, and accommodation requirements associated with face-to-face visits might be more burdensome for people living in the Middle East and justify the higher satisfaction scores found in patients treated by tele–mental health [89]. However, this study had marginal influence on the global effect size. Subgroup analyses showed homogeneity among studies focusing on psychotherapy, as opposed to those focusing on telepsychiatry interventions, which yielded substantially heterogeneous effect sizes. Psychiatric consultations are characterized by high variability, consisting of meetings of variable duration, separated by variable intervals, with variable content based on patients’ incidental needs and medication management. On the contrary, psychotherapies are “healing relationships” [90] developing over a series of evenly distributed contacts of preestablished duration that use evidence-based (often manualized) methods [91]. A previous systematic review [88], which compared tele–mental health to face-to-face–delivered psychotherapeutic interventions, similarly found that patients were equally satisfied with both approaches but highlighted limitations (some of which are also relevant to our study). Most studies included in our review were affected by some risk of bias from high attrition rates, which led to small, underpowered sample sizes. Satisfaction scores were available just for the fraction of patients who remained in treatment. It is plausible that dissatisfaction with treatment was responsible for participants dropping out of the studies and becoming unavailable for satisfaction assessment. However, attrition rates in the 2 treatment arms (tele–mental health vs face-to-face) were similar, most likely causing satisfaction score inflation in both arms with negligible impact on the difference. A selection bias could have been introduced even before randomization, since 6 studies [49,56,57,62,63,87] excluded eligible participants who did not have access to computer and internet connection. In 16 studies [33,34,38,41-48,50-53,60] tele–mental health sessions were held in rooms fully equipped with high-definition video-based teleconferencing units and broadband internet access. In 2 studies, tele–mental health interventions were performed at home, but participants were provided with videophones [58] or computers [55] and a dedicated line. Therefore, in most cases, technology-based factors, which contribute to shape patient satisfaction [88], could have been minimized. This limits the generalizability of their results to ecological contexts: (1) many patients may be marginalized due to lack of access to technology and skills; and (2) problems with video definition, audio lag, or connection could dampen the perceived consultation quality. Another limitation is that we only considered overall patient satisfaction. This is a complex clinical outcome that includes several factors related to patient, disease, provider, therapy, environment, and technology [87]. Rohland et al [92] showed that patients rated tele–mental health higher than face-to-face for convenience, ease, technical skills, attention given, and time spent, while face-to-face was preferable to tele–mental health for self-reporting outcome, helpfulness, and eye contact. It has been suggested that tele–mental health patients develop lower levels of therapeutic alliance, resulting in worse continuity of care [38,93,94], but data are still inconclusive. Whether the relative preference for tele–mental health or face-to-face care has an impact on clinical outcomes in specific domains needs to be determined in future longitudinal studies.

    In conclusion, evidence for the use of tele–mental health is robust, but it is concentrated in a few countries. The initial enthusiasm around tele–mental health did not translate to clinical application. During the COVID-19 pandemic, many mental health professionals resorted to tele–mental health, not without some aversion, feeling that “they had no other choice [6].” It is advisable that mental health services should become equipped with tele–mental health to increase the ability to efficiently cope with public health crises. We believe that this does not necessarily contradict the preferences of both clinicians and patients for in-person meaningful therapeutic rapports.

    Acknowledgments

    This work was supported by the University of Pisa (PRA 2020-21 awarded to GR).

    Authors' Contributions

    GR and RM designed the study. GR and RM designed the survey; performed the literature search, study selection, data extraction, and data analysis; and prepared the figures. GR wrote the first draft of the manuscript with input from RM.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Supplementary information.

    DOCX File , 5994 KB

    References

    1. Marazziti D, Stahl SM. The relevance of COVID-19 pandemic to psychiatry. World Psychiatry 2020 Jun;19(2):261 [FREE Full text] [CrossRef] [Medline]
    2. de Girolamo G, Cerveri G, Clerici M, Monzani E, Spinogatti F, Starace F, et al. Mental health in the coronavirus disease 2019 emergency-the Italian response. JAMA Psychiatry 2020 Sep 01;77(9):974-976. [CrossRef] [Medline]
    3. Percudani M, Corradin M, Moreno M, Indelicato A, Vita A. Mental health services in Lombardy during COVID-19 outbreak. Psychiatry Res 2020 Jun;288:112980 [FREE Full text] [CrossRef] [Medline]
    4. Carpiniello B, Tusconi M, di Sciascio G, Zanalda E, di Giannantonio M, Executive Committee of the Italian Society of Psychiatry. Mental health services in Italy during the COVID-19 pandemic. Psychiatry Clin Neurosci 2020 Aug;74(8):442-443 [FREE Full text] [CrossRef] [Medline]
    5. D'Agostino A, Demartini B, Cavallotti S, Gambini O. Mental health services in Italy during the COVID-19 outbreak. Lancet Psychiatry 2020 May;7(5):385-387 [FREE Full text] [CrossRef] [Medline]
    6. Fagiolini A, Cuomo A, Frank E. COVID-19 diary from a psychiatry department in Italy. J Clin Psychiatry 2020 Mar 31;81(3):20com13357 [FREE Full text] [CrossRef] [Medline]
    7. Zhou X, Snoswell CL, Harding LE, Bambling M, Edirippulige S, Bai X, et al. The role of telehealth in reducing the mental health burden from COVID-19. Telemed J E Health 2020 Apr;26(4):377-379. [CrossRef] [Medline]
    8. Zhou J, Liu L, Xue P, Yang X, Tang X. Mental health response to the COVID-19 outbreak in China. Am J Psychiatry 2020 Jul 01;177(7):574-575. [CrossRef] [Medline]
    9. Kalin ML, Garlow SJ, Thertus K, Peterson MJ. Rapid implementation of telehealth in hospital psychiatry in response to COVID-19. Am J Psychiatry 2020 Jul 01;177(7):636-637. [CrossRef] [Medline]
    10. Pereira-Sanchez V, Adiukwu F, El Hayek S, Bytyçi DG, Gonzalez-Diaz JM, Kundadak GK, et al. COVID-19 effect on mental health: patients and workforce. Lancet Psychiatry 2020 Jun;7(6):e29-e30 [FREE Full text] [CrossRef] [Medline]
    11. Telepsychiatry. American Psychiatric Association.   URL: https://www.psychiatry.org/psychiatrists/practice/telepsychiatry [accessed 2020-06-30]
    12. Digital technologies and telepsychiatry. Oxford Health Biomedical Research Centre.   URL: https://oxfordhealthbrc.nihr.ac.uk/our-work/oxppl/table-5-digital-technologies-and-telepsychiatry/ [accessed 2020-06-30]
    13. Understanding Gartner's Hype Cycles. Gartner.   URL: https://www.gartner.com/en/documents/3887767/understanding-gartner-s-hype-cycles [accessed 2020-07-02]
    14. Fusar-Poli P. The Hype Cycle of the clinical high risk state for psychosis: the need of a refined approach. Schizophr Bull 2018 Mar;44(2):250-253. [CrossRef]
    15. Fusar-Poli P, Davies C, Solmi M, Brondino N, De Micheli A, Kotlicka-Antczak M, et al. Preventive treatments for psychosis: umbrella review (just the evidence). Front Psychiatry 2019;10:764 [FREE Full text] [CrossRef] [Medline]
    16. O'Keefe M, White K, Jennings JC. Asynchronous telepsychiatry: a systematic review. J Telemed Telecare 2021 Apr;27(3):137-145. [CrossRef] [Medline]
    17. Cowan KE, McKean AJ, Gentry MT, Hilty DM. Barriers to use of telepsychiatry: clinicians as gatekeepers. Mayo Clin Proc 2019 Dec;94(12):2510-2523. [CrossRef] [Medline]
    18. Deng Z, Wang H, Chen Z, Wang T. Bibliometric analysis of dendritic epidermal T cell (DETC) research from 1983 to 2019. Front Immunol 2020;11:259 [FREE Full text] [CrossRef] [Medline]
    19. Campello R, Ricardo JG, Moulavi D, Sander J. Density-based clustering based on hierarchical density estimates. In: Advances in Knowledge Discovery and Data Mining. 2013 Presented at: Pacific-Asia Conference on Knowledge Discovery and Data Mining; April 14-17; Gold Coast, Australia p. 160-172. [CrossRef]
    20. Telemental_health. GitHub.   URL: https://github.com/raffaelemazziotti/telemental_health [accessed 2020-07-01]
    21. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    22. Balduzzi S, Rücker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health 2019 Nov;22(4):153-160. [CrossRef] [Medline]
    23. Viechtbauer W. Conducting meta-analyses in with the metafor package. J Stat Soft 2010;36(3):1-48. [CrossRef]
    24. Harrer M, Cuijpers P, Furukawa T, Ebert D. Doing Meta-Analysis With R: A Hands-On Guide. Boca Raton, Florida: CRC Press; 2021.
    25. DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemp Clin Trials 2015 Nov;45(Pt A):139-145 [FREE Full text] [CrossRef] [Medline]
    26. Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002 Jun 15;21(11):1539-1558. [CrossRef] [Medline]
    27. Olkin I, Dahabreh IJ, Trikalinos TA. GOSH - a graphical display of study heterogeneity. Res Synth Methods 2012 Sep;3(3):214-223. [CrossRef] [Medline]
    28. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997 Sep 13;315(7109):629-634 [FREE Full text] [CrossRef] [Medline]
    29. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019 Aug 28;366:l4898. [CrossRef] [Medline]
    30. Dwyer TF. Telepsychiatry: psychiatric consultation by interactive television. Am J Psychiatry 1973 Aug;130(8):865-869. [CrossRef] [Medline]
    31. Dongier M, Tempier R, Lalinec-Michaud M, Meunier D. Telepsychiatry: psychiatric consultation through two-way television. a controlled study. Can J Psychiatry 1986 Feb;31(1):32-34. [CrossRef] [Medline]
    32. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One 2010 Jun;5(6):e10939 [FREE Full text] [CrossRef] [Medline]
    33. O'Reilly R, Bishop J, Maddox K, Hutchinson L, Fisman M, Takhar J. Is telepsychiatry equivalent to face-to-face psychiatry? results from a randomized controlled equivalence trial. Psychiatr Serv 2007 Jun;58(6):836-843. [CrossRef] [Medline]
    34. Ruskin PE, Silver-Aylaian M, Kling MA, Reed SA, Bradham DD, Hebel JR, et al. Treatment outcomes in depression: comparison of remote treatment through telepsychiatry to in-person treatment. Am J Psychiatry 2004 Aug;161(8):1471-1476. [CrossRef] [Medline]
    35. Elford R, White H, Bowering R, Ghandi A, Maddiggan B, St John K, et al. A randomized, controlled trial of child psychiatric assessments conducted using videoconferencing. J Telemed Telecare 2000;6(2):73-82. [CrossRef] [Medline]
    36. Fortney JC, Pyne JM, Edlund MJ, Williams DK, Robinson DE, Mittal D, et al. A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med 2007 Aug;22(8):1086-1093 [FREE Full text] [CrossRef] [Medline]
    37. May C, Gask L, Atkinson T, Ellis N, Mair F, Esmail A. Resisting and promoting new technologies in clinical practice: the case of telepsychiatry. Soc Sci Med 2001 Jun;52(12):1889-1901. [CrossRef] [Medline]
    38. Frueh BC, Monnier J, Yim E, Grubaugh AL, Hamner MB, Knapp RG. A randomized trial of telepsychiatry for post-traumatic stress disorder. J Telemed Telecare 2007;13(3):142-147. [CrossRef] [Medline]
    39. Raue P, Schulberg H, Heo M, Klimstra S, Bruce M. Patients' depression treatment preferences and initiation, adherence, and outcome: a randomized primary care study. Psychiatr Serv 2009 Mar;60(3):337-343 [FREE Full text] [CrossRef] [Medline]
    40. Titov N, Dear BF, Johnston L, Lorian C, Zou J, Wootton B, et al. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: randomised controlled trial. PLoS One 2013 Jul;8(7):e62873 [FREE Full text] [CrossRef] [Medline]
    41. Bishop JE, O'Reilly RL, Maddox K, Hutchinson LJ. Client satisfaction in a feasibility study comparing face-to-face interviews with telepsychiatry. J Telemed Telecare 2002;8(4):217-221. [CrossRef] [Medline]
    42. Greene CJ, Morland LA, Macdonald A, Frueh BC, Grubbs KM, Rosen CS. How does tele-mental health affect group therapy process? Secondary analysis of a noninferiority trial. J Consult Clin Psychol 2010 Oct;78(5):746-750. [CrossRef] [Medline]
    43. Morland LA, Pierce K, Wong MY. Telemedicine and coping skills groups for Pacific Island veterans with post-traumatic stress disorder: a pilot study. J Telemed Telecare 2004;10(5):286-289. [CrossRef] [Medline]
    44. Morland LA, Mackintosh M, Greene CJ, Rosen CS, Chard KM, Resick P, et al. Cognitive processing therapy for posttraumatic stress disorder delivered to rural veterans via telemental health: a randomized noninferiority clinical trial. J Clin Psychiatry 2014 May;75(5):470-476. [CrossRef] [Medline]
    45. Tse YJ, McCarty CA, Stoep AV, Myers KM. Teletherapy delivery of caregiver behavior training for children with attention-deficit hyperactivity disorder. Telemed J E Health 2015 Jun;21(6):451-458. [CrossRef] [Medline]
    46. Dossetor D, Nunn K, Fairley M, Eggleton D. A child and adolescent psychiatric outreach service for rural New South Wales: a telemedicine pilot study. J Paediatr Child Health 1999 Dec;35(6):525-529. [CrossRef] [Medline]
    47. Shore JH, Brooks E, Savin D, Orton H, Grigsby J, Manson SM. Acceptability of telepsychiatry in American Indians. Telemed J E Health 2008 Jun;14(5):461-466 [FREE Full text] [CrossRef] [Medline]
    48. Chong J, Moreno F. Feasibility and acceptability of clinic-based telepsychiatry for low-income Hispanic primary care patients. Telemed J E Health 2012 May;18(4):297-304. [CrossRef] [Medline]
    49. Himelhoch S, Medoff D, Maxfield J, Dihmes S, Dixon L, Robinson C, et al. Telephone based cognitive behavioral therapy targeting major depression among urban dwelling, low income people living with HIV/AIDS: results of a randomized controlled trial. AIDS Behav 2013 Oct;17(8):2756-2764. [CrossRef] [Medline]
    50. Brodey BB, Claypoole KH, Motto J, Arias RG, Goss R. Satisfaction of forensic psychiatric patients with remote telepsychiatric evaluation. Psychiatr Serv 2000 Oct;51(10):1305-1307. [CrossRef] [Medline]
    51. Manguno-Mire G, Thompson JJ, Shore J, Croy C, Artecona J, Pickering J. The use of telemedicine to evaluate competency to stand trial: a preliminary randomized controlled study. J Am Acad Psychiatry Law 2007;35(4):481-489. [Medline]
    52. Morgan RD, Patrick AR, Magaletta PR. Does the use of telemental health alter the treatment experience? inmates' perceptions of telemental health versus face-to-face treatment modalities. J Consult Clin Psychol 2008 Feb;76(1):158-162. [CrossRef] [Medline]
    53. Xie Y, Dixon JF, Yee OM, Zhang J, Chen YA, Deangelo S, et al. A study on the effectiveness of videoconferencing on teaching parent training skills to parents of children with ADHD. Telemed J E Health 2013 Mar;19(3):192-199. [CrossRef] [Medline]
    54. Baca CT, Manuel JK. Satisfaction with long-distance motivational interviewing for problem drinking. Addict Disord Their Treat 2007 Mar;6(1):39-41. [CrossRef]
    55. Hulsbosch AM, Nugter MA, Tamis P, Kroon H. Videoconferencing in a mental health service in The Netherlands: A randomized controlled trial on patient satisfaction and clinical outcomes for outpatients with severe mental illness. J Telemed Telecare 2017 Jun;23(5):513-520. [CrossRef] [Medline]
    56. Berger T, Krieger T, Sude K, Meyer B, Maercker A. Evaluating an e-mental health program ("deprexis") as adjunctive treatment tool in psychotherapy for depression: results of a pragmatic randomized controlled trial. J Affect Disord 2018 Dec;227:455-462. [CrossRef] [Medline]
    57. Hungerbuehler I, Valiengo L, Loch AA, Rössler W, Gattaz WF. Home-based psychiatric outpatient care through videoconferencing for depression: a randomized controlled follow-up trial. JMIR Ment Health 2016 Aug 03;3(3):e36 [FREE Full text] [CrossRef] [Medline]
    58. Egede LE, Acierno R, Knapp RG, Walker RJ, Payne EH, Frueh BC. Psychotherapy for depression in older veterans via telemedicine: effect on quality of life, satisfaction, treatment credibility, and service delivery perception. J Clin Psychiatry 2016 Dec;77(12):1704-1711. [CrossRef] [Medline]
    59. Luxton DD, Pruitt LD, Wagner A, Smolenski DJ, Jenkins-Guarnieri MA, Gahm G. Home-based telebehavioral health for U.S. military personnel and veterans with depression: a randomized controlled trial. J Consult Clin Psychol 2016 Nov;84(11):923-934. [CrossRef] [Medline]
    60. Stubbings DR, Rees CS, Roberts LD, Kane RT. Comparing in-person to videoconference-based cognitive behavioral therapy for mood and anxiety disorders: randomized controlled trial. J Med Internet Res 2013 Nov 19;15(11):e258 [FREE Full text] [CrossRef] [Medline]
    61. Lovell K, Cox D, Haddock G, Jones C, Raines D, Garvey R, et al. Telephone administered cognitive behaviour therapy for treatment of obsessive compulsive disorder: randomised controlled non-inferiority trial. BMJ 2006 Oct 28;333(7574):883 [FREE Full text] [CrossRef] [Medline]
    62. King VL, Stoller KB, Kidorf M, Kindbom K, Hursh S, Brady T, et al. Assessing the effectiveness of an internet-based videoconferencing platform for delivering intensified substance abuse counseling. J Subst Abuse Treat 2009 Apr;36(3):331-338. [CrossRef] [Medline]
    63. King VL, Brooner RK, Peirce JM, Kolodner K, Kidorf MS. A randomized trial of web-based videoconferencing for substance abuse counseling. J Subst Abuse Treat 2014 Jan;46(1):36-42 [FREE Full text] [CrossRef] [Medline]
    64. Haghnia Y, Samad-Soltani T, Yousefi M, Sadr H, Rezaei-Hachesu P. Telepsychiatry-based care for the treatment follow-up of Iranian war veterans with post-traumatic stress disorder: a randomized controlled trial. Iran J Med Sci 2019 Jul;44(4):291-298 [FREE Full text] [CrossRef] [Medline]
    65. Gros DF, Lancaster CL, López CM, Acierno R. Treatment satisfaction of home-based telehealth versus in-person delivery of prolonged exposure for combat-related PTSD in veterans. J Telemed Telecare 2018 Jan;24(1):51-55. [CrossRef] [Medline]
    66. Botella C, Gallego MJ, Garcia-Palacios A, Guillen V, Baños RM, Quero S, et al. An internet-based self-help treatment for fear of public speaking: a controlled trial. Cyberpsychol Behav Soc Netw 2010 Aug;13(4):407-421. [CrossRef] [Medline]
    67. Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev 2015 Mar 05(3):CD011565. [CrossRef] [Medline]
    68. Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
    69. Sunjaya AP, Chris A, Novianti D. Efficacy, patient-doctor relationship, costs and benefits of utilizing telepsychiatry for the management of post-traumatic stress disorder (PTSD): a systematic review. Trends Psychiatry Psychother 2020;42(1):102-110 [FREE Full text] [CrossRef] [Medline]
    70. Domhardt M, Steubl L, Baumeister H. Internet- and mobile-based interventions for mental and somatic conditions in children and adolescents. Z Kinder Jugendpsychiatr Psychother 2020 Jan;48(1):33-46 [FREE Full text] [CrossRef] [Medline]
    71. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual research review: digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry 2017 Apr;58(4):474-503. [CrossRef] [Medline]
    72. Costanza A, Mazzola V, Radomska M, Amerio A, Aguglia A, Prada P, et al. Who consult an adult psychiatric emergency department? pertinence of admissions and opportunities for telepsychiatry. Medicina (Kaunas) 2020 Jun 13;56(6):295 [FREE Full text] [CrossRef] [Medline]
    73. Deslich S, Stec B, Tomblin S, Coustasse A. Telepsychiatry in the 21(st) century: transforming healthcare with technology. Perspect Health Inf Manag 2013;10:1f [FREE Full text] [Medline]
    74. Bhugra D, Tasman A, Pathare S, Priebe S, Smith S, Torous J, et al. The WPA-Lancet Psychiatry Commission on the future of psychiatry. Lancet Psychiatry 2017 Oct;4(10):775-818. [CrossRef] [Medline]
    75. Schueller SM, Washburn JJ, Price M. Exploring mental health providers' interest in using web and mobile-based tools in their practices. Internet Interv 2016 May;4(2):145-151 [FREE Full text] [CrossRef] [Medline]
    76. Goldstein F, Glueck D. Developing rapport and therapeutic alliance during telemental health sessions with children and adolescents. J Child Adolesc Psychopharmacol 2016 Apr;26(3):204-211. [CrossRef] [Medline]
    77. Weightman M. Digital psychotherapy as an effective and timely treatment option for depression and anxiety disorders: Implications for rural and remote practice. J Int Med Res 2020 Jun 12;48(6):030006052092868. [CrossRef] [Medline]
    78. Ramalho R, Adiukwu F, Gashi Bytyçi D, El Hayek S, Gonzalez-Diaz JM, Larnaout A, et al. Telepsychiatry and healthcare access inequities during the COVID-19 pandemic. Asian J Psychiatr 2020 Oct;53:102234 [FREE Full text] [CrossRef] [Medline]
    79. Gerke S, Stern AD, Minssen T. Germany's digital health reforms in the COVID-19 era: lessons and opportunities for other countries. NPJ Digit Med 2020;3:94 [FREE Full text] [CrossRef] [Medline]
    80. Corruble E. A viewpoint from Paris on the COVID-19 pandemic: a necessary turn to telepsychiatry. J Clin Psychiatry 2020 Mar 31;81(3):20com13361 [FREE Full text] [CrossRef] [Medline]
    81. Stewart R, Martin E, Broadbent M. Mental health service activity during COVID-19 lockdown: South London and Maudsley data on working age community and home treatment team services and mortality from February to mid-May 2020. medRxiv. Published online on June 16, 2020 2021 [FREE Full text] [CrossRef]
    82. David KB, Solomon JK, Yunusa I, Lawal BK, Marshal CS, Okereke M, et al. Telemedicine: an imperative concept during COVID-19 pandemic in Africa. Pan Afr Med J 2020;35(Suppl 2):129 [FREE Full text] [CrossRef] [Medline]
    83. Looi JC, Pring W. Private metropolitan telepsychiatry in Australia during Covid-19: current practice and future developments. Australas Psychiatry 2020 Oct;28(5):508-510. [CrossRef] [Medline]
    84. APA calls for comprehensive telehealth coverage. American Psychological Association Services, Inc. 2020 Apr 02.   URL: https://www.apaservices.org/practice/reimbursement/health-codes/comprehensive-telehealth-coverage [accessed 2020-06-13]
    85. Sammons MT, VandenBos GR, Martin JN. Psychological practice and the COVID-19 crisis: a rapid response survey. J Health Serv Psychol 2020 May 08:1-7 [FREE Full text] [CrossRef] [Medline]
    86. Moreno C, Wykes T, Galderisi S, Nordentoft M, Crossley N, Jones N, et al. How mental health care should change as a consequence of the COVID-19 pandemic. Lancet Psychiatry 2020 Sep;7(9):813-824 [FREE Full text] [CrossRef] [Medline]
    87. Pruitt LD, Vuletic S, Smolenski DJ, Wagner A, Luxton DD, Gahm GA. Predicting post treatment client satisfaction between behavioural activation for depression delivered either in-person or via home-based telehealth. J Telemed Telecare 2019 Sep;25(8):460-467. [CrossRef] [Medline]
    88. Jenkins-Guarnieri MA, Pruitt LD, Luxton DD, Johnson K. Patient perceptions of telemental health: systematic review of direct comparisons to in-person psychotherapeutic treatments. Telemed J E Health 2015 Aug;21(8):652-660. [CrossRef] [Medline]
    89. Harper Shehadeh MJ, Abi Ramia J, Cuijpers P, El Chammay R, Heim E, Kheir W, et al. Step-by-step, an e-mental health intervention for depression: a mixed methods pilot study from Lebanon. Front Psychiatry 2019;10:986 [FREE Full text] [CrossRef] [Medline]
    90. Lemert EM, Frank JD. Persuasion and Healing: A Comparative Study of Psychotherapy. Baltimore: John Hopkins University Press; 1991.
    91. Campbell LF, Norcross JC, Vasquez MJT, Kaslow NJ. Recognition of psychotherapy effectiveness: the APA resolution. Psychotherapy (Chic) 2013 Mar;50(1):98-101. [CrossRef] [Medline]
    92. Rohland B. Telepsychiatry in the heartland: if we build it, will they come? Community Ment Health J 2001 Oct;37(5):449-459. [CrossRef] [Medline]
    93. Farabee D, Calhoun S, Veliz R. An experimental comparison of telepsychiatry and conventional psychiatry for parolees. Psychiatr Serv 2016 May 01;67(5):562-565. [CrossRef] [Medline]
    94. Newbould L, Mountain G, Ariss S, Hawley M. Remote health care provision in care homes in England: an exploratory mixed methods study of Yorkshire and the Humber. Technologies 2019 Feb 15;7(1):24 [FREE Full text] [CrossRef]

    ‎
    COVID-19: coronavirus disease 2019
    HIV: human immunodeficiency virus

    Edited by J Torous; submitted 01.12.20; peer-reviewed by V Strotbaum, P Yellowlees; comments to author 07.02.21; revised version received 13.02.21; accepted 13.05.21; published 29.07.21

    Copyright

    ©Raffaele Mazziotti, Grazia Rutigliano. Originally published in JMIR Mental Health (https://mental.jmir.org), 29.07.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.