This paper is in the following e-collection/theme issue:

Mobile Health in Psychiatry (426) E-Health / Health Services Research and New Models of Care (517) Use and User Demographics of mHealth (335)

Published on in Vol 6, No 11 (2019): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14018, first published .
Acceptance and Expectations of Medical Experts, Students, and Patients Toward Electronic Mental Health Apps: Cross-Sectional Quantitative and Qualitative Survey Study

Acceptance and Expectations of Medical Experts, Students, and Patients Toward Electronic Mental Health Apps: Cross-Sectional Quantitative and Qualitative Survey Study

Acceptance and Expectations of Medical Experts, Students, and Patients Toward Electronic Mental Health Apps: Cross-Sectional Quantitative and Qualitative Survey Study

Authors of this article:

Gwendolyn Mayer1 Author Orcid Image ;   Nadine Gronewold1 Author Orcid Image ;   Simone Alvarez2 Author Orcid Image ;   Bastian Bruns1 Author Orcid Image ;   Thomas Hilbel3 Author Orcid Image ;   Jobst-Hendrik Schultz1 Author Orcid Image
Article Authors Cited by (44) Tweetations (4) Metrics

    Original Paper

    1Department of General Internal Medicine and Psychosomatics, Heidelberg University Hospital, Heidelberg, Germany

    2Medical Faculty of Heidelberg, Heidelberg University, Heidelberg, Germany

    3Electrical Engineering and Applied Sciences, Westphalian University of Applied Sciences, Gelsenkirchen, Germany

    Corresponding Author:

    Gwendolyn Mayer, DipPsych

    Department of General Internal Medicine and Psychosomatics

    Heidelberg University Hospital

    Im Neuenheimer Feld 130.3

    Heidelberg, 69120

    Germany

    Phone: 49 62215635685

    Fax:49 6221567505

    Email: gwendolyn.mayer@med.uni-heidelberg.de


    Background: The acceptability of electronic mental (e-mental) health apps has already been studied. However, the attitudes of medical experts, students, and patients taking into account their knowledge of and previous experiences with e-mental health apps have not been investigated.

    Objective: The aim of this study was to explore the attitudes, expectations, and concerns of medical experts, including physicians, psychotherapists and nursing staff, students of medicine or psychology, and patients toward e-mental health apps when considering their knowledge of and former experiences with e-mental health apps.

    Methods: This cross-sectional quantitative and qualitative survey was based on a self-developed questionnaire. A total of 269 participants were included (104 experts, 80 students, and 85 patients), and 124 eligible participants answered a paper version and 145 answered an identical online version of the questionnaire. The measures focused on existing knowledge of and experiences with e-mental health apps, followed by a question on whether electronic health development was generally accepted or disliked. Further, we asked about the expectations for an ideal e-mental health app and possible concerns felt by the participants. All items were either presented on a 5-point Likert scale or as multiple-choice questions. Additionally, 4 items were presented as open text fields.

    Results: Although 33.7% (35/104) of the experts, 15.0% (12/80) of the students, and 41.2% (35/85) of the patients knew at least one e-mental health app, few had already tried one (9/104 experts [8.7%], 1/80 students [1.3%], 22/85 patients [25.9%]). There were more advocates than skeptics in each group (advocates: 71/104 experts [68.3%], 50/80 students [62.5%], 46/85 patients [54.1%]; skeptics: 31/104 experts [29.8%], 20/80 students [25.0%], 26/85 patients [30.6%]). The experts, in particular, believed, that e-mental health apps will gain importance in the future (mean 1.08, SD 0.68; 95% CI 0.94-1.21). When asked about potential risks, all groups reported slight concerns regarding data security (mean 0.85, SD 1.09; 95% CI 0.72-0.98). Patient age was associated with several attitudes toward e-mental health apps (future expectations: r=–0.31, P=.005; total risk score: r=0.22, P=.05). Attitudes toward e-mental health apps correlated negatively with the professional experience of the experts (rs(94)=–0.23, P=.03).

    Conclusions: As opposed to patients, medical experts and students lack knowledge of and experience with e-mental health apps. If present, the experiences were assessed positively. However, experts show a more open-minded attitude with less fear of risks. Although some risks were perceived regarding data security, the attitudes and expectations of all groups were rather positive. Older patients and medical experts with long professional experience tend to express more skepticism.

    Trial Registration: German Clinical Trials Register DRKS00013095; https://www.drks.de/drks_web/navigate.do? navigationId=trial.HTML&TRIAL_ID=DRKS00013095

    JMIR Ment Health 2019;6(11):e14018

    doi:10.2196/14018

    Keywords

    acceptance; telemedicine; telehealth; eHealth; mHealth; interventions; depression; patients; students; experts; expectation; risk 


    Background

    Smartphone apps for mental disorders, so called electronic mental (e-mental) health apps, have the potential to deliver immediate therapeutic help for various illnesses like substance abuse, bipolar disorders, depression, anxiety, psychosis, and even suicide [1]. They offer support at any time and place and provide context-aware interventions and real-time feedback. A recent review found 165 primary research studies on smartphone interventions for mental health in 2017-2018, and much evidence has been provided according to the effectiveness of these interventions [2]. In particular, patients suffering from depression benefit from e-mental health apps [3], and some clinical experts have welcomed this development to empower patients toward improved care [4].

    Nevertheless, e-mental health apps available in the app stores of the market leaders Google and Apple are rarely clinically validated, and only a few are registered under the European Regulation on medical devices [5]. Thus, it is hardly surprising that there is criticism about the potential adverse effects such as low quality of therapeutic content or replacement of health care contacts [6]. In fact, the real-world user engagement of e-mental health apps beyond the clinical setting is rather low. There is a high attrition rate due to drop outs after a few days or weeks of use [7]. Recently, Fleming and Bavin [8] showed that the completion or sustained use of these programs varied from 0.5% to 28.6% [8]. One of the reasons for this gap between clinical trials and real-world engagement lies in different target populations of trials and people using apps in the real world [9]. However, some authors found that low usability, concerns about privacy, and a lack of trust prevent potential users to create the necessary confidence in e-mental health apps [10].

    To bridge this gap and make clinically valid, effective interventions available to the broad public, a deeper understanding of the attitudes of all stakeholders is necessary. These are first and foremost patients in clinical and outpatient settings, medical experts who are in close contact with the psychotherapeutic process (such as physicians, psychological and medical psychotherapists, and nursing staff) and finally, future professionals who are current students of medicine or psychology.

    Attitudes of Patients

    Previous studies on attitudes toward e-mental health apps observed that the majority of the participants prefer face-to-face therapy over Web-based interventions. Interestingly, anonymity is the least important concern for rejecting e-mental health apps, while helpfulness, credibility, and accessibility are more important [11]. Research on privacy concerns reveals an inconsistent picture: The intention of patients to share personal health information with health care providers, in general, is highly influenced by privacy concerns [12]. This may result in a lower willingness to trust e-mental health apps, which of course implies sharing intimate experiences with software of unknown origin. In qualitative interviews, patients have expressed concerns about becoming dependent on apps or of losing social support [13].

    Despite many doubts, patients also see advantages in using apps targeting mental disorders. These possibilities range from the acquisition of new skills, social connectedness, and feelings of a “safety netting” [13] to a deeper understanding of personal mood and triggers of their mental health problems [14] and even alarm functions and reminders for clinical appointments for patients with psychosis [15]. Internet interventions, in general, were rated as helpful, while guided programs or videoconferencing were preferred over unguided self-help programs [16]. Recent results show that especially patients with negative care experiences tend to prefer electronic health (eHealth) services, in particular, those with lower educational levels [17].

    The relationship with physicians also seems to play a role in the acceptance of e-mental health apps for patients. A strong “doctor-related locus of control” has been negatively associated with the intention to use e-mental health apps [18]. Similarly, the willingness of patients to use e-mental health apps and programs depends to a high degree on their acceptance by the respective clinician [19] and even on the awareness of experts in teaching related topics to their medical students [20].

    Attitudes of Medical Experts

    Physicians in Germany show a positive attitude toward future eHealth developments, in general; nonetheless, some voice concerns about immature technology and neglected privacy [21]. However, only half of the established physicians in Germany feel adequately informed about these developments [22].

    What expectations of and knowledge about e-mental health apps do physicians really have? A glance at topics in American mental health–related conferences in 2013-2015 shows that only 0.3% of the sessions addressed e-mental health apps [20]. This number, of course, may be higher today. A closer look reveals the underlying divided opinions of medical experts about digital health interventions. On the one hand, mental health care staff fear that internet-based services could replace face-to-face support. On the other hand, access to helpful information at any time and place, the possibility to express oneself in forums, and the incorporation of psychoeducational material is perceived as a great asset. Finally, internet-based services have been seen to possibly lower the threshold to initiate psychotherapy [23]. One recent study, which specifically investigated the attitudes of physicians and psychotherapists, showed that experts doubt the possibility of effective treatment via the internet, but they regard telemedicine as a possible potential supplement to conventional face-to-face therapy [24]. A direct comparison of the attitudes of patients with depression and psychotherapists found more negative attitudes among psychotherapists than patients. Similarly, patients in clinical settings seemed to be more skeptical than patients recruited via the internet [25].

    Missing Aspects

    Three important aspects of attitudes toward e-mental health are missing in the studies highlighted above. First, the opinion of the future medical experts, that is, students of medicine or psychology, has not been studied. Apart from the integration of eHealth topics in single universities in the medical curriculum [26,27] and the willingness to use mental health apps by students themselves [28,29], data on the attitudes of students toward the topic are limited.

    Second, the attitudes and expectations of the nursing staff, who are in close contact with patients who have used mental health apps or plan to do so, need to be considered. A study that inquired about their views on eHealth development, in general, not specific to e-mental health apps, reported that staff fear a loss of quality in social interaction caused by care robots while benefiting from process improvements due to digital documentation systems [22]. However, their opinions specifically about e-mental health apps are missing.

    Third, and probably most important, none of the studies have asked the participants about their previous knowledge and personal experiences with e-mental health apps. It is conceivable that this contributes to the insights delivered by the research on user engagement mentioned in the beginning. Despite a growing amount of evidence-based interventions available on e-mental health apps, the dropout rates are high. This has been explained by poor usability, lack of trust, and concerns about missing data security [10]. The following question arises: Have the reservations about these tools increased or decreased depending on existing knowledge and personal experiences with e-mental health apps?

    In this study, we have integrated these missing aspects in our research design to contribute to a deeper understanding of the barriers and facilitators of real-world engagement of e-mental health apps.

    Objective

    This study investigated the attitudes, expectations, and concerns toward e-mental health apps of physicians, psychological psychotherapists, psychotherapists in training, nurses, students of medicine or psychology, and patients with relevant support needs who could be prospective users of e-mental health apps.

    The main objectives were based on the following research questions: How many e-mental health apps do the participants know of? How many have they tried and what were their experiences? Are e-mental health apps accepted or disliked, and for what reasons? What expectations do these groups have and what risks do they see?

    Finally, the study also reports on the possible determinants of attitudes like age and sex of all groups as well as the number of years of professional experience of the medical experts.


    Study Design

    We conducted a cross-sectional quantitative and qualitative study using a questionnaire that was designed for the purpose of this survey. Data were collected between September 2017 and June 2018. The questionnaire was distributed either as a Web-based version via the online tool SoSci Survey [30] or in a paper-pencil version. We obtained a sample of 269 participants; 124 eligible participants answered the paper version and 145 answered the online version.

    The study “Acceptance and expectations of experts, students and patients according to health apps for mental disorders” was registered in the German Clinical Trials Register (DRKS00013095).

    Target Population and Recruitment

    This study focused on physicians in the disciplines of General Medicine, Internal Medicine, Psychosomatics, Psychiatry, or Psychotherapy as well as psychological psychotherapists, trainee psychotherapists and nurses, students of medicine or psychology, and patients with a psychosomatic disorder.

    Recruitment was performed with postings and mail distribution services to medical experts and students within the University Hospital of Heidelberg’s Department of Internal Medicine and Psychosomatics in the Medical Faculty and in the Psychological Institute of the University of Heidelberg. More than 120 patients of the inpatient and outpatient services in the Department of Internal Medicine were approached with the paper-pencil version. Further participants were recruited via the internet. We wrote more than 800 personalized emails to physicians and psychotherapists published by the Association of Statutory Health Insurance Physicians. Further, students and patients received an invitation via Facebook, XING, LinkedIn, SurveyCircle, deutsche depressionsliga, and Diskussionsforum Depression. All groups and forums gave their consent in advance. Patients were excluded if they stated that they did not have a mental disorder or another disease that affects mental health (in the Web version only). No reward was given for participation in the study.

    Ethical approval was obtained by the Ethics Commission of the Medical Faculty of Heidelberg (S336-2017) prior to data collection.

    Sample Description

    We collected 285 completed questionnaires. We then excluded 16 questionnaires; in one case, we received no informed consent, and 15 participants did not fulfill the inclusion criteria.

    The sample consisted of 269 participants aged between 18 and 77 years (mean 37.39 years, SD 14.14 years). The demographic characteristics of the sample are presented in Table 1. Nearly two-thirds were female (173/269 [64.3%]). The sample comprised 104 medical experts, 80 students, and 85 patients. The questionnaire was filled in by 43 physicians, 33 psychological psychotherapists, 16 psychotherapists in training, and 13 nurses. As one physician was also a psychotherapist in training, there were 104 experts in total. Of the 80 students, 54 were students of medicine and 28 were students of psychology (two of them studied both). Finally, 41 patients were recruited from the University Hospital of Heidelberg (Internal Medicine), and 44 patients were recruited via the internet.

    Table 1. Descriptive statistics of the study sample (N=269).
    CharacteristicsParticipants, n (%)
    Age (years), mean (SD)37.39 (14.14)
    Sex

    		Female173 (64.31)

    		Male96 (35.69)

    		Other0 (0)
    Nationality

    		German251 (93.31)

    		Other18 (0.69)
    Family status

    		Single158 (58.74)

    		Married86 (31.97)

    		Separated4 (1.49)

    		Divorced14 (5.20)

    		Widowed1 (0.37)

    		Other5 (1.86)

    		Missing1 (0.37)
    Profession (experts)a

    		General medical practitioner7 (6.73)

    		Specialist in internal medicine23 (22.12)

    		Specialist in psychosomatics12 (11.54)

    		Specialist in psychiatry9 (8.65)

    		Specialist in psychotherapy6 (5.77)

    		Physician (total)43 (41.35)

    		Psychological psychotherapist33 (31.73)

    		Psychotherapist in training16 (15.38)

    		Nurse13 (12.50)

    		Experts (total)104 (100)
    Subject (students)a

    		Medicine54 (67.50)

    		Psychology28 (35.00)

    		Students (total)80 (100)
    Patients

    		Patient of the University Hospital Heidelberg41 (48.24)

    		Patient recruited via the internet44 (51.76)

    		Patients (total)85 (100)
    Education (patients)a

    		Still in school1 (1.18)

    		Secondary school13 (15.29)

    		Secondary high school24 (28.24)

    		Higher school certificate26 (30.59)

    		Study exam18 (21.18)

    		Other3 (3.53)

    aMultiple choice possible.

    Measures

    We developed a 10-minute, structured questionnaire that consisted of two major parts: The first section was titled “Demographics” and contained nine items; the second section was titled “Attitudes towards e-mental health apps” and contained 25 items. We distributed three versions of the questionnaire to the various target groups: medical experts, students, and patients. The versions differed slightly regarding the first part.

    The sociodemographic data obtained were age, sex, nationality, marital status, socioeconomic status, and either profession or study subject. The medical experts were further asked to state their professional experience in years.

    The items of the section “Attitudes towards e-mental health apps,” the results of which are reported in this article, are listed in Table 2 and sorted by issues in the same order presented in the questionnaire.

    The first part of the section “Attitudes towards e-mental health apps” addressed individual knowledge and prior experiences with the most common e-mental health apps at the time of the survey. If present, the experiences could be rated with a 5-point Likert scale from “negative” to “positive.” After that, the questions asked for attitudes toward e-mental health apps, in general. The participants were asked to choose one option of five statements, from “I am concerned about the development” to “I think, there’s great potential in the development.” A further item asked for the participant’s opinion about whether e-mental health apps will gain importance in the future, with a 5-point Likert scale ranging from “no” to “yes.”

    The next section of the questionnaire asked for expectations toward an ideal e-mental health app in eight statements (ie, “Privacy should be respected” or “The design should be appealing”). Another part asked for four different risks referring to no helpfulness, harmfulness to health, loss of social contacts, and lack of data security. In this section, the Cronbach alpha was .56 as well. Both parts, expectations and risks, were presented as a 5-point Likert scale from “not important” to “very important” and “no risk” to “high risk,” respectively.

    Additionally, four items were presented as open-text fields in order to give the participants the opportunity to express their opinions in a more detailed way.

    Table 2. Items of the section “Attitudes towards e-mental health apps” sorted by issues.
    Issue and itemType
    Knowledge

    		Which of the following apps do you know? (list)Multiple choice
    Experiences

    		Which of the following apps did you already try? (list)Multiple choice

    		How do you rate your experiences?5-point Likert scalea
    Attitudes (1)

    		What do you think in general of e-mental health apps?Rank order (ordinal)
    Attitudes (2)

    		Do you think, that e-mental health apps will gain importance in the future?5-point Likert scalea
    Positive aspects

    		Which positive aspects do you see regarding e-mental health apps?Open-text field
    Expectations toward an ideal e-mental health app

    		What functions or properties would you like to have in an ideal e-mental health app? (8 sub items)5-point Likert scalea

    		Further functions or propertiesOpen-text field
    Negative aspects

    		What would stop you from using an e-mental health app?Open-text field
    Risks

    		What risks do you see in e-mental health apps? (4 sub items)5-point Likert scalea

    		Further risksOpen-text field

    aRange: –2 to +2.

    Statistical Analysis

    Data analysis was conducted using SPSS Statistics (version 24; IBM Corp, Armonk, New York) [31]. The data of the three groups were recorded simultaneously and analyzed for group differences. The preliminary exploration of the descriptive statistics was performed by calculating frequencies, means and SDs, and reporting 95% CIs. The range of all continuous items was coded from –2 to +2.

    We explored differences between the three groups in attitudes by using the Pearson chi-square tests for attitudes rated on an ordinal scale and by using a 1-way analysis of variance (ANOVA) for attitudes rated on a 5-point Likert scale. In case of variance homogeneity, we calculated post-hoc tests according to Scheffé; if variance homogeneity was missing, we chose Dunnett-T3 due to its ability to discover even small differences among groups [32].

    To explore the expectations toward an ideal e-mental health app, we carried out descriptive measures and an additional factor analysis (root cause analysis) in order to identify main components of the expectations toward an ideal e-mental health app and conducted a further ANOVA with the factors. The Cronbach alpha was calculated to assess the internal consistency of this section. It was rather low with an α=.56, which could be due to the heterogeneity of the concept [33].

    The risks seen by the three groups were calculated via descriptive measures and an ANOVA. Additionally, we calculated a total risk score by averaging the four risks.

    Looking for possible determinants of attitudes, expectations, and risks, we calculated correlations for age and professional experience of the experts. When appropriate, we calculated Pearson correlations (r) and Spearman rank correlations (rs). We looked for sex differences by calculating the Mann-Whitney U tests or an ANOVA, depending on the scale levels of the items.

    The qualitative data in open-text fields were analyzed manually by building inductive categories and taking into account the recommendations of content analysis and its possible quantification of categories following Mayring [34].


    Knowledge and Previous Experiences With Electronic Mental Health Apps

    Of a short list of common e-mental health apps, 33.7% (35/104) of the experts and 15.0% (12/80) of the students indicated that they knew at least one of the apps (Table 3). The percentage of experts and students who had already tried one e-mental health app was 8.7% (9/104) and 1.3% (1/80), respectively. In the group of the patients, 41.2% (35/85) knew at least one app and 25.9% (22/85) had at least tried one app. The patients who had already tried at least one app were patients of the University Hospital of Heidelberg in three cases, and the other 19 were recruited via the internet.

    When prior experiences with an e-mental health app were present, they were evaluated as positive. The nine experts who stated to already have experiences with an app rated the experiences with a mean of 1.22 (SD 0.44, 95% CI 0.88-1.56). The 22 patients rated their experiences with a mean of 1.18 (SD 0.96, 95% CI 0.76-1.61).

    Table 3. Results for the question, “Which of the following apps do you know?”
     AppExperts (n=54), nStudents (n=16), nPatients (n=58), nTotal (N=128), n (%)
    ARYA2147 (5.47)
    DepressionsCoach (TK)144927 (21.0)
    Deprexis24160319 (14.84)
    Human Progress0325 (3.91)
    Meplus0011 (0.78)
    Minddistrict2024 (3.13)
    Moodgym5049 (7.03)
    Moodpath121114 (10.94)
    MyTherapy24713 (10.16)
    Novego0011 (0.78)
    Selfapy52310 (7.81)
    Others701118 (14.10)

    Attitudes Toward Electronic Mental Health Apps

    Quantitative Results

    There were more proponents than skeptics against e-mental health apps (Table 4): 28.6% answered skeptically by choosing one of the first two options (31/104 [29.8%] experts, 20/80 [25.0%] students, 26/85 [30.6%] patients), 62.1% showed positive attitudes by choosing one of the last two options (71/104 [68.3%] experts, 50/80 [62.5%] students, 46/85 [54.1%] patients), and 7.4% were neutral (1/104 [1.0%] experts, 10/80 [12.5%] students, 9/85 [10.6%] patients). Experts expressed significantly more positive attitudes than skepticism compared with students and patients. The differences were significant (χ²3=11.45, n=184, P=.01 [experts vs students]; χ²3=12.19, n=189, P=.01 [experts vs patients]), but not for students compared with patients.

    Patients of the clinic expressed skepticism (15/41 [36.6%]), positive attitudes (18/41 [43.9%]), and neutral attitudes (4/41 [9.8%]; missing: 4/41 [9.8%]), while patients recruited via the internet answered skeptically (11/44 [25.0%]), positively (28/44 [63.6%]), and neutrally (5/44 [11.4%]). This difference was not significant (χ²3=6.80, n=85, P=.08).

    The three groups differed in their opinion about whether e-mental health apps will gain importance in the future (F2,263=7.64, P=.001). The experts believed this (mean 1.08, SD 0.68, 95% CI 0.94-1.21), while students and patients did not share this attitude (students: mean 0.79, SD 0.85, 95% CI 0.60-0.98; patients: mean 0.60, SD 0.99, 95% CI 0.39-0.82). There was a significant difference between experts and patients (Dunnett-T3: Mdiff [mean difference]=0.48, SE 0.13, P=.001, 95% CI 0.17-0.78) and between experts and students (Mdiff=0.29, SE 0.12, P=.04, 95% CI 0.01-0.57) but not between students and patients (Mdiff=0.19, SE 0.14, P=.49, 95% CI –0.53 to 0.16).

    Table 4. Results for the question, “What do you think in general of e-mental health apps?”
    ResponseExperts (n=104), nStudents (n=80), nPatients (n=85), nTotal (N=269), n (%)
    I think, there\'s great potential in the development.22151653 (19.70)
    I\'m basically in favour of the development.493530114 (42.38)
    I\'m in favour of the development of health apps, but not for mental disorders.110920 (7.43)
    I am sceptical about the development.25162465 (24.16)
    I am concerned about the development.64212 (4.46)
    Not answered1045 (1.86)
    Qualitative Remarks

    Many participants (n=199) took the opportunity to comment by answering the question: “Which positive aspects do you see regarding e-mental health apps?” These data were analyzed using content analysis by building inductive categories as described above. The most frequent category of answers was that an e-mental health app may deliver low-threshold access to psychotherapy. This view was shared by 70 participants (experts: 36, students: 21, patients: 13). Further, 53 statements expressed the belief that such an app improved everyday support (experts: 23, students: 9, patients: 21), and 50 remarks referred to improved self-management with such apps (experts: 23, students: 13, patients: 14). Finally, 23 statements mentioned the possibility of documentation/monitoring of therapeutic progress (experts: 7, students: 6, patients: 10). In particular, the experts believed that an e-mental health app had a good chance to improve psychoeducation (22 remarks in total; experts: 16, students: 5, patients: 1).

    Expectations

    Quantitative Results

    The highest rated expectation toward an ideal e-mental health app was “privacy should be respected” with a mean score of 1.85 (SD 0.59, 95% CI 1.78-1.93). The lowest rated item was “it should be changeable and adaptable by me” with a mean of 0.66 (SD 1.14, 95% CI 0.52-0.80). All results are presented in detail in Table 5.

    By principal component analysis, three dimensions could be extracted from the initial eight items: transparency, costs, and design/customizability; these accounted for 55.1% of the variance (Table 6). A subsequent ANOVA showed that the groups differ in their expectations (F2,261=3.28, P=.04). The experts attached more importance to transparency than the patients (Scheffé: Mdiff=0.38, SE 0.15, P=.04, 95% CI 0.01-0.74), while the latter put more emphasis on costs compared to the experts (Mdiff=0.59, SE 0.14, P<.001, 95% CI 0.23-0.94) and design/customizability compared to the experts (Mdiff=0.45, SE 0.15, P=.01, 95% CI 0.09-0.81). All other comparisons were without significant differences.

    Table 5. Results for the question, “What functions or properties would you expect in an ideal e-mental health app?” (for all: min=–2, max=2).
    ItemParticipants (N=269), n (%)Mean (SD)95% CI
    It should be changeable and adaptable by me.267 (99.26)0.66 (1.14)0.52-0.80
    It should not cost much.266 (98.88)1.05 (1.06)0.92-1.18
    It should be covered by the health insurance.266 (98.88)0.67 (1.20)0.52-0.81
    The purpose of the exercises should be clear and concise.267 (98.26)1.61 (0.68)1.53-1.69
    It should be clear who designed the app.268 (99.63)1.04 (1.14)0.90-1.18
    There should be scientific evidence of efficacy.268 (99.63)1.21 (0.92)1.10-1.32
    The design should be appealing.268 (99.63)1.13 (0.91)1.02-1.24
    Privacy should be respected.268 (99.63)1.85 (0.59)1.78-1.93
    Table 6. Matrix of components after varimax-rotation (the rotation is converged in 5 iterations; method of extraction: main component analysis).
    ComponentTransparencyCostsDesign/customizability
    It should be clear who designed the app.0.72–0.17–0.08
    There should be scientific evidence of efficacy.0.700.030.03
    Privacy should be respected.0.620.160.10
    The purpose of the exercises should be clear and concise.0.540.400.12
    It should be covered by the health insurance.0.110.710.25
    It should not cost much.–0.050.79–0.18
    The design should be appealing.0.420.26–0.56
    It should be changeable and adaptable by me.0.250.180.77
    Eigenvalue2.181.221.01
    Percentage of total variance27.2715.2512.56
    Total variance—a—a55.09b

    aNot applicable.

    bDeviations due to rounding.

    Qualitative Remarks

    The quantitative results are supported by a closer look at the statements made in open-text fields: Of the 19 statements in total, those referring to usability and customizability were nearly all expressed by students and patients (usability: experts: 1, students: 4, patients: 1; customizability: experts: 0, students: 0, patients: 2). The remarks of the experts in contrast referred to an improved risk management and possibilities for the patients to get in contact with a psychotherapist (risk management: experts: 3, students: 1, patients: 1; contact: experts: 2, students: 0, patients: 1).

    Risks

    All groups had concerns regarding the lack of data protection (mean 0.85, SD 1.09, 95% CI 0.72-0.98). All results are presented in Table 7. An ANOVA showed that there were no differences between the groups except in one item: “The exercises don’t help.” Students and patients showed significantly more concerns than experts (F2,261=6.03, P=.003; students vs experts: Mdiff=0.48, SE 0.15, P=.004, 95% CI 0.13-0.83; patients vs experts: Mdiff=0.42, SE 0.16, P=.003, 95% CI 0.04-0.81).

    The mean total score of all four risks was 0.11 (SD 0.74, 95% CI 0.02-0.19). There was no significant result after comparison of the three groups via an ANOVA (F2,261=0.28, P=.76).

    Table 7. Results for the question, “What risks do you see in e-mental health apps?” (for all: min=–2, max=2).
    ItemParticipants, n (%)Mean (SD)95% CI
    The exercises don’t help.

    		Experts103 (99.04)–0.01 (1.04)–0.21 to 0.19

    		Students79 (98.75)0.47 (0.92)0.26 to 0.67

    		Patients82 (96.47)0.41 (1.12)0.17 to 0.66

    		Total264 (98.14)0.27 (1.05)0.14 to 0.39
    The exercises are harmful for the health.

    		Experts103 (99.04)–0.50 (0.96)–0.68 to –0.31

    		Students79 (98.75)–0.46 (0.95)–0.67 to –0.24

    		Patients81 (95.29)–0.47 (1.21)–0.74 to –0.20

    		Total263 (97.77)–0.48 (1.03)–0.60 to –0.35
    By using such apps social contacts get lost.

    		Experts104 (100.00)–0.16 (1.32)–0.42 to 0.09

    		Students79 (98.75)–0.25 (1.21)–0.53 to 0.02

    		Patients83 (97.65)–0.31 (1.42)–0.62 to 0.00

    		Total266 (98.88)–0.24 (1.32)–0.40 to –0.08
    The data are not protected.

    		Experts104 (100.00)0.91 (1.04)0.71 to 1.12

    		Students79 (98.75)0.73 (1.11)0.49 to 0.98

    		Patients83 (97.65)0.88 (1.13)0.63 to 1.13

    		Total266 (98.88)0.85 (1.09)0.72 to 0.98
    Qualitative Remarks

    The item “What would stop you from using an e-mental health app?” received 69 comments; all three groups expressed the fear that the development of e-mental health apps could promote a tendency toward the “transparent patient” caused by a missing protection of privacy. Missing scientific background of an e-mental health app was named as the second most frequent barrier (31 comments) followed by the fear of replacement of real-life psychotherapy (25 comments). Another question for specific risks demonstrated that some of the experts and patients (n=7 in total) expressed their concerns that an e-mental health app could become a substitute for real-life psychotherapy. Finally, 8 participants stated that the patient is left alone without a feedback by using e-mental health apps. Another participant, a patient, wrote: “One is reminded of the illness. Every day.”

    Possible Determinants of Attitudes, Expectations, and Risks

    Age

    There was no significant correlation between age and general attitudes toward e-mental health apps. We found a slight but nonsignificant negative correlation of general attitudes with the age of the experts (rs(103)=–0.17, P=.09; students: rs(80)=–0.16, P=.15; patients: rs(81)=–0.07, P=.53).

    The expectation that e-mental health apps will gain importance in the future correlated negatively with the age of the patients (r=–0.31, P-.005; experts: r=–0.19, P=.05; students: r=–0.01, P=.95).

    The expectations toward an ideal e-mental health app were not related to the age of the participants, except in the factor transparency in the group of the patients: The older the patient, the more emphasis he or she put on the opinion that an ideal e-mental health app should be transparent (ie, clear who designed the app, scientific evidence, etc) with a highly significant association (r=.35, P=.002; experts: r=–0.01, P=.88; students: r=–0.02, P=.87).

    In the group of patients, age also correlated positively with the total risk score (r=0.22, P=.05; experts: r=0.17, P=.09; students: r=–0.03, P=.81).

    Professional Experience of the Experts

    The professional experience of the experts ranged between 1 and 33 years (physicians: mean 14, SD 10; psychological psychotherapists: mean 17, SD 10; psychological psychotherapists in training: mean 3, SD 2; nurses: mean 18, SD 10).

    Regarding the attitudes toward e-mental health apps, there was a significant negative correlation with the professional experience of the experts (rs(94)=–0.23, P=.03). The longer a medical expert had already practiced his or her profession, the lower was the acceptance of e-mental health apps, in general. The opinion that e-mental health apps will gain importance in the future also correlated negatively with the years of professional experience (r=–0.23, P=.03).

    The professional experience of the medical experts did not correlate with expectations toward an ideal e-mental health app (transparency: r=0.07, P=.50; costs: r=–0.18, P=.09; design/customizability: r=–0.13, P=.21).

    There was a significant positive correlation between professional experience and the total risk score (r=0.21, P=.04).

    Sex

    We found no differences in sex regarding general attitudes (experts: U48,55=519.5, P=.57; students: U27,53=561.0, P=.10; patients: U18,63=519.5, P=.57) or the expectation of the role of e-mental health apps in the future (experts: F1,101=0.25, P=.62; students: F1,78=1.07, P=.30; patients: F1,81=1.39, P=.24).


    Principal Findings

    This study investigated the attitudes, expectations, and concerns toward e-mental health apps of medical experts, students of medicine or psychology, and patients. A special interest was to explore the role of previous knowledge of and former experiences with e-mental health apps made by the participants. Do these factors help reduce reservations about the utility of these tools, and thus, in the long term, increase user engagement? The results showed that in spite of a moderate knowledge of e-mental health apps, there was very little experience with these apps, especially in the group of medical experts and students who will be the future experts. However, a distinct group of patients were already in touch with e-mental health apps; they were not patients of the clinics but patients who joined the study via the internet and had self-reported mental health problems. If present, the personal experiences were rated positively. Trial of an e-mental health app led to a decrease in the reservations about the issue. However, the number of those patients who had personal experiences was too low to allow for conclusions based on further statistics.

    In general, the highest acceptance of the development of e-mental health apps was expressed by the medical experts. They more often expected a growing importance of the topic in the future than students and patients. They recognized that online interventions could reduce the threshold to psychotherapy and deliver potentially useful psychoeducation for patients.

    When asked for the possible risks, all groups reported concerns regarding data security. The experts were less worried about a potential risk of ineffective interventions. This risk was considered more likely by students and patients. Current medical and psychological experts may trust the state of the development more than others. Yet, the results show that experts especially express their wish to be informed about scientific evidence, purpose, and the developers of apps, which was summarized in the factor “transparency” in this study. From the patient’s perspective, financial aspects and customizability were more relevant.

    Regarding the topic e-mental health apps, which is very close to technical innovations, in general, one could expect a strong moderating role of the factor age. In fact, March et al [18] showed earlier that this is only the case for therapist-assisted e-mental health services, but not for self-help interventions. In our results, age, as a determinant for expectations or perceived risks, was only found in the group of the patients (aged 18-77 years). In the group of the experts, the years of professional experience (range: 1-33 years) could explain more than age: The higher the experience, the lower was the acceptance of e-mental health apps, in general, and the expectation that e-mental health apps will gain importance in the future.

    Implications and Recommendations

    The implications of the results should be reflected separately for the three groups investigated.

    A closer look at the attitudes of medical experts reveals that psychotherapists, especially in the surveys of Schröder et al [25] and Tonn et al [24], expressed more critical opinions (in both cases, toward internet interventions, in general) than they did in our study. However, the often-reported fear of therapists of being replaced by internet-based therapy [8] could not be confirmed in our study. Maybe our results help explain the concerns in a more subtle way. Looking at the qualitative remarks of the experts, many therapists do not fear losing their patients to the internet, but the therapeutic relationship could be missing for the patients, which is not the same. The importance of the therapeutic relationship is well investigated and documented [35-37], and the focus of further research should lie on the question of how responsibilities of experts change with parallel therapeutic offers via an app. Some of their patients may rely on possible untrustworthy content of semiprofessional apps, which may be potentially harmful [23]. In this context, we recommend increasing the awareness of changing professional roles by promoting vivid discussions about e-mental health topics via conferences. As reported in the introduction, there is a current lack of exchange on the topic [20], especially among experienced by senior medical experts. Regarding these new responsibilities due to technical innovations, experts should have access to guidelines in order to assess the quality of e-mental health apps, as the German Federal Chamber of Psychotherapists already requires [38].

    The implications have a direct impact on the role of the future experts—the students of medicine or psychology. Although some of them are already in touch with e-mental health apps for their own use, as indicated by studies conducted in the United States [29], students expressed little knowledge and experiences in our study. Regarding the still-growing market of e-mental health apps, assessing the seriousness and scientific evidence of these apps should become part of the curricula. There are only a few years left until today’s students will have to be able to make reasoned decisions and assess the circumstances when confronted with patients who want to use an e-mental health app or already did and were confused by them.

    In this study, patients were the group with the largest amount of knowledge and the widest experience with e-mental health apps. There is a certain group of patients who act autonomously and care for its own recovery and self-management. This goes along with the observation that by being digital, patients increasingly gain independence [39]. Similar to our work, Schröder et al [25] reported that patients in nonclinical settings showed more positive attitudes toward internet interventions than clinical patients [25]. In our study, we found a similar, but not statistically significant, tendency. Recent results show that patients with negative care experiences tend to prefer eHealth services; this is especially true for those patients with a lower educational level [17]. March et al [18] reported that the willingness to use internet-based mental health services depends on the individual diagnosis. Patients with depression are more likely to prefer these services than patients with anxiety [18].

    Based on these results, we recommend a personalized approach. Patients in clinical settings may need encouragement to try an established, validated e-mental health app (eg, for self-management of symptoms during after-care). Patients with negative prior care experiences or fear of clinical settings are difficult to be reached directly. For them, established standards for the development of e-mental health apps are highly needed. These implications support the recommendations referring to the expanded responsibilities of the experts and the future experts, as discussed above.

    Study Limitations

    As part of our study, we presented the participants with a short list of current programs and apps. This list could be complemented manually by the participants, and many of them took advantage of this possibility. Nevertheless, the development of e-mental health apps is a rapidly changing market, and the results may soon become obsolete.

    Furthermore, the willingness to respond to a questionnaire with a focus on e-mental health apps depends to a high degree on an affinity to the internet or technology-related topics. This may be one reason for the positive responses toward the future of e-mental health apps. In particular, patients in the clinic who refused to take part in the study told us that they were not used to smartphone technology at all and that they did not feel competent enough to complete the questionnaire. A similar effect might exist in case of the experts contacted via email.

    Another limitation lies within the group of the patients. Less than one half of these patients could be recruited in the Department of Internal Medicine of the University Hospital of Heidelberg. The remainder was found in self-help groups and forums on the internet and they were, not surprisingly, more in touch with the e-mental health app topic than the patients of the clinic. A bigger survey with a more representative sample of patients, also including outpatient services, would reveal a broader insight into the perspective of the patients. Further, some of the online patients may be medical experts or students as well, which was not asked explicitly.

    We did not ask for the type of mental disorder of the patients, because patients of the psychosomatic departments often suffer from somatic diseases, which are accompanied by mental strains. Asking for a specific diagnosis would have raised the threshold for taking part in this survey. Further studies should focus on a more differentiated picture of the different types of mental disorders.

    Finally, more research is necessary on the role of previous experiences with health care providers and the specific diagnosis of the patients as determinants of attitudes toward e-mental health apps.

    Conclusions

    This study revealed a lack of knowledge and experience of e-mental health apps in experts, students of medicine and psychology, as well as in patients. Even though some concerns were expressed regarding the potential negligence of private data protection, the attitudes and expectations of the target groups were rather positive. This was associated with a younger age of the patients and less professional experience of the medical experts. In consideration of a growing market with professional and semi-professional offers in the app stores, a deeper understanding and awareness of experts and students is an urgent necessity.

    Acknowledgments

    This study was part of the SELFPASS (Self-administered Psycho Therapy SystemS) project funded by the Federal Ministry of Education and Research (FKZ 13GW0157B).

    We acknowledge financial support by Deutsche Forschungsgemeinschaft within the funding program Open Access Publishing, by the Baden-Württemberg Ministry of Science, Research and the Arts and by Ruprecht-Karls-Universität Heidelberg.

    We thank Stefan Bubolz, BSc-Psych, for help with the design of the online questionnaire of the study and Christiana Jung, MEd, for assistance in recruiting participants and data acquisition.

    Conflicts of Interest

    None declared.

    1. Torous J, Roberts LW. Needed Innovation in Digital Health and Smartphone Applications for Mental Health. JAMA Psychiatry. May 01, 2017;74(5):437. [CrossRef]
    2. Mehrotra S, Tripathi R. Recent developments in the use of smartphone interventions for mental health. Current Opinion in Psychiatry. 2018;31(5):379-388. [CrossRef]
    3. Kerst A, Zielasek J, Gaebel W. Smartphone applications for depression: a systematic literature review and a survey of health care professionals' attitudes towards their use in clinical practice. Eur Arch Psychiatry Clin Neurosci. Jan 03, 2019. [CrossRef] [Medline]
    4. Hilty DM, Chan S, Hwang T, Wong A, Bauer AM. Advances in mobile mental health: opportunities and implications for the spectrum of e-mental health services. Mhealth. Aug 2017;3:34. [FREE Full text] [CrossRef] [Medline]
    5. Parlament E, Rat E. April 2017 über Medizinprodukte, zur Änderung der Richtlinie 2001/83/EG, der Verordnung (EG) Nr. 178/2002 und der Verordnung (EG) Nr. 1223/2009 und zur Aufhebung der Richtlinien 90/385/EWG und 93/42/EWG des Rates (MPV). Symbole 8D-Bericht. URL: http://data.europa.eu/eli/reg/2017/745/oj [accessed 2019-09-26]
    6. Stevens WJM, van der Sande R, Beijer LJ, Gerritsen MG, Assendelft WJ. eHealth Apps Replacing or Complementing Health Care Contacts: Scoping Review on Adverse Effects. J Med Internet Res. Mar 01, 2019;21(3):e10736. [FREE Full text] [CrossRef] [Medline]
    7. Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, et al. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol. Med. Jun 24, 2013;44(4):741-752. [CrossRef]
    8. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the Trial: Systematic Review of Real-World Uptake and Engagement With Digital Self-Help Interventions for Depression, Low Mood, or Anxiety. J Med Internet Res. Jun 06, 2018;20(6):e199. [FREE Full text] [CrossRef] [Medline]
    9. Baumel A, Kane JM. Examining Predictors of Real-World User Engagement with Self-Guided eHealth Interventions: Analysis of Mobile Apps and Websites Using a Novel Dataset. J Med Internet Res. Dec 14, 2018;20(12):e11491. [FREE Full text] [CrossRef] [Medline]
    10. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health. Aug 05, 2018;21(3):116-119. [CrossRef] [Medline]
    11. Musiat P, Goldstone P, Tarrier N. Understanding the acceptability of e-mental health--attitudes and expectations towards computerised self-help treatments for mental health problems. BMC Psychiatry. 2014;14:109. [FREE Full text] [CrossRef] [Medline]
    12. Abdelhamid M, Gaia J, Sanders GL. Putting the Focus Back on the Patient: How Privacy Concerns Affect Personal Health Information Sharing Intentions. J Med Internet Res. Sep 13, 2017;19(9):e169. [FREE Full text] [CrossRef] [Medline]
    13. Pung A, Fletcher SL, Gunn JM. Mobile App Use by Primary Care Patients to Manage Their Depressive Symptoms: Qualitative Study. J Med Internet Res. Sep 27, 2018;20(9):e10035. [FREE Full text] [CrossRef] [Medline]
    14. Cristol S. Patient's Perspective on Using Mobile Technology as an Aid to Psychotherapy. JMIR Ment Health. Oct 24, 2018;5(4):e10015. [FREE Full text] [CrossRef] [Medline]
    15. Bonet L, Llácer B, Hernandez-Viadel M, Arce D, Blanquer I, Cañete C, et al. Differences in the Use and Opinions About New eHealth Technologies Among Patients With Psychosis: Structured Questionnaire. JMIR Ment Health. Jul 25, 2018;5(3):e51. [FREE Full text] [CrossRef] [Medline]
    16. Apolinário-Hagen J, Harrer M, Kählke F, Fritsche L, Salewski C, Ebert DD. Public Attitudes Toward Guided Internet-Based Therapies: Web-Based Survey Study. JMIR Ment Health. May 15, 2018;5(2):e10735. [FREE Full text] [CrossRef] [Medline]
    17. Senft N, Everson J. eHealth Engagement as a Response to Negative Healthcare Experiences: Cross-Sectional Survey Analysis. J Med Internet Res. Dec 05, 2018;20(12):e11034. [FREE Full text] [CrossRef] [Medline]
    18. March S, Day J, Ritchie G, Rowe A, Gough J, Hall T, et al. Attitudes Toward e-Mental Health Services in a Community Sample of Adults: Online Survey. J Med Internet Res. Feb 19, 2018;20(2):e59. [FREE Full text] [CrossRef] [Medline]
    19. Wade VA, Eliott JA, Hiller JE. Clinician acceptance is the key factor for sustainable telehealth services. Qual Health Res. May 2014;24(5):682-694. [CrossRef] [Medline]
    20. East ML, Havard BC. Mental Health Mobile Apps: From Infusion to Diffusion in the Mental Health Social System. JMIR Ment Health. Mar 31, 2015;2(1):e10. [FREE Full text] [CrossRef] [Medline]
    21. Obermann K, Müller P, Woernds S. Digitalisierung des Arztberufs. 2016. URL: http://www.stiftung-gesundheit.de/pdf/studien/Aerzte_im_Zukunftsmarkt_Gesundheit_2016.pdf [accessed 2019-09-24]
    22. Dockweiler C. Akzeptanz der Telemedizin. In: Fischer F, Krämer A, editors. eHealth in Deutschland. Berlin. Springer; 2016:257-271.
    23. Berry N, Bucci S, Lobban F. Use of the Internet and Mobile Phones for Self-Management of Severe Mental Health Problems: Qualitative Study of Staff Views. JMIR Ment Health. Nov 01, 2017;4(4):e52. [FREE Full text] [CrossRef] [Medline]
    24. Tonn P, Reuter SC, Kuchler I, Reinke B, Hinkelmann L, Stöckigt S, et al. Development of a Questionnaire to Measure the Attitudes of Laypeople, Physicians, and Psychotherapists Toward Telemedicine in Mental Health. JMIR Ment Health. Oct 03, 2017;4(4):e39. [FREE Full text] [CrossRef] [Medline]
    25. Schröder J, Berger T, Meyer B, Lutz W, Hautzinger M, Späth C, et al. Attitudes Towards Internet Interventions Among Psychotherapists and Individuals with Mild to Moderate Depression Symptoms. Cogn Ther Res. Apr 22, 2017;41(5):745-756. [CrossRef]
    26. Brockes C, Grischott T, Dutkiewicz M, Schmidt-Weitmann S. Evaluation of the Education. Telemed J E Health. Dec 2017;23(11):899-904. [CrossRef] [Medline]
    27. Bozikov J, Hercigonja Szekeres M. Training of Students for Critical Evaluation of Mobile Health Applications. Stud Health Technol Inform. 2017;245:1320. [Medline]
    28. Lungu A, Sun M. Time for a Change: College Students' Preference for Technology-Mediated Versus Face-to-Face Help for Emotional Distress. Telemed J E Health. Dec 2016;22(12):991-1000. [CrossRef] [Medline]
    29. Toscos T, Carpenter M, Drouin M, Roebuck A, Kerrigan C, Mirro M. College Students' Experiences with, and Willingness to Use, Different Types of Telemental Health Resources: Do Gender, Depression/Anxiety, or Stress Levels Matter? Telemed J E Health. Apr 16, 2018;24(12):998-1005. [CrossRef] [Medline]
    30. Leiner D, Leiner S. SoSci Survey (Version 2.6.00). München. SoSci Survey GmbH; 2018. URL: https://www.soscisurvey.de/ [accessed 2019-09-26]
    31. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY. IBM Corp; 2016. URL: https://www.ibm.com/support/pages/how-cite-ibm-spss-statistics-or-earlier-versions-spss [accessed 2019-09-27]
    32. Lee S, Lee DK. What is the proper way to apply the multiple comparison test? Korean J Anesthesiol. Oct 2018;71(5):353-360. [FREE Full text] [CrossRef] [Medline]
    33. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. Jun 27, 2011;2:53-55. [FREE Full text] [CrossRef] [Medline]
    34. Mayring P. Qualitative Inhaltsanalyse. In: Handbuch Qualitative Forschung in der Psychologie. Weinheim. Beltz; 2010:601-613.
    35. Silberschatz G. Improving the yield of psychotherapy research. Psychotherapy Research. Sep 11, 2015;27(1):1-13. [CrossRef] [Medline]
    36. Dolev T, Zilcha-Mano S. The role of the therapeutic relationship in the association between interpersonal behaviors and outcome: Comparison of two competing models. Psychother Res. Jul 08, 2019;29(5):553-564. [CrossRef] [Medline]
    37. Szymańska A, Dobrenko K, Grzesiuk L. Characteristics and experience of the patient in psychotherapy and the psychotherapy’s effectiveness. A structural approach. Psychiatr Pol. 2017;51(4):619-631. [CrossRef]
    38. BPtK-Standpunkt. BPtK Bundes-Psychotherapeuten Kammer; 2017. URL: https://www.bptk.de/wp-content/uploads/2019/01/20170629_bptk_standpunkt_internet.pdf [accessed 2019-09-27]
    39. Schachinger A. Der digitale Patient: Analyse eines neuen Phänomens der partizipativen Vernetzung und Kollaboration von Patienten im Internet. Baden-Baden. Nomos; 2014.

    ‎
    ANOVA: analysis of variance
    eHealth: electronic health
    e-mental: electronic mental
    SELFPASS: Self-administered Psycho Therapy SystemS

    Edited by J Torous; submitted 14.03.19; peer-reviewed by P Athilingam, N Bartle, T Fleming; comments to author 21.04.19; revised version received 12.06.19; accepted 02.09.19; published 25.11.19.

    Copyright

    ©Gwendolyn Mayer, Nadine Gronewold, Simone Alvarez, Bastian Bruns, Thomas Hilbel, Jobst-Hendrik Schultz. Originally published in JMIR Mental Health (http://mental.jmir.org), 25.11.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.