Background: Blended cognitive behavioral therapy (bCBT)—the combination of cognitive behavioral therapy and digital mental health applications—has been increasingly used to treat depression and anxiety disorders. As a resource-efficient treatment approach, bCBT appears promising for addressing the growing need for mental health care services, for example, as an early intervention before the chronification of symptoms. However, further research on the efficacy and feasibility of integrated bCBT interventions is needed.
Objective: This study aimed to evaluate the efficacy of a novel bCBT program comprising short (25 min), weekly face-to-face therapy sessions combined with a smartphone-based digital health app for treating mild to moderate symptoms of depression or anxiety.
Methods: This prospective uncontrolled trial comprised 2 measurement points (before and after treatment) and 2 intervention groups. We recruited university students with mild to moderate symptoms of depression or anxiety. On the basis of the primary symptoms, participants were assigned to either a depression intervention group (n=67 completers) or an anxiety intervention group (n=33 completers). Participants in each group received 6 weekly individual psychotherapy sessions via videoconference and completed modules tailored to their respective symptoms in the smartphone-based digital health app.
Results: The depression group displayed medium to large improvements in the symptoms of depression (Cohen d=−0.70 to −0.90; P<.001). The anxiety group experienced significant improvements in the symptoms of generalized anxiety assessed with the Generalized Anxiety Disorder-7 scale with a large effect size (Cohen d=−0.80; P<.001) but not in symptoms of anxiety assessed with the Beck Anxiety Inventory (Cohen d=−0.35; P=.06). In addition, both groups experienced significant improvements in their perceived self-efficacy (Cohen d=0.50; P<.001 in the depression group and Cohen d=0.71; P<.001 in the anxiety group) and quality of life related to psychological health (Cohen d=0.87; P<.001 in the depression group and Cohen d=0.40; P=.03 in the anxiety group). Work and social adjustment of patients improved significantly in the depression group (Cohen d=−0.49; P<.001) but not in the anxiety group (Cohen d=−0.06; P=.72). Patients’ mental health literacy improved in the anxiety group (Cohen d=0.45; P=.02) but not in the depression group (Cohen d=0.21; P=.10). Patient satisfaction with the bCBT program and ratings of the usability of the digital app were high in both treatment groups.
Conclusions: This study provides preliminary evidence for the feasibility and efficacy of a novel brief bCBT intervention. The intervention effects were generalized across a broad spectrum of patient-reported outcomes. Hence, the newly developed bCBT intervention appears promising for treating mild to moderate depression and anxiety in young adults.
Depression and anxiety disorders are widespread [, ]. Both disorders have severe consequences for those affected, impairing their social and occupational lives and physical health [ - ]. According to the World Health Organization, depression is the most disabling disease in high- and middle-income countries, topping heart and cerebrovascular diseases [ ]. Similarly, anxiety disorders are linked to disability, low educational attainment, and low income [ , ].
University students are particularly at risk for developing depression and anxiety disorders . Several studies suggest that up to 30% of university students display elevated and distressing symptoms of depression or anxiety [ - ]. Some groups of students, such as female students [ , ], first-year students [ , ], and health students [ , ], are at an even greater risk for depression or anxiety. Many university students are in a major transitory phase of their life, maturing from adolescence into adulthood [ ]. Changing life habits, family dynamics, academic challenges, and concerns about occupational life after graduation contribute to the elevated risk of developing symptoms of depression or anxiety in university students [ , ]. Elevated symptoms of depression and anxiety induce high costs for university students. For example, depression and anxiety are linked to poorer academic success [ - ] and related withdrawals from university [ ].
In up to 75% of the cases, both disorders start in mid-20 years of age . Early interventions for depression and anxiety, implemented before the disorders have reached more severe and chronic states, are frequently recommended [ - ]. Thus, interventions targeting mild to moderate symptoms of depression or anxiety in university students constitute a promising approach for easing the burden of mental illness and preventing severe and chronic trajectories of both disorders. As university students are typically tech-savvy [ ], using digital treatment tools may be a promising approach for early interventions in this population.
Commonly recommended treatment options for depression and anxiety include psychotherapy and pharmacotherapy . Regarding psychotherapeutic interventions, cognitive behavioral therapy (CBT) is a well-researched and highly effective approach for treating depression and anxiety [ ]. In recent years, computerized and internet-based CBT (iCBT) have increasingly been used [ , ]. However, many studies suggest that stand-alone digital treatments without personal contact with a health care provider may suffer from several shortcomings. For example, adherence to digitalized stand-alone treatment is typically low [ ], and the rates of premature treatment termination are high [ , ]. iCBT is rarely personalized and often falls short of meeting patients’ individual needs [ ]. Thus, its efficacy and effectiveness are limited when delivered in a stand-alone format [ , ]. The problems of stand-alone digital treatments are often also attributed to the lack of psychotherapist contact providing support and guidance throughout the treatment [ , , ]. Thus, blended CBT (bCBT) programs, combining face-to-face CBT with digital tools, have been increasingly advocated [ , ].
Previous studies support the efficacy and feasibility of bCBT programs for treating depression and anxiety [- ]. Some of these studies found stronger symptom reduction in patients receiving bCBT compared with CBT [ - ]. Other studies have compared bCBT programs with CBT programs that comprised more or longer face-to-face sessions than bCBT programs [ , - ]. Importantly, in these studies, the reduced face-to-face time in bCBT compared with CBT did not lead to the reduced efficacy of bCBT programs compared with more extensive CBT programs [ , - ]. As resources for mental health care are scarce, even in high-income countries [ ], bCBT may be an effective and efficient alternative to face-to-face CBT only. In particular, as an early intervention for mild to moderate symptoms of depression and anxiety in young adults, bCBT may represent a promising treatment option. However, there is little evidence on the efficacy and acceptability of bCBT in this domain. Furthermore, bCBT has seldom been investigated in the vulnerable population of university students.
Further arguments for the use of bCBT can be found in the acceptability of bCBT treatments. Acceptance ratings by psychotherapists and patients have been found to be higher for bCBT than for stand-alone iCBT programs [, ]. For example, a recent study investigating the attitudes of licensed psychotherapists in Germany found that most practitioners prefer bCBT over stand-alone iCBT programs [ ]. During our study period (late 2021 and early 2022), the COVID-19 pandemic caused increased rates of mental disorders and psychological distress in Germany [ , ], further straining the mental health care system and leading to a growing need for mental health services worldwide [ , ]. As the prevalence rates of depression and anxiety have increased over the last few years, also outside of pandemic phases [ , ], the need for effective and resource-efficient programs, such as bCBT, is becoming increasingly pressing.
This study aimed to evaluate the efficacy and feasibility of a novel smartphone-based digital health app, elona therapy, for use in bCBT. This app was designed to overcome the limitations of iCBT and the existing bCBT programs. Previous bCBT interventions typically use face-to-face sessions and a smartphone-based digital health app as distinct therapeutic elements [, ]. In contrast, elona therapy offers an integrated synthesis of digital and face-to-face elements where therapists can adapt the individually relevant therapeutic content that patients can access on their smartphones. Specifically, digital content incorporated into the treatment is customized by the therapist according to the individual symptoms, personal needs of the patient, and the current psychotherapeutic focus. University students with either mild to moderate symptoms of depression or anxiety received a 6-week bCBT intervention, including face-to-face individual CBT sessions (comprising six 25-min sessions) via videoconference combined with the depression or anxiety module of elona therapy. Despite the briefness of the face-to-face sessions, we expected that using the bCBT program with elona therapy is associated with significant improvements in mental health outcomes.
This study evaluated the efficacy and feasibility of 2 novel bCBT programs, one targeting mild to moderate symptoms of depression and one targeting mild to moderate symptoms of anxiety. We conducted a 6-week trial comprising 2 assessment points (pretreatment [T0] and posttreatment [T1]) and 2 intervention groups (depression and anxiety). During the 6-week intervention, participants received weekly individual CBT sessions (25 min each) over the web via videoconference and used the depression or anxiety module of elona therapy on their smartphones. At both assessment points, participants completed a broad set of self-report measures capturing symptoms of depression, symptoms of anxiety, daily functioning level, competencies, and satisfaction with the treatment. All participants provided their informed consent for study participation and data processing for research purposes before the study.
Ethics approval for this study was obtained from the Ethical Board of the University of Mannheim (EK Mannheim 27/2021).
This study comprised 2 university student samples, one with mild to moderate symptoms of depression and one with mild to moderate symptoms of anxiety. A total of 107 participants were initially recruited in the study. Depending on their symptomatology, 71 participants were assigned to the depression group and 36 participants were assigned to the anxiety intervention group. Between the pre- and postintervention points, 3 participants dropped out from the depression sample and 3 participants dropped out from the anxiety sample. The data of 1 participant from the depression sample were lost because of a technical problem. In the depression sample, 67 participants (58/67, 87% female; 9/67, 13% male; age: mean 23.61, SD 3.72; range 18-36 years) completed the study by taking part in both assessment points. In the anxiety sample, 33 participants (30/33, 91% female; 3/33, 9% male; age: mean 24.7, SD 3.5; range 19-38 years) completed the study by taking part in both assessment points.displays the study flowchart. shows the demographic characteristics of study completers.
Post hoc power analyses (Cronbach α=.05) showed that the study had a high power to detect a large (99.99% for Cohen d=0.80), a high power to detect a medium (98.09% for Cohen d=0.50), and a small power to detect a small (36.46% for Cohen d=0.20) within-group effect in the depression group. Similarly, post hoc power analyses (α=.05) showed that the study had a high power to detect a large (99.37% for Cohen d=0.8), a moderate power to detect a medium (79.54% for Cohen d=0.5), and a small power to detect a small (20% for Cohen d=0.2) within-group effect in the anxiety group.
|Characteristics||Depression sample (n=67)||Anxiety sample (n=33)|
|Age (years), mean (SD)||23.61 (3.72)||24.70 (3.50)|
|Female, n (%)||58 (87)||30 (91)|
|University major, n (%)|
|Sports||1 (1)||0 (0)|
|Psychology||11 (16)||4 (12)|
|Medicine and related fields (eg, dentistry and pharmacy)||13 (19)||5 (15)|
|Social sciences and humanities||22 (33)||15 (45)|
|Science and engineering||12 (18)||3 (9)|
|Management||3 (4)||4 (12)|
|Recent graduates (nonuniversity students)||5 (7)||2 (6)|
University students with mild to moderate symptoms of depression or anxiety were recruited from October 2021 to February 2022 through flyers distributed on campuses and student email groups of several universities in North Rhine-Westphalia, Germany. The study comprised a 2-step inclusion process. In the first step (prescreening), participants completed a web-based survey consisting of the Patient Health Questionnaire-9 (PHQ-9) [, ], the Generalized Anxiety Disorder-7 (GAD-7) scale [ , ], demographic questions, and further questions examining specific exclusion criteria. Participants who met all the inclusion criteria and none of the exclusion criteria addressed in the web-based assessment were invited to the second step (screening) where they were interviewed by a clinical psychologist (Master of Science).
The inclusion criteria used in the study were as follows: (1) mild to moderate symptoms of depression or anxiety (ie, PHQ-9 or GAD-7 scores between 5 and 15 [, ]), (2) aged between 18 and 65 years, (3) adequate proficiency in German for engaging in psychotherapy and using the elona therapy app, and (4) having a smartphone with iOS or Android operating systems and an internet connection. The exclusion criteria used in the study were as follows: (1) acute suicidality (a score of 2 or 3 on the item 9 of the PHQ-9), (2) current or prior International Classification of Diseases, 10th Revision (ICD-10) bipolar disorder, (3) current or prior ICD-10 borderline personality disorder, (4) current or prior ICD-10 psychotic disorder, (5) symptoms of posttraumatic stress disorder or dissociative experiences, (6) current or prior self-harm (addressed only in the screening with a therapist), (7) current or prior ICD-10 alcohol or substance dependency (except nicotine), and (8) psychotherapeutic or psychopharmacological treatment in the previous year.
The second step, the screening process, comprised a video-based interview with a clinical psychologist currently in advanced postgraduate psychotherapeutic training (interview duration: approximately 25 min). The 2-step inclusion process was used to cross-validate self-report responses with clinical assessments and to assess sensitive inclusion criteria (eg, suicidality) thoroughly and empathically. In the screening interview, inclusion and exclusion criteria were revisited by psychologists, and participants with nonelevated or severe symptoms of depression or anxiety using ICD-10 criteria and those presenting suicidality or other exclusion criteria were excluded. Those with severe symptom levels were advised to consult appropriate health care services and were provided with contact information of emergency services. Participants meeting all the inclusion criteria and none of the exclusion criteria after the second screening step were recruited to the study and assigned to either the depression or the anxiety intervention group, depending on their respective symptomatology. For persons who displayed both elevated symptoms of depression and anxiety, assignment to the depression or anxiety group was jointly decided by the participant and interviewing clinical psychologist (shared decision-making). All interviewing clinical psychologists received interview training from an experienced licensed psychotherapist before the study and supervision throughout the recruitment phase.
bCBT Intervention Program
The participants received bCBT for depression or anxiety over 6 weeks. The weekly 25-minute video-based individual CBT sessions comprised psychoeducation, internet-based therapeutic exercises (eg, creation of a behavioral activation plan and creation of a personal toolbox), and joint reflections on those tasks.summarizes the standardized content of the face-to-face sessions in this study. The treatment manual included 18 digital tasks for the depression group and 16 digital tasks for the anxiety group. The face-to-face sessions were developed based on well-established cognitive behavioral treatment manuals for depression and anxiety [ - ]. CBT sessions were provided by clinical psychologists who currently underwent postgraduate psychotherapeutic training, were regularly seeing outpatients, and received supervision by licensed psychotherapists.
abCBT: blended cognitive behavioral therapy.
To improve their progress between the weekly sessions, patients engaged in additional between-session homework using elona therapy on their smartphones. elona therapy provides patients with relevant interventions, helpful activities, exercises, and psychoeducational resources related to their respective symptoms. The app content can be individualized according to the patient’s needs, for example, based on their diagnosis, therapy progress, and joint therapeutic decisions. elona therapy has been developed based on current treatment recommendations and evidence-based CBT interventions [- ]. The app is designed to strengthen the active participation of patients in outpatient psychotherapy and integrates therapeutic content into their daily lives. elona therapy provides a 2-sided interface for patients (smartphone app) and psychotherapists (web application). Through the web application, psychotherapists can track individual progress, adapt resources and interventions displayed in the patient’s app, and assign tasks based on the content of the face-to-face CBT sessions. Psychotherapists can choose from over 400 available content pieces. Through the smartphone app, patients can access psychoeducational content (text, video, or audio materials), complete interactive homework tasks, record their mood in a diary format, write notes or questions relevant to their therapy sessions, and plan or schedule new activities. In addition, participants can create their personal toolkit of preferred content and tasks within the app. Although the intervention program for this study was manualized to some degree, the participating therapists were encouraged to further individualize the digital content in the elona therapy app appropriate for the patient’s needs. Patients were encouraged to use the app approximately 25 minutes per week, mirroring a session duration. shows the available content in the elona therapy app for patients with symptoms of depression or anxiety. The participating clinical psychologists received training on the use and functions of elona therapy by a senior licensed psychotherapist before the study.
|Module and chapter||Content|
|About depression||Psychoeducational content and disorder-specific knowledge about psychotherapy, the symptoms of depression, individual symptoms, comorbid conditions, and factors that promote the development and maintenance of depression.|
|My behavior||Relationship between activities and depression, behavior analysis, and activity building and planning. Values-based work.|
|My thoughts||Relationship between thoughts and depression. Methods of cognitive restructuring and introduction of the concept of detached mindfulness.|
|My emotions||Relationship between emotions and depression. Psychoeducational content to perceive emotions in a more differentiated way, to understand them as indications of needs and support in dealing with emotions.|
|My relationships||Depression in a social context. Acquisition of social skills and competencies.|
|Relapse prevention for depression||Methods of general relapse prevention such as recognizing individual early warning signs and building a toolbox for difficult situations.|
|About anxiety||Psychoeducational content and disorder-specific knowledge about psychotherapy, the symptoms of anxiety disorders, individual symptoms, comorbidities, and the vicious cycle of anxiety.|
|Emergence and maintenance of anxiety||Psychoeducational content on the development and maintenance conditions of anxiety disorders and individual application of the acquired knowledge.|
|Exposure methods||Information on exposure procedures. More specifically, the mechanisms of exposure are taught and the patient is given the opportunity to prepare for exposure interventions.|
|Thoughts and anxiety||Support in identifying and dealing with anxiety-related thoughts. Practicing methods to examine and change anxiety-related thoughts.|
|Emotions and anxiety||Psychoeducational content on differentiating emotions and interpreting them as indicators of psychological needs. Developing strategies for dealing with emotions.|
|ACTa for anxiety||Learning and practicing methods of ACT to deal with symptoms of anxiety|
|Relapse prevention for anxiety||Methods of general relapse prevention. Motivating patients to continuously practice what they have learned.|
|About relaxation||Psychoeducational content on relaxation techniques, focus on physical processes|
|PMRc||Mechanisms of action and goals of PMR according to Jacobson and related exercises|
|Imagination||Psychoeducation on imagination exercises and related exercises|
|Mindfulness and meditation||Psychoeducation on awareness and meditation techniques and related exercises|
aACT: Acceptance and Commitment Therapy.
bThe Relaxation module was available to both intervention groups.
cPMR: progressive muscle relaxation.
This study comprised a broad set of outcome measures, covering symptoms of depression and anxiety, perceived functioning, well-being, satisfaction with the treatment program, and competencies [- ]. We assessed the symptoms of depression and anxiety in both intervention groups as these symptoms regularly co-occur [ , ].
Symptoms of Depression
The symptoms of depression were the primary outcome for the depression group and a secondary outcome for the anxiety group. We assessed the symptoms of depression with 2 well-established self-report instruments: the PHQ-9 [, ] and the Beck Depression Inventory-II (BDI-II) [ , ]. The PHQ-9 assesses the frequency of symptoms of depression over the past 2 weeks with 9 items (eg, feeling tired or having little energy) using a 4-point Likert scale ranging from 0 (“not at all”) to 3 (“nearly every day”). The BDI-II assesses the severity of symptoms of depression over the past 2 weeks with 21 items (eg, loss of interest) using a 4-point Likert scale ranging from 0 (eg, “I have not lost interest in other people or activities”) to 3 (eg, “It’s hard to get interested in anything”). Many studies have demonstrated the reliability and validity of the German PHQ-9 and BDI-II instruments [ - ]. The Cronbach α value in this study was .85 at T0 and .88 at T1 for BDI-II and .70 at T0 and .78 at T1 for PHQ-9.
Symptoms of Anxiety
The symptoms of anxiety were the primary outcome for the anxiety group and a secondary outcome for the depression group. We assessed the symptoms of anxiety with 2 well-established self-report instruments: the GAD-7 [, ] and the Beck Anxiety Inventory (BAI) [ , ]. The GAD-7 contains 7 items assessing the severity of symptoms of generalized anxiety (eg, trouble relaxing) over the past 2 weeks using a 4-point Likert scale ranging from 0 (“not at all”) to 3 (“nearly every day”). The BAI comprises 21 items assessing the severity of common symptoms of anxiety (eg, numbness or tingling) over the past month on a 4-point Likert scale ranging from 0 (“not at all”) to 3 (“severely—it bothered me a lot”). The German GAD-7 and BAI instruments display satisfactory reliability and validity [ , ]. The Cronbach α value in this study was .79 at T0 and .84 at T1 for GAD-7 and .90 at T0 and .91 at T1 for BAI.
Quality of Life
Quality of life was assessed with the WHOQOL-BREF scale [, ]. This standardized 26-item self-report instrument comprises 5 subscales: overall quality of life and general health (2 items), physical health (7 items), psychological health (6 items), quality of life related to social relationships (3 items), and quality of life related to one’s environment (8 items). Participants rated each item on a 5-point Likert scale (eg, 1=“not at all” to 5=“completely”). Previous studies have demonstrated the reliability and validity of the German WHOQOL-BREF [ ]. The Cronbach α value in this study was .72 at T0 and .72 at T1 for the overall quality of life and general health domain, .70 at T0 and .77 at T1 for the physical health domain, .78 at T0 and .81 at T1 for the psychological health domain, .47 at T0 and .61 at T1 for the social relationships domain, and .72 at T0 and .73 at T1 for the environment domain.
Functioning in Work and Social Life
Patients’ adjustment to work and social life was assessed with the Work and Social Adjustment Scale (WSAS) [, ]. This instrument captures impairments in relevant life domains beyond the symptom level [ , ]. The WSAS contains 5 items (eg, “Because of my depression/anxiety, my ability to work is impaired”) rated on a 9-point Likert scale ranging from 0 (“not at all impaired”) to 8 (“very severely impaired to the point I can’t work”). Higher scores on this scale indicate higher maladjustment. The German WSAS has been previously shown to be a reliable and valid measurement instrument [ ]. The Cronbach α value in this study was .77 at T0 and .82 at T1.
We assessed the participants’ perceived self-efficacy with the General Perceived Self-Efficacy Scale (GSE) . This scale measures beliefs about one’s capacity to cope with new, difficult, or stressful situations. The GSE contains 10 items (eg, “It is easy for me to stick to my aims and accomplish my goals”) on a 4-point Likert scale ranging from 1 strongly disagree to 4 strongly agree. Previous work demonstrated the reliability and validity of the GSE [ , ]. The Cronbach α value in this study was .88 at T0 and .90 at T1.
Mental Health Literacy
We assessed mental health literacy (ie, patients’ knowledge about mental disorders and their treatments) with a 20-item version of the Mental Health Literacy Scale (MHLS) [, ]. The 20-item MHLS includes items assessing general mental health literacy (eg, “I am confident that I know where to seek information about mental illness”) using a 5-point Likert scale ranging from 1 (“strongly disagree”) to 5 (“strongly agree”). The MHLS items used in our study were the items numbered 16 to 35 in the original 35-item MHLS. The items not used in this study focused on identifying different specific mental disorder diagnoses, which is beyond the scope of this study. The original MHLS demonstrated adequate internal consistency and validity [ ]. The Cronbach α value for the MHLS in this study was .83 at T0 and .82 at T1.
Satisfaction With Treatment
We assessed patient satisfaction with their treatment at T1 using the German version of the Client Satisfaction Questionnaire (CSQ-8) [, ]. The CSQ-8 captures patient satisfaction using 8 items (eg, “Have the services you received helped you to deal more effectively with your problems?”) with a 4-point Likert scale ranging from 1 (eg, “No, they seemed to make things worse”) to 4 (eg, “Yes, they helped a great deal”). Higher CSQ-8 scores (range 8-32) indicate higher patient satisfaction. The German CSQ-8 has been extensively validated for assessing patient satisfaction in mental health treatment [ , ]. The Cronbach α value in this study was .93.
Usability of the elona therapy App
The usability of the elona therapy app was assessed with the German version of the System Usability Scale (SUS) [, ]. The SUS is a self-report instrument comprising 10 items with a 5-point Likert scale ranging from 1 (“strongly disagree”) to 5 (“strongly agree”). The SUS raw scores are summed and then multiplied by 2.5 to facilitate interpretation. According to the SUS interpretation guidelines, a transformed score >68 indicates an above-average usability [ , ]. The German version of the SUS has shown satisfactory reliability and validity in various studies [ ]. The Cronbach α value in this study was .89.
Serious Adverse Events
Participating therapists were instructed to report in case their patients experienced any serious adverse events. Serious adverse events include negative events that are harmful or threatening beyond transient distress and are life-threatening or require hospitalization. Self-harm, suicidal ideation, or attempts are potentially serious adverse events.
We evaluated the changes from before to after the treatment using 2-tailed paired sample t tests. Statistical analyses were conducted based on completers, including only participants who have completed at least 1 questionnaire after treatment. In addition, we evaluated our measurement instruments’ reliability (Cronbach α), and we computed the pre-post effect sizes (Cohen d) for each outcome. For the depression group, we treated the depression measures (PHQ-9 and BDI-II) as primary outcomes and all other measures as secondary outcomes. For the anxiety group, we treated the anxiety measures (BAI and GAD-7) as primary outcomes and all other measures as secondary outcomes.
Engagement With the Intervention
On average, participants in the depression sample completed 21.6 tasks and spent a total of 88.12 minutes using the elona therapy app during the intervention period. Participants in the anxiety group completed on average 24.1 tasks and spent 106.9 minutes using the elona therapy app during the intervention period. The mean number of completed face-to-face sessions in the depression group was 5.64 (SD 0.64; range 2-6), and the mean number of completed face-to-face sessions in the anxiety group was 5.55 (SD 0.62; range 4-6). Patient engagement with elona therapy was high; only 1 patient in the depression group was a minimal user, which is defined as using elona therapy for <15 minutes for the entire study period, with only 7 minutes of total use time.
displays the means and SDs for the pre- and postintervention points, the results of the 2-tailed paired samples t tests, and the corresponding effect sizes (Cohen d) for the depression group. Regarding symptoms of depression (primary outcome), we observed significant improvements in the PHQ-9 score (t63=−7.23, P<.001; Cohen d=−0.90) and the BDI-II score (t65=−5.66, P<.001; Cohen d=−0.70). Regarding the secondary outcomes, the symptoms of anxiety assessed with the GAD-7 (t60=−6.71, P<.001; Cohen d=−0.86) and BAI (t35=−4.55, P<.001; Cohen d=−0.76) and maladjustment assessed with the WSAS (t58=−3.74, P<.001; Cohen d=−0.49) significantly decreased. In addition, perceived self-efficacy assessed with the GSE (t60=3.93, P<.001; Cohen d=0.50) and quality of life assessed with the WHOQOL-BREF (t61=6.48, P<.001; Cohen d=0.82 for overall quality of life and general health; t61=6.31, P<.001; Cohen d=0.80 for physical health; t61=6.85, P<.001; Cohen d=0.87 for psychological health; t61=4.27, P<.001; Cohen d=0.54 for quality of life related to social relationships; and t63=4.79, P<.001; Cohen d=0.61 for quality of life related to one’s environment) significantly improved. One secondary outcome, mental health literacy in the MHLS, did not significantly improve during the intervention period (t59=1.66, P=.10; Cohen d=0.21). At the postintervention point, participants rated the usability of the smartphone-based digital health app for treating depression as high (mean usability score of 74.15 out of 100 on the SUS, corresponding to above-average usability [ ]). Participants also rated their satisfaction with the bCBT intervention program as high (mean 26.14 out of 32 on the CSQ-8). There were no serious adverse events reported.
|Outcome||Preintervention point, mean (SD)||Postintervention point, mean (SD)||t test (df)a||P value||Effect sizeb, Cohen d||Sample size, nc|
|Symptoms of depression (PHQ-9d)||10.82 (3.72)||6.69 (3.47)||−7.23 (63)||<.001||−0.90||64|
|Symptoms of depression (BDI-IIe)||19.40 (7.46)||13.33 (7.54)||−5.66 (65)||<.001||−0.70||66|
|Symptoms of anxiety (GAD-7f)||8.12 (3.83)||5.23 (2.88)||−6.71 (60)||<.001||−0.86||61|
|Symptoms of anxiety (BAIg)||12.83 (8.25)||8.61 (5.99)||−4.55 (35)||<.001||−0.76||36|
|Perceived self-efficacy (GSEh)||25.03 (5.37)||27.07 (5.34)||3.93 (60)||<.001||0.5||61|
|Work and social adjustment (WSASi)||18.09 (7.68)||14.42 (7.71)||−3.74 (58)||<.001||−0.49||59|
|Mental health literacy (MHLSj)||84.91 (8.10)||85.80 (8.3)||1.66 (59)||.10||0.21||60|
|Quality of life and general health (WHOQOL-BREFk)||6.63 (1.66)||7.65 (1.51)||6.48 (61)||<.001||0.82||62|
|Physical health (WHOQOL-BREF)||25.10 (3.83)||27.98 (3.62)||6.31 (61)||<.001||0.8||62|
|Psychological health (WHOQOL-BREF)||17.51 (3.58)||20.26 (3.78)||6.85 (61)||<.001||0.87||62|
|Quality of life—social (WHOQOL-BREF)||9.85 (2.2)||10.97 (2.27)||4.27 (61)||<.001||0.54||62|
|Quality of life—environmental (WHOQOL-BREF)||31.34 (4.48)||33.24 (3.85)||4.79 (61)||<.001||0.61||62|
aThe t test value associated with the 2-tailed paired sample t test assessing pre-post differences.
bEffect size=Cohen d associated with the respective pre-post difference.
cSome participants failed to complete specific postintervention measures, leading to slight deviations in the sample size between the comparisons.
dPHQ-9: Patient Health Questionnaire-9 [, ].
eBDI-II: Beck Depression Inventory-II [, ].
fGAD-7: Generalized Anxiety Disorder-7 [, ].
gBAI: Beck Anxiety Inventory [, ].
hGSE: General Self-Efficacy Scale .
iWSAS: Work and Social Adjustment Scale [, ].
jMHLS: Mental Health Literacy Scale [, ].
kWHOQOL−BREF: World Health Organization Quality of Life Scale (brief version) [, ].
displays the means and SDs for the pre- and postintervention points, the results of the 2-tailed paired samples t tests, and the corresponding effect sizes (Cohen d) for the anxiety group. Regarding symptoms of anxiety (primary outcome), we observed significant improvements in the GAD-7 score (t30=−4.45, P<.001; Cohen d=−0.80) but not in the BAI score (t31=−1.96, P=.06; Cohen d=−0.35). Regarding the secondary outcomes, the symptoms of depression assessed with the PHQ-9 (t32=−4.58, P<.001; Cohen d=−0.8) and the BDI-II (t21=−3.53, P=.002; Cohen d=−0.75) significantly decreased. In addition, perceived self-efficacy (t30=3.97, P<.001; Cohen d=0.71), mental health literacy (t28=2.07, P=.048; Cohen d=0.38), overall quality of life and general health (t30=2.53, P=.02; Cohen d=0.45), and psychological health (t30=2.21, P=.03; Cohen d=0.40) assessed with the WHOQOL-BREF significantly improved. Maladjustment assessed with the WSAS (t31=−0.36, P=.72; Cohen d=−0.06), physical health (t30=1.75, P=.09; Cohen d=0.31), quality of life related to social relationships (t30=0.18, P=.86; Cohen d=0.03), and quality of life related to one’s environment (t30=1.38, P=.18; Cohen d=0.25) assessed with the WHOQOL-BREF did not significantly improve in the intervention period. At the postintervention point, participants rated the usability of the smartphone-based digital health app for treating symptoms of anxiety as high (mean usability score of 81.68 [out of 100] on the SUS, corresponding to above-average usability [ ]). Participants also rated their satisfaction with the bCBT intervention program as high (mean 28.21 out of 32 on the CSQ-8). There were no serious adverse events reported.
|Outcome||Preintervention point, mean (SD)||Postintervention point, mean (SD)||t test (df)a||P value||Effect sizeb, Cohen d||Sample size, nc|
|Symptoms of anxiety (GAD-7d)||11.33 (3.93)||8.23 (4.91)||−4.45 (30)||<.001||−0.80||31|
|Symptoms of anxiety (BAIe)||21 (10.48)||17.69 (10.47)||−1.96 (31)||.06||−0.35||32|
|Symptoms of depression (PHQ-9f)||10.7 (4.28)||7.3 (4.33)||−4.58 (32)||<.001||−0.80||33|
|Symptoms of depression (BDI-IIg)||18.06 (9.92)||12.95 (9.22)||−3.53 (21)||.002||−0.75||22|
|Perceived self-efficacy (GSEh)||24.94 (6.43)||27.19 (5.95)||3.97 (30)||<.001||0.71||31|
|Work and social adjustment (WSASi)||16.61 (8.21)||16.03 (8.15)||−0.36 (31)||.72||−0.06||32|
|Mental health literacy (MHLSj)||86.45 (8.17)||87.34 (7.18)||2.07 (28)||.048||.38||29|
|Quality of life and general health (WHOQOL-BREFk)||6.7 (1.74)||7.16 (1.66)||2.53 (30)||.02||0.45||31|
|Physical health (WHOQOL-BREF)||24.79 (4.31)||25.9 (4.77)||1.75 (30)||.09||0.31||31|
|Psychological health (WHOQOL-BREF)||18.39 (4.59)||19.13 (4.64)||2.21 (30)||.03||0.4||31|
|Quality of life—social (WHOQOL-BREF)||10.33 (2.29)||10.32 (2.23)||0.18 (30)||.86||0.03||31|
|Quality of life—environmental (WHOQOL-BREF)||30.09 (4.33)||30.68 (3.78)||1.38 (30)||.18||0.25||31|
aThe t test value associated with the 2-tailed paired sample t test assessing pre-post differences.
bEffect size=Cohen d associated with the respective pre-post difference.
cSome participants failed to complete some postintervention measures, leading to slight deviations in the sample size between the comparisons.
dGAD-7: Generalized Anxiety Disorder-7 [, ].
eBAI: Beck Anxiety Inventory [, ].
fPHQ-9: Patient Health Questionnaire-9 [, ].
gBDI-II: Beck Depression Inventory-II [, ].
hGSE: General Perceived Self-Efficacy Scale .
iWSAS: Work and Social Adjustment Scale [, ].
jMHLS: Mental Health Literacy Scale [, ].
kWHOQOL-BREF: World Health Organization Quality of Life Scale (brief version) [, ].
This study investigated the efficacy and feasibility of a 6-week bCBT program for mild to moderate depression or anxiety in university students. Overall, participants showed improvements in their primary symptoms, which generalized to other symptom domains, quality of life, and self-efficacy. In addition, participants reported high satisfaction with the intervention program and the elona therapy app. The intervention evaluated in this study produced within-group effect sizes that were considerably larger than those observed in waitlist control groups over similar periods [, ] suggesting that the brief bCBT program may be efficacious.
In addition, a previous meta-analysis on the effectiveness of standard evidence-based psychological treatments in practices revealed similarly large pre-post treatment effect sizes for depression (Cohen d=0.87) and anxiety (Cohen d=0.88), as observed in our study . Importantly, the studies included in this meta-analysis comprised, on average, more face-to-face time with a therapist [ ]. Similarly, the effect sizes obtained in this study were similar to the effect sizes observed in previous, more extensive bCBT programs for patients with depression or anxiety disorders, ranging from 12 to 15 weeks [ , ]. Various factors such as measurement instruments and sample characteristics may influence effect sizes, so treatment effects should only be cautiously compared between studies. However, the current data indicated that compared with previous approaches, the evaluated bCBT program is at least an equally effective and resource-efficient intervention for reducing the symptoms of depression and anxiety in university students.
Regarding within-group differences between intervention outcomes, we observed larger effects for the symptoms of depression or anxiety than for secondary outcomes such as work and social adjustment or self-efficacy. This pattern of findings is in line with a previous meta-analysis  and may be explained by the explicit focus of our intervention on symptom reduction. In the anxiety group, significant symptom reduction was observed in the GAD-7 but not in the BAI (P=.06). The GAD-7 primarily assesses symptoms of generalized (cognitive) anxiety, whereas the BAI comprises various items capturing physical symptoms of anxiety (eg, heat sensations, accelerated heart rate, and shakiness). Our intervention’s focus on the cognitive aspects of anxiety and psychoeducation rather than in vivo exposure therapy may explain this finding.
Although work and social adjustment improved significantly in the depression group, this was not the case for the anxiety group. This may be because of worse work and social adjustment before the intervention in the individuals with the symptoms of depression compared with those with symptoms of anxiety, as it has been found earlier in the literature [, ]. Similarly, all domains of quality of life significantly improved in the depression group and only some improved in the anxiety group. Follow-up studies with similar sample sizes across symptom groups (ie, depression or anxiety) are required to understand whether the observed differences may be because of the lower sample size of the anxiety group compared with the depression group in this study. Moreover, the intervention program we offered in the anxiety group appeared significantly effective in improving mental health literacy, whereas an improvement in this domain was not significant in the depression group. It remains unclear whether this is because of different patient characteristics in groups or different content provided for those groups, either within sessions or in the elona therapy app ( and ).
Regarding the feasibility of the evaluated program, participants in both study groups rated their satisfaction with the intervention as high. Satisfaction ratings were similar to those provided in the study by Romijn et al , who found that participants rated their satisfaction with the bCBT program to be 25.61 (SD 4.21) on average or higher than that reported by Kooistra et al [ ], who found the mean satisfaction to be 22.71 (SD 4.82). Notably, these previous bCBT programs are more extensive than those we provided, with the one in the study by Romijn et al [ ] lasting 15 weeks and the one in the study by Kooistra et al [ ] lasting 10 weeks. Participants also indicated high usability of the elona therapy modules, corresponding to acceptable usability according to guidelines [ ]. High feasibility was also evident from the remarkably low attrition rates. Approximately 94% of the participants in the depression group and 92% of the participants in the anxiety group completed the 6-week bCBT program as intended. Moreover, the participants’ good engagement with the app during the study period further supports the feasibility of the elona therapy app. The blended treatment program introduced in this study appears to be a safe treatment option, as no adverse events were reported in this study.
Limitations and Future Research Directions
As this study was planned as an uncontrolled trial, several limitations point toward future research directions. First, this study comprised only 2 measurement points (before and after the assessment). Future research with more measurement points is needed to establish the ideal duration of the evaluated bCBT intervention (eg, to establish whether incremental symptom reduction occurs after the sixth week). Further work should also test the stability of the obtained improvements through follow-up assessments. Second, as a feasibility study focusing on within-group improvements, this study did not include control groups. Thus, future research using appropriate active (eg, standard CBT treatment) or passive (eg, waitlist control) groups is needed. In addition, dismantling studies could be useful for evaluating the effects of different elements of the elona therapy modules. Third, we only performed a completer analysis in this trial, which may have overestimated the intervention’s benefits. However, given the small number of dropouts in both groups, the differences in completer and intent-to-treat analyses were small. Fourth, this study focused on a convenience sample, including a vulnerable population (ie, university students) with mild to moderate symptoms. The findings may have limited generalizability to populations with different characteristics and to other student samples in different cultures or different regions of Germany. Fifth, we only used patient-reported outcomes in this study. The use of only self-report may provide a limited depiction of patients’ situations owing to patients’ possible inaccurate evaluation of their situation or response biases such as social desirability bias. Further work can complement some of the outcome measures (eg, symptoms of depression and anxiety) by a clinical rating scale such as therapist rating questionnaires.
This trial used only a quantitative study design, which may inform researchers only limitedly about the investigated program. To explore patients’ in-depth experiences with this novel bCBT program, an additional qualitative study was needed. As a follow-up to this trial, we invited all participants who had completed the last face-to-face session with their therapist to participate in a follow-up interview conducted by an interviewer who was not involved in the therapeutic process. The results of this qualitative investigation were published in a separate article .
Further work is needed to assess the efficacy and feasibility of the evaluated bCBT program in other groups and individuals with more severe symptomatology.
In summary, this feasibility study provided preliminary evidence based on an uncontrolled pre-post trial design for the efficacy of a novel bCBT program for treating mild to moderate depression and anxiety. Promisingly, substantial symptom improvements were generalized across various outcomes and were accompanied by high patient satisfaction and adherence. Future research is needed to evaluate the efficacy of the evaluated program in different populations and over longer time intervals.
The authors would like thank all therapists and patients who participated in the study. This project was funded by the Robert-Enke Foundation (Robert-Enke-Stiftung), a nonprofit organization that supports projects, initiatives, and institutions that educate people about depressive disorders and facilitates their research or treatment. The funding source did not play any role in the study design, analysis, or interpretation of the results. The authors would like to thank the Heinrich Heine University Düsseldorf for funding the publication fees (University Library, Open-Access-Funds of the Heinrich Heine University Düsseldorf).
Conflicts of Interest
EA declares that she is employed by Elona Health, the manufacturer of the elona therapy app used in this study. JS and AP are consultants to Elona Health. MS is a shareholder of Elona Health.
- Bandelow B, Michaelis S. Epidemiology of anxiety disorders in the 21st century. Dialogues Clin Neurosci 2022 Apr 01;17(3):327-335 [https://doi.org/10.31887/DCNS.2015.17.3/bbandelow] [CrossRef]
- Gelenberg AJ. The prevalence and impact of depression. J Clin Psychiatry 2010 Mar 15;71(3):e06 [https://doi.org/10.4088/JCP.8001tx17c] [CrossRef] [Medline]
- Haller H, Cramer H, Lauche R, Gass F, Dobos GJ. The prevalence and burden of subthreshold generalized anxiety disorder: a systematic review. BMC Psychiatry 2014 May 01;14(1):128 [https://doi.org/10.1186/1471-244X-14-128] [CrossRef]
- Hansson L. Quality of life in depression and anxiety. Int Rev Psychiatry 2009 Jul 11;14(3):185-189 [https://doi.org/10.1080/09540260220144966] [CrossRef]
- Saarijärvi S, Salminen JK, Toikka T, Raitasalo R. Health-related quality of life among patients with major depression. Nord J Psychiatry 2002;56(4):261-264 [https://doi.org/10.1080/08039480260242741] [CrossRef] [Medline]
- The global burden of disease : 2004 update. World Health Organization. 2008. URL: https://apps.who.int/iris/handle/10665/43942 [accessed 2022-04-25]
- Hendriks SM, Spijker J, Licht CM, Beekman AT, Hardeveld F, de Graaf R, et al. Disability in anxiety disorders. J Affect Disord 2014 Oct;166:227-233 [https://doi.org/10.1016/j.jad.2014.05.006] [CrossRef] [Medline]
- Kessler RC, Ruscio AM, Shear K, Wittchen HU. Epidemiology of anxiety disorders. In: Stein M, Steckler T, editors. Behavioral Neurobiology of Anxiety and Its Treatment. Berlin, Germany: Springer; 2009.
- Mofatteh M. Risk factors associated with stress, anxiety, and depression among university undergraduate students. AIMS Public Health 2021;8(1):36-65 [https://europepmc.org/abstract/MED/33575406] [CrossRef] [Medline]
- de Paula W, Breguez GS, Machado EL, Meireles AL. Prevalence of anxiety, depression, and suicidal ideation symptoms among university students: a systematic review. Braz J Health Rev 2020;3(4):8739-8756 [https://doi.org/10.34119/bjhrv3n4-119] [CrossRef]
- Ozen NS, Ercan I, Irgil E, Sigirli D. Anxiety prevalence and affecting factors among university students. Asia Pac J Public Health 2010 Jan;22(1):127-133 [https://doi.org/10.1177/1010539509352803] [CrossRef] [Medline]
- Shamsuddin K, Fadzil F, Ismail WS, Shah SA, Omar K, Muhammad NA, et al. Correlates of depression, anxiety and stress among Malaysian university students. Asian J Psychiatr 2013 Aug;6(4):318-323 [https://doi.org/10.1016/j.ajp.2013.01.014] [CrossRef] [Medline]
- Bahrami F, Yousefi N. Females are more anxious than males: a metacognitive perspective. Iran J Psychiatry Behav Sci 2011;5(2):83-90 [https://europepmc.org/abstract/MED/24644451] [Medline]
- Boggiano AK, Barrett M. Gender differences in depression in college students. Sex Roles 1991 Dec;25(11-12):595-605 [https://doi.org/10.1007/BF00289566] [CrossRef]
- Bayram N, Bilgel N. The prevalence and socio-demographic correlations of depression, anxiety and stress among a group of university students. Soc Psychiatry Psychiatr Epidemiol 2008 Aug;43(8):667-672 [https://doi.org/10.1007/s00127-008-0345-x] [CrossRef] [Medline]
- Mahmoud JS, Staten R, Hall LA, Lennie TA. The relationship among young adult college students' depression, anxiety, stress, demographics, life satisfaction, and coping styles. Issues Ment Health Nurs 2012 Mar;33(3):149-156 [https://doi.org/10.3109/01612840.2011.632708] [CrossRef] [Medline]
- Moir F, Yielder J, Sanson J, Chen Y. Depression in medical students: current insights. Adv Med Educ Pract 2018 May;Volume 9:323-333 [https://doi.org/10.2147/AMEP.S137384] [CrossRef]
- Sarokhani D, Delpisheh A, Veisani Y, Sarokhani MT, Manesh RE, Sayehmiri K. Prevalence of depression among university students: a systematic review and meta-analysis study. Depress Res Treat 2013;2013:373857 [https://doi.org/10.1155/2013/373857] [CrossRef] [Medline]
- Beiter R, Nash R, McCrady M, Rhoades D, Linscomb M, Clarahan M, et al. The prevalence and correlates of depression, anxiety, and stress in a sample of college students. J Affect Disord 2015 Mar 01;173:90-96 [https://doi.org/10.1016/j.jad.2014.10.054] [CrossRef] [Medline]
- Depression and college students. National Institute of Mental Health. 2012. URL: https://www.pugetsound.edu/sites/default/files/file/1784_depression-and-college-students_0.pdf [accessed 2022-04-25]
- Andrews B, Wilding JM. The relation of depression and anxiety to life-stress and achievement in students. Br J Psychol 2004 Nov;95(Pt 4):509-521 [https://doi.org/10.1348/0007126042369802] [CrossRef] [Medline]
- Eisenberg D, Gollust SE, Golberstein E, Hefner JL. Prevalence and correlates of depression, anxiety, and suicidality among university students. Am J Orthopsychiatry 2007 Oct;77(4):534-542 [https://doi.org/10.1037/0002-94126.96.36.1994] [CrossRef] [Medline]
- Hysenbegasi A, Hass SL, Rowland CR. The impact of depression on the academic productivity of university students. J Ment Health Policy Econ 2005 Sep;8(3):145-151 [Medline]
- Meilman P, Manley C, Gaylor MS, Turco JH. Medical withdrawals from college for mental health reasons and their relation to academic performance. J Am Coll Health 1992 Mar;40(5):217-223 [https://doi.org/10.1080/07448481.1992.9936283] [CrossRef] [Medline]
- Kessler RC, Amminger GP, Aguilar-Gaxiola S, Alonso J, Lee S, Üstün TB. Age of onset of mental disorders: a review of recent literature. Curr Opinion Psychiatry 2007 Jul;20(4):359-364 [https://doi.org/10.1097/YCO.0b013e32816ebc8c] [CrossRef]
- Cuijpers P, van Straten A, Smit F, Mihalopoulos C, Beekman A. Preventing the onset of depressive disorders: a meta-analytic review of psychological interventions. Am J Psychiatry 2008 Oct;165(10):1272-1280 [https://core.ac.uk/reader/15456304?utm_source=linkout] [CrossRef] [Medline]
- Feldner MT, Zvolensky MJ, Schmidt NB. Prevention of anxiety psychopathology: a critical review of the empirical literature. Clin Psychol Sci Pract 2004;11(4):405-424 [https://doi.org/10.1093/clipsy.bph098] [CrossRef]
- van't Veer-Tazelaar PJ, van Marwijk HW, van Oppen P, van der Horst HE, Smit F, Cuijpers P, et al. Prevention of late-life anxiety and depression has sustained effects over 24 months: a pragmatic randomized trial. Am J Geriatr Psychiatry 2011 Mar;19(3):230-239 [https://doi.org/10.1097/JGP.0b013e3181faee4d] [CrossRef] [Medline]
- Prensky M. Digital natives, digital immigrants part 1. On Horizon 2001 Sep;9(5):1-6 [https://doi.org/10.1108/10748120110424816] [CrossRef]
- Cuijpers P, Sijbrandij M, Koole SL, Andersson G, Beekman AT, Reynolds CF. The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: a meta-analysis of direct comparisons. World Psychiatry 2013 Jun;12(2):137-148 [https://europepmc.org/abstract/MED/23737423] [CrossRef] [Medline]
- David D, Cristea I, Hofmann SG. Why cognitive behavioral therapy is the current gold standard of psychotherapy. Front Psychiatry 2018;9:4 [https://europepmc.org/abstract/MED/29434552] [CrossRef] [Medline]
- Andersson G, Titov N. Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry 2014 Feb;13(1):4-11 [https://europepmc.org/abstract/MED/24497236] [CrossRef] [Medline]
- Cuijpers P, Kleiboer A, Karyotaki E, Riper H. Internet and mobile interventions for depression: opportunities and challenges. Depress Anxiety 2017 Jul;34(7):596-602 [https://doi.org/10.1002/da.22641] [CrossRef] [Medline]
- Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther 2012 Aug;50(7-8):463-468 [https://doi.org/10.1016/j.brat.2012.04.001] [CrossRef] [Medline]
- Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342 [https://doi.org/10.1016/j.cpr.2012.02.004] [CrossRef] [Medline]
- Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2008 Sep 24;39(5):705-712 [https://doi.org/10.1017/S0033291708004224] [CrossRef]
- Andrews G, Williams AD. Internet psychotherapy and the future of personalized treatment. Depress Anxiety 2014 Nov;31(11):912-915 [https://doi.org/10.1002/da.22302] [CrossRef] [Medline]
- Weisel K, Fuhrmann LM, Berking M, Baumeister H, Cuijpers P, Ebert DD. Standalone smartphone apps for mental health-a systematic review and meta-analysis. NPJ Digit Med 2019;2:118 [https://doi.org/10.1038/s41746-019-0188-8] [CrossRef] [Medline]
- Wright J, Mishkind M, Eells TD, Chan SR. Computer-assisted cognitive-behavior therapy and mobile apps for depression and anxiety. Curr Psychiatry Rep 2019 Jun 27;21(7):62 [https://doi.org/10.1007/s11920-019-1031-2] [CrossRef] [Medline]
- Newman MG, Szkodny LE, Llera SJ, Przeworski A. A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy? Clin Psychol Rev 2011 Feb;31(1):89-103 [https://doi.org/10.1016/j.cpr.2010.09.008] [CrossRef] [Medline]
- Erbe D, Eichert HC, Riper H, Ebert DD. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: systematic review. J Med Internet Res 2017 Sep 15;19(9):e306 [https://www.jmir.org/2017/9/e306/] [CrossRef] [Medline]
- Newby J, Mason E, Kladnistki N, Murphy M, Millard M, Haskelberg H, et al. Integrating internet CBT into clinical practice: a practical guide for clinicians. Clin Psychol 2021 Mar 15;25(2):164-178 [https://doi.org/10.1080/13284207.2020.1843968] [CrossRef]
- Bruinsma A, Kampman M, Exterkate CC, Hendriks GJ. [An exploratory study of 'blended' cognitive behavioural therapy (CBT) for patients with a panic disorder: results and patients' experiences]. Tijdschr Psychiatr 2016;58(5):361-370 [https://www.tijdschriftvoorpsychiatrie.nl/en/tijdschrift/artikel/TVPart_10859] [Medline]
- Kooistra LC, Wiersma JE, Ruwaard J, Neijenhuijs K, Lokkerbol J, van Oppen P, et al. Cost and effectiveness of blended versus standard cognitive behavioral therapy for outpatients with depression in routine specialized mental health care: pilot randomized controlled trial. J Med Internet Res 2019 Oct 29;21(10):e14261 [https://www.jmir.org/2019/10/e14261/] [CrossRef] [Medline]
- Lungu A, Jun JJ, Azarmanesh O, Leykin Y, Chen CE. Blended care-cognitive behavioral therapy for depression and anxiety in real-world settings: pragmatic retrospective study. J Med Internet Res 2020 Jul 06;22(7):e18723 [https://www.jmir.org/2020/7/e18723/] [CrossRef] [Medline]
- Berger T, Krieger T, Sude K, Meyer B, Maercker A. Evaluating an e-mental health program ("deprexis") as adjunctive treatment tool in psychotherapy for depression: results of a pragmatic randomized controlled trial. J Affect Disord 2018 Feb;227:455-462 [https://doi.org/10.1016/j.jad.2017.11.021] [CrossRef] [Medline]
- Klein JP, Berger T, Schröder J, Späth C, Meyer B, Caspar F, et al. Effects of a psychological internet intervention in the treatment of mild to moderate depressive symptoms: results of the EVIDENT study, a randomized controlled trial. Psychother Psychosom 2016;85(4):218-228 [https://europepmc.org/abstract/MED/27230863] [CrossRef] [Medline]
- Schuster R, Laireiter AR, Berger T, Moritz S, Meyer B, Hohagen F, et al. Immediate and long-term effectiveness of adding an internet intervention for depression to routine outpatient psychotherapy: subgroup analysis of the EVIDENT trial. J Affect Disord 2020 Sep 01;274:643-651 [https://doi.org/10.1016/j.jad.2020.05.122] [CrossRef] [Medline]
- Ly KH, Topooco N, Cederlund H, Wallin A, Bergström J, Molander O, et al. Smartphone-supported versus full behavioural activation for depression: a randomised controlled trial. PLoS One 2015;10(5):e0126559 [https://dx.plos.org/10.1371/journal.pone.0126559] [CrossRef] [Medline]
- Thase ME, Wright JH, Eells TD, Barrett MS, Wisniewski SR, Balasubramani GK, et al. Improving the efficiency of psychotherapy for depression: computer-assisted versus standard CBT. Am J Psychiatry 2018 Mar 01;175(3):242-250 [https://europepmc.org/abstract/MED/28969439] [CrossRef] [Medline]
- Thase ME, McCrone P, Barrett MS, Eells TD, Wisniewski SR, Balasubramani GK, et al. Improving cost-effectiveness and access to cognitive behavior therapy for depression: providing remote-ready, computer-assisted psychotherapy in times of crisis and beyond. Psychother Psychosom 2020 May 12;89(5):307-313 [https://doi.org/10.1159/000508143] [CrossRef] [Medline]
- Saxena S, Thornicroft G, Knapp M, Whiteford H. Resources for mental health: scarcity, inequity, and inefficiency. Lancet 2007 Sep;370(9590):878-889 [https://doi.org/10.1016/S0140-6736(07)61239-2] [CrossRef]
- Phillips EA, Himmler S, Schreyögg J. Preferences of psychotherapists for blended care in Germany: a discrete choice experiment. BMC Psychiatry 2022 Feb 12;22(1):112 [https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-022-03765-x] [CrossRef] [Medline]
- Topooco N, Riper H, Araya R, Berking M, Brunn M, Chevreul K, E-COMPARED consortium. Attitudes towards digital treatment for depression: a European stakeholder survey. Internet Interv 2017 Jun;8:1-9 [https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(16)30044-6] [CrossRef] [Medline]
- Kohls E, Baldofski S, Moeller R, Klemm SL, Rummel-Kluge C. Mental health, social and emotional well-being, and perceived burdens of university students during COVID-19 pandemic lockdown in Germany. Front Psychiatry 2021;12:643957 [https://europepmc.org/abstract/MED/33889102] [CrossRef] [Medline]
- Mata J, Wenz A, Rettig T, Reifenscheid M, Möhring K, Krieger U, et al. Health behaviors and mental health during the COVID-19 pandemic: a longitudinal population-based survey in Germany. Soc Sci Med 2021 Oct;287:114333 [https://europepmc.org/abstract/MED/34455337] [CrossRef] [Medline]
- Byrne A, Barber R, Lim CH. Impact of the COVID-19 pandemic – a mental health service perspective. Prog Neurol Psychiatry 2021;25(2):27-33b [https://doi.org/10.1002/pnp.708] [CrossRef]
- The impact of COVID-19 on mental, neurological and substance use services. World Health Organization. 2020 Oct 5. URL: https://www.who.int/publications/i/item/978924012455 [accessed 2022-04-25]
- Duffy ME, Twenge JM, Joiner TE. Trends in mood and anxiety symptoms and suicide-related outcomes among U.S. undergraduates, 2007-2018: evidence from two national surveys. J Adolesc Health 2019 Nov;65(5):590-598 [https://doi.org/10.1016/j.jadohealth.2019.04.033] [CrossRef] [Medline]
- Etzelmueller A, Radkovsky A, Hannig W, Berking M, Ebert DD. Patient's experience with blended video- and internet based cognitive behavioural therapy service in routine care. Internet Interv 2018 Jun;12:165-175 [https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(17)30089-1] [CrossRef] [Medline]
- Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [https://europepmc.org/abstract/MED/11556941] [CrossRef] [Medline]
- Löwe B, Spitzer RL, Zipfel S, Herzog W. Gesundheitsfragebogen für Patienten (PHQ-D). Manual und Testunterlagen. (2. Ed.). Universitätsklinikum Heidelberg. New York, NY: Pfizer; 2002. URL: https://www.klinikum.uni-heidelberg.de/fileadmin/Psychosomatische_Klinik/download/PHQ_Manual1.pdf [accessed 2022-03-01]
- Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, et al. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care 2008 Mar;46(3):266-274 [https://doi.org/10.1097/MLR.0b013e318160d093] [CrossRef] [Medline]
- Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097 [CrossRef] [Medline]
- Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatric Annals 2002 Sep;32(9):509-515 [https://doi.org/10.3928/0048-5713-20020901-06] [CrossRef]
- Faßbinder E, Klein JP, Sipos V, Schweiger U. Therapie-Tools Depression: Mit E-Book inside und Arbeitsmaterial. Weinheim, Germany: Beltz; Oct 5, 2015.
- Hautzinger M. Kognitive Verhaltenstherapie bei Depressionen: Mit Online-Materialien. Weinheim, Germany: Beltz; Apr 14, 2021.
- Neudeck P, Mühlig S. Therapie-Tools Verhaltenstherapie: Therapieplanung, Probatorik, Verhaltensanalyse. Mit E-Book inside und Arbeitsmaterial (Beltz Therapie-Tools). Weinheim, Germany: Beltz; Mar 11, 2020.
- Schaub A, Roth E, Goldmann U. Kognitiv-psychoedukative Therapie zur Bewältigung von Depressionen: Ein Therapiemanual. Göttingen, Germany: Hogrefe Verlag; Aug 1, 2006.
- Beck AT. Cognitive models of depression. J Cognit Psychother 1987;1(1):5-37 [https://psycnet.apa.org/record/1989-11913-001]
- Ankuta GY, Abeles N. Client satisfaction, clinical significance, and meaningful change in psychotherapy. Prof Psychol Res Pr 1993 Feb;24(1):70-74 [https://doi.org/10.1037/0735-7028.24.1.70] [CrossRef]
- Jerusalem M. Persönliche Ressourcen. Vulnerabilität und Streßerleben. Göttingen, Germany: Hogrefe Verlag; Jan 1, 1990.
- Kolovos S, Kleiboer A, Cuijpers P. Effect of psychotherapy for depression on quality of life: meta-analysis. Br J Psychiatry 2016 Dec;209(6):460-468 [https://doi.org/10.1192/bjp.bp.115.175059] [CrossRef] [Medline]
- Rytsälä HJ, Melartin TK, Leskelä US, Lestelä-Mielonen PS, Sokero TP, Isometsä ET. Determinants of functional disability and social adjustment in major depressive disorder: a prospective study. J Nerv Ment Dis 2006 Aug;194(8):570-576 [https://doi.org/10.1097/01.nmd.0000230394.21345.c4] [CrossRef] [Medline]
- Zahra D, Qureshi A, Henley W, Taylor R, Quinn C, Pooler J, et al. The work and social adjustment scale: reliability, sensitivity and value. Int J Psychiatry Clin Pract 2014 Jun;18(2):131-138 [https://doi.org/10.3109/13651501.2014.894072] [CrossRef] [Medline]
- Aina Y, Susman JL. Understanding comorbidity with depression and anxiety disorders. J Am Osteopath Assoc 2006 May;106(5 Suppl 2):S9-14 [Medline]
- Hirschfeld RM. The comorbidity of major depression and anxiety disorders: recognition and management in primary care. Prim Care Companion J Clin Psychiatry 2001 Dec;3(6):244-254 [https://europepmc.org/abstract/MED/15014592] [CrossRef] [Medline]
- Beck AT, Steer RA, Brown G. Manual for the Beck Depression Inventory-II. Psychological Corporation. 1996. URL: https://www.nctsn.org/measures/beck-depression-inventory-second-edition [accessed 2022-03-01]
- Kühner C, Bürger C, Keller F, Hautzinger M. [Reliability and validity of the Revised Beck Depression Inventory (BDI-II). Results from German samples]. Nervenarzt 2007 Jun;78(6):651-656 [https://doi.org/10.1007/s00115-006-2098-7] [CrossRef] [Medline]
- Dolle K, Schulte-Körne G, O'Leary AM, von Hofacker N, Izat Y, Allgaier AK. The Beck depression inventory-II in adolescent mental health patients: cut-off scores for detecting depression and rating severity. Psychiatry Res 2012 Dec 30;200(2-3):843-848 [https://doi.org/10.1016/j.psychres.2012.05.011] [CrossRef] [Medline]
- Hinz A, Mehnert A, Kocalevent RD, Brähler E, Forkmann T, Singer S, et al. Assessment of depression severity with the PHQ-9 in cancer patients and in the general population. BMC Psychiatry 2016 Feb 02;16:22 [https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-016-0728-6] [CrossRef] [Medline]
- Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 1988;56(6):893-897 [https://doi.org/10.1037/0022-006X.56.6.893] [CrossRef]
- Hoyer J, Margraf J. Fragebögen und Ratingskalen zur globalen Messung von Angst, Ängstlichkeit und Angststörungen. In: Angstdiagnostik. Berlin, Germany: Springer; 2003.
- Geissner E, Huetteroth A. [beck anxiety inventory German version - a reliable, valid, patientfriendly instrument for measuring clinical anxiety]. Psychother Psychosom Med Psychol 2018 Mar;68(3-4):118-125 [https://doi.org/10.1055/s-0043-122941] [CrossRef] [Medline]
- Angermeyer MC, Kilian R, Matschinger H. WHOQOL - 100 und WHOQOL - BREF: Handbuch für die deutschsprachige Version der WHO Instrumente zur Erfassung von Lebensqualität. Göttingen, Germany: Hogrefe; 2000.
- World Health Organization. Division of Mental Health. WHOQOL-BREF : introduction, administration, scoring and generic version of the assessment : field trial version, December 1996. World Health Organization. 1996. URL: https://apps.who.int/iris/handle/10665/63529 [accessed 2022-03-01]
- Skevington SM, Lotfy M, O'Connell KA. The World Health Organization's WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A Report from the WHOQOL Group. Qual Life Res 2004 Mar;13(2):299-310 [https://doi.org/10.1023/B:QURE.0000018486.91360.00] [CrossRef]
- Heissel A, Bollmann J, Kangas M, Abula K, Rapp M, Sanchez A. Validation of the German version of the work and social adjustment scale in a sample of depressed patients. Research Square. Preprint posted online January 29, 2021 2023 [https://doi.org/10.21203/rs.3.rs-151437/v1] [CrossRef]
- Mundt JC, Marks IM, Shear MK, Greist JH. The Work and Social Adjustment Scale: a simple measure of impairment in functioning. Br J Psychiatry 2002 May;180:461-464 [https://doi.org/10.1192/bjp.180.5.461] [CrossRef] [Medline]
- Schwarzer R, Jerusalem M. Generalized self-efficacy scale. In: Weinman J, Wright S, Johnston M, editors. Measures in Health Psychology: A User’s Portfolio. Causal and Control Beliefs. Windsor, UK: NFER-Nelson; 1995.
- Schwarzer R, Jerusalem M. Skalen zur Erfassung von Lehrer- und Schülermerkmalen. Dokumentation der psychometrischen Verfahren im Rahmen der Wissenschaftlichen Begleitung des Modellversuchs Selbstwirksame Schulen. Berlin, Germany: Freie Universität Berlin; Jan 1999.
- Schwarzer R, Mueller J, Greenglass E. Assessment of perceived general self-efficacy on the internet: data collection in cyberspace. Anxiety Stress Coping 1999 Jan;12(2):145-161 [https://doi.org/10.1080/10615809908248327] [CrossRef]
- Lalk C. Online-based self-help intervention for GAD. OSF Home. 2021 Dec 15. URL: https://osf.io/tj4re/ [accessed 2023-08-03]
- O'Connor M, Casey L. The Mental Health Literacy Scale (MHLS): a new scale-based measure of mental health literacy. Psychiatry Res 2015 Sep 30;229(1-2):511-516 [https://doi.org/10.1016/j.psychres.2015.05.064] [CrossRef] [Medline]
- Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plan 1982;5(3):233-237 [https://doi.org/10.1016/0149-7189(82)90074-X] [CrossRef] [Medline]
- Schmidt J, Lamprecht F, Wittmann WW. [Satisfaction with inpatient management. Development of a questionnaire and initial validity studies]. Psychother Psychosom Med Psychol 1989 Jul;39(7):248-255 [Medline]
- Boß L, Lehr D, Reis D, Vis C, Riper H, Berking M, et al. Reliability and validity of assessing user satisfaction with web-based health interventions. J Med Internet Res 2016 Aug 31;18(8):e234 [https://www.jmir.org/2016/8/e234/] [CrossRef] [Medline]
- Brooke J. SUS: a 'quick and dirty' usability scale. In: Usability Evaluation In Industry. Boca Raton, FL: CRC Press; 1996.
- Gao M, Kortum P, Oswald FL. Multi-language toolkit for the system usability scale. Int J Human Comput Interact 2020 Aug 19;36(20):1883-1901 [https://doi.org/10.1080/10447318.2020.1801173] [CrossRef]
- Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. In J Human Comput Interact 2008 Jul 30;24(6):574-594 [https://doi.org/10.1080/10447310802205776] [CrossRef]
- Brooke J. SUS: a retrospective. J Usab Stud 2013 Jan;8(2):40
- Posternak MA, Miller I. Untreated short-term course of major depression: a meta-analysis of outcomes from studies using wait-list control groups. J Affect Disord 2001 Oct;66(2-3):139-146 [https://doi.org/10.1016/S0165-0327(00)00304-9] [CrossRef] [Medline]
- Steinert C, Stadter K, Stark R, Leichsenring F. The effects of waiting for treatment: a meta-analysis of waitlist control groups in randomized controlled trials for social anxiety disorder. Clin Psychol Psychother 2017 May;24(3):649-660 [https://doi.org/10.1002/cpp.2032] [CrossRef] [Medline]
- Wakefield S, Kellett S, Simmonds-Buckley M, Stockton D, Bradbury A, Delgadillo J. Improving Access to Psychological Therapies (IAPT) in the United Kingdom: a systematic review and meta-analysis of 10-years of practice-based evidence. Br J Clin Psychol 2021 Mar 23;60(1):1-37 [https://eprints.whiterose.ac.uk/162614/] [CrossRef] [Medline]
- Romijn G, Batelaan N, Koning J, van Balkom A, de Leeuw A, Benning F, et al. Acceptability, effectiveness and cost-effectiveness of blended cognitive-behavioural therapy (bCBT) versus face-to-face CBT (ftfCBT) for anxiety disorders in specialised mental health care: a 15-week randomised controlled trial with 1-year follow-up. PLoS One 2021 Nov 12;16(11):e0259493 [https://dx.plos.org/10.1371/journal.pone.0259493] [CrossRef] [Medline]
- Hendriks SM, Spijker J, Licht CM, Hardeveld F, de Graaf R, Batelaan NM, et al. Long-term work disability and absenteeism in anxiety and depressive disorders. J Affect Disord 2015 Jun 01;178:121-130 [https://doi.org/10.1016/j.jad.2015.03.004] [CrossRef] [Medline]
- Kennedy BL, Lin Y, Schwab JJ. Work, social, and family disabilities of subjects with anxiety and depression. South Med J 2002;95(12):1424-1427 [https://doi.org/10.1097/00007611-200295120-00014] [CrossRef]
- Kooistra LC, Ruwaard J, Wiersma JE, van Oppen P, van der Vaart R, van Gemert-Pijnen JE, et al. Development and initial evaluation of blended cognitive behavioural treatment for major depression in routine specialized mental health care. Internet Interv 2016 May;4:61-71 [https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(16)00003-8] [CrossRef] [Medline]
- Braun P, Atik E, Guthardt L, Apolinário-Hagen J, Schückes M. Barriers to and facilitators of a blended cognitive behavioral therapy program for depression and anxiety based on experiences of university students: qualitative interview study. JMIR Form Res 2023 Apr 12;7:e45970 [https://formative.jmir.org/2023//e45970/] [CrossRef] [Medline]
|BAI: Beck Anxiety Inventory|
|bCBT: blended cognitive behavioral therapy|
|BDI-II: Beck Depression Inventory-II|
|CBT: cognitive behavioral therapy|
|CSQ-8: Client Satisfaction Questionnaire|
|GAD-7: Generalized Anxiety Disorder-7|
|GSE: General Self-Efficacy Scale|
|iCBT: internet-based cognitive behavioral therapy|
|ICD-10: International Classification of Diseases, 10th Revision|
|MHLS: Mental Health Literacy Scale|
|PHQ-9: Patient Health Questionnaire-9|
|SUS: System Usability Scale|
|WSAS: Work and Social Adjustment Scale|
Edited by J Torous; submitted 01.12.22; peer-reviewed by I Choi, M Subotic-Kerry, A AL-Asadi; comments to author 26.04.23; revised version received 25.05.23; accepted 25.06.23; published 25.08.23Copyright
©Ece Atik, Johannes Stricker, Magnus Schückes, Andre Pittig. Originally published in JMIR Mental Health (https://mental.jmir.org), 25.08.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.