Background: Mental ill-health presents a major public health problem. A potential part solution that is receiving increasing attention is computer-delivered psychological therapy, particularly during the COVID-19 pandemic as health care systems moved to remote service delivery. However, computerized cognitive behavioral therapy (cCBT) requires active engagement by service users, and low adherence may minimize treatment effectiveness. Therefore, it is important to investigate the acceptability of cCBT to understand implementation issues and maximize potential benefits.
Objective: This study aimed to produce a critical appraisal of published reviews about the acceptability of cCBT for adults.
Methods: An umbrella review informed by the Joanna Briggs Institute (JBI) methodology identified systematic reviews about the acceptability of cCBT for common adult mental disorders. Acceptability was operationalized in terms of uptake of, dropping out from, or completion of cCBT treatment; factors that facilitated or impeded adherence; and reports about user, carer, and health care professional experience and satisfaction with cCBT. Databases were searched using search terms informed by relevant published research. Review selection and quality appraisal were guided by the JBI methodology and the AMSTAR tool and undertaken independently by 2 reviewers.
Results: The systematic searches of databases identified 234 titles, and 9 reviews (covering 151 unique studies) met the criteria. Most studies were comprised of service users with depression, anxiety, or specifically, panic disorder or phobia. Operationalization of acceptability varied across reviews, thereby making it difficult to synthesize results. There was a similar number of guided and unguided cCBT programs; 34% of guided and 36% of unguided users dropped out; and guidance included email, telephone, face-to-face, and discussion forum support. Guided cCBT was completed in full by 8%-74% of the participants, while 94% completed one module and 67%-84% completed some modules. Unguided cCBT was completed in full by 16%-66% of participants, while 95% completed one module and 54%-93% completed some modules. Guided cCBT appeared to be associated with adherence (sustained via telephone). A preference for face-to-face CBT compared to cCBT (particularly for users who reported feeling isolated), internet or computerized delivery problems, negative perceptions about cCBT, low motivation, too busy or not having enough time, and personal circumstances were stated as reasons for dropping out. Yet, some users favored the anonymous nature of cCBT, and the capacity to undertake cCBT in one’s own time was deemed beneficial but also led to avoidance of cCBT. There was inconclusive evidence for an association between sociodemographic variables, mental health status, and cCBT adherence or dropping out. Users tended to be satisfied with cCBT, reported improvements in mental health, and recommended cCBT. Overall, the results indicated that service users’ preferences were important considerations regarding the use of cCBT.
Conclusions: The review indicated that “one size did not fit all” regarding the acceptability of cCBT and that individual tailoring of cCBT is required in order to increase population reach, uptake, and adherence and therefore, deliver treatment benefits and improve mental health.
Globally, mental health problems and mental disorders are a major public health concern . Collectively, the results of multiple studies and systematic reviews over the last few decades appear to point to the overall effectiveness of cognitive behavioral therapy (CBT) [ , ]. CBT aims to help individuals overcome, change, and challenge aberrant thought processes and behaviors [ ]. The advent of the internet and surge in the use of computer technology led to the delivery of CBT via computer- or web-based platforms, with demonstrable effectiveness to treat and manage mental health conditions and symptoms [ - ]. Computerized delivery of CBT (cCBT) is a generic term that encompasses web-based or internet-delivered CBT (iCBT). It can be delivered in the community, at home, or in a health care setting and can be self-directed or self-guided by the user or is guided via telephone or email by a health care professional or practitioner [ ]. There is evidence to suggest that cCBT may be as effective as face-to-face delivered CBT [ , ], and, for example, the UK National Institute for Health and Clinical Excellence (NICE) recommends the use of cCBT for specific conditions such as mild depression [ ]. Compared to face-to-face CBT, cCBT may remove or minimize barriers associated with uptake and be more accessible (eg, for rural dwellers and individuals who have mobility issues) and affordable; thereby, it may reduce waiting times and be more cost-effective [ , ]. In addition, the absence of in-person contact with cCBT delivery may reduce stigma associated with mental health service use for some individuals even though perceptions towards help-seeking for mental health in many countries have taken a positive step over recent years [ ].
The onset of a global pandemic of the SARS-CoV-2 virus occurred during the conduct of this umbrella review. Government-directed measures designed to control the transmission of the virus such as social distancing, quarantine, and self-isolation were implemented widely. These measures necessitated a move to remote-delivered therapies .
Mental health interventions such as CBT require active engagement from participants in order to affect therapeutic change. Adherence and the degree to which an individual engages with, and completes, a web-based intervention is a potential limitation. Poor adherence limits exposure to a web or computerized program, and an insufficient “dose” may impact treatment effectiveness and reduce the likelihood of an improved outcome for people with mental health problems, particularly if they drop out of a program. It is important to enhance understanding about acceptability and adherence in order to gain insights about the implementation of cCBT and to increase its effectiveness . Improving understanding about acceptability is even more important now given that mental disorders and mental health problems appear to be increasing due to the direct and indirect effects of COVID-19 and the possibility that remotely delivered psychological therapies may become the norm in the context of strategies designed to minimize virus transmission [ ]. The results of our initial scoping activity indicated that there has been an exponential increase in the number of systematic reviews and there are international collaborations that specialize in systematic reviews and maintain databases of systematic reviews. Therefore, we decided to synthesize the evidence from existing systematic reviews regarding the acceptability of cCBT by undertaking a review of reviews or an “umbrella” review [ ]. We used cCBT to encompass also iCBT, as some programs are delivered without the internet (eg, CD-ROMs), and our focus was on mental health generally to encompass experiences with both psychological symptoms and clinically diagnosed disorders. Acceptability [ ] was defined broadly as encompassing factors that facilitate or impede uptake, adherence, and completion or that contribute to attrition (including reported reasons for dropping out) and satisfaction. This review also captured service users’ and therapists’ or clinicians’ views of cCBT [ ].
The search strategy for the umbrella review  was informed by the use of the following key concepts from published reviews: “systematic reviews,” “cCBT,” “iCBT,” “barriers,” “facilitators,” and “common mental disorders and symptoms” [ - ]. The Boolean operators “AND” and “OR” were used to combine search terms between and within concepts, respectively. The PubMed, EMBASE, MEDLINE, PsycINFO, and CINAHL databases were searched from inception until December 18, 2019 (see ). Titles and abstracts were imported into an Excel spreadsheet, and duplicates were removed by CT. MD and CT independently assessed each title and abstract against the eligibility criteria and independently applied the Joanne Briggs Institute critical appraisal checklist for systematic reviews [ ], supplemented with one item from the AMSTAR (A Measurement Tool to Assess Systematic Reviews) checklist [ ]. Any disagreements regarding inclusion of articles were resolved through discussion. Results of the quality appraisal checklist are reported in . Review papers were included in the umbrella review if they met the criteria that are detailed in the following sections.
MEDLINE search strategy (adapted for other databases).
#1 ((internet or web or online) adj3 (cognitive or behavio*)).ti,ab,kf. OR (iCBT or i-CBT or ePsych* or e-Psych or cCBT or c-CBT).ti,ab,kf
#2 computer communication networks/ or internet/ or blogging/ or social media/ OR cell phones/ or smartphone/ or text messaging/ or videoconferencing/ or webcasts as topic/ or wireless technology/ OR Telemedicine/ OR (eLearning or blended learning).ti,kf. OR (videoconferenc* or video conferenc*).ti,kf OR (synchronous or asynchronous or (electronic adj2 deliver*)).ti,kf. OR android.ti,ab,kf. OR (app or apps or blog*).ti,ab,kf. OR (cell phone or cellphone or chat room or computer* or cyber* or digital or technology based or DVD).ti,ab,kf. OR CD-ROM.ti,ab,kf. OR (eHealth or electronic health or email*).ti,ab,kf. OR (ePortal or eTherap* or forum* or gaming or information technolog* or instant messag* or messaging or internet* or ipad or iphone or ipod or podcast or smart phone or smartphone or social network* site* or social networking or mHealth or mobile or multimedia or online* or personal digital assistant or PDA or SMS or social medi* or software or telecomm* or telehealth* or telemed* or telemonitor* or telepsych* or teletherap* or text messag* or texting or virtual* or web* or WWW).ti,ab,kf,hw.
#3 (behavio* or cognitive).ti. OR ((cognitive or behavio*) adj2 (activat* or component? or defusion or modif* or restructur* or technique* or intervention or treatment* or therap* or train*)).mp. OR (psychotherap* or psychological therap* or cognitive behavio*).mp. OR ((acceptance* or commitment*) adj3 therap*).mp. OR (rational emotive or RET or problem sol* or PST or problem focus* or solution focus* or trauma focus* or psychoeducat* or psychodrama or mindfulness* or third wave or self control).mp. OR (self* adj3 (control or efficacy)).mp. OR (stress manage* or exposure or reality therap*).mp. OR (anxiety adj3 (management or therap* or train*)).mp. OR (relaxation or guided imagery or present cent* or person cent* or person* construct* or therapeutic process* or schema? or schemata).mp. OR (thought* adj3 suppress*).mp. OR rumination.mp.
#4 #2 AND #3
#5 #1 OR #4
#6 “Systematic Review”/ OR systematic review.ti,ab,kf.
#7 #5 AND #6
#8 program evaluation.mp. OR program evaluation/ OR process evaluation.mp. OR “process assessment (health care)”/ OR (process evaluation* OR qualitative component* OR qualitative aspect* OR qualitative approach* OR systematic evaluation* OR participant observation OR simulation OR implementation audit).mp. OR (audit OR feedback).tw. OR qualitative research/ OR qualitative.mp. OR qualitative research.mp.
#9 #7 AND #8
#10 depression/ OR depression.mp. OR anxiety/ OR anxiety.mp. OR Obsessive-Compulsive Disorder/ OR obsessive compulsive disorder.mp. OR Mental Disorders/ OR common mental disorders.mp. OR Occupational Stress/ OR Stress Disorders, Post-Traumatic/ OR stress.mp. OR Stress, Psychological/ OR mental health.ti OR mental illness.ti. OR psychiatric.ti OR mood disorder.tw OR neurotic disorder.sh
#11 #9 AND #10
Population and Focus of the Review
The target population was people with common mental disorders or experiencing an increase in psychological symptoms (eg, depression, anxiety, and distress) . Systematic reviews of studies of mixed populations and interventions were included if it was possible to disaggregate data in relation to the use of cCBT to target psychological symptoms or mental disorders. Reviews that addressed the topic of acceptability including reviews of quantitative and qualitative studies were included, and reviews that assessed effectiveness only were excluded unless they investigated one or more aspects of acceptability. Systematic reviews only were included as, usually, they include a focused research question with defined parameters and follow an a priori agreed methodology. Other review types such as scoping reviews were excluded because, typically, they tend to address a broad review question and, often, are limited due to time and scoping restraints [ ].
Reports of interventions had to indicate that that they were grounded in CBT in terms of, for example, aiming to improve or maintain mental health outcomes by changing aberrant thought patterns and behaviors. CBT interventions delivered via the internet or via computer only were included . Interventions were included if they were guided (by email, telephone, or online therapist support) or self-directed/unguided (no therapist support) and accessed in a clinical, community, or home-based setting. The duration of an intervention or the intensity with which it was delivered was not used as a criterion for excluding a review.
The scoping activity and the review team discussions identified the following key outcomes or indicators of acceptability: percentage of uptake or refusal following invitation to cCBT; adherence (number of completed intervention sessions or components); proportion of “drop-outs” (including reasons for, time of, and factors associated with dropping out); satisfaction with intervention components or mode of delivery; and user, carer, or health care professional reports about their views of cCBT.
There were no language nor year restrictions. Three non-English language papers were identified, but from examining available English abstracts, they did not meet the review criteria. Double counting of primary studies across reviews is an inherent bias of an umbrella review, and where present, efforts have been made to indicate study overlap. Some reviews included physical health conditions and non-cCBT interventions, and these findings were excluded from the umbrella review.
We followed best guidance on the conduct of umbrella reviews  and studied examples of the methodological approach that were reported in other umbrella reviews [ ]. Data were extracted by CT and checked by MD. The following data were extracted into tabular format: first author, publication year, study design, research question, searches, type of study and setting, participant details, type of intervention, quality appraisal method and scores, and research synthesis method. Thus, data were synthesized narratively at review level and under the following headings: uptake or refusal, adherence or cCBT completion, proportion and timing of “drop-outs,” satisfaction or acceptability of cCBT, and users’ views of cCBT (including therapists’ views).
The database searches identified 234 titles: 23 full-text papers remained following duplicate removal and abstract screening, and a further 8 titles were identified through citation lists of the 23 reviews. Of the 31 papers, 9 met the eligibility criteria for inclusion in the umbrella review (seefor the Preferred Reporting Items for Systematic Reviews and Meta-Analyses [PRISMA] flowchart). The 9 reviews covered 151 unique studies, with 27 of the 151 studies reported in more than one review. Individual studies varied in terms of study design including experimental design (n=111), qualitative studies (n=15), mixed methods (n=8), feasibility studies (n=7), cohort or survey design (n=4), and a systematic review (n=1). This systematic review was not identified during the electronic searches because it did not meet one or more of the concepts that informed the search strategy and it focused on the effectiveness of online mental health programs rather than directly addressing the umbrella review question. Regarding reviews that included intervention studies, most control groups were waitlist (n=21), were attention controls (n=13), received no treatment (n=12), or received treatment as usual (n=10).
Most review studies focused on adult populations (n=119), while a smaller number examined adolescent populations (n=8) and one study included both adults and adolescents . One review [ ] did not report the target population, and so, 7 of the 16 primary studies in this review were examined (9/16 studies were not accessible because of paywall restrictions). Studies covered a range of mental disorders and symptoms including depression (n=80), anxiety (n=24), panic disorder or phobia (n=26), stress or distress (n=9), insomnia (n=6), posttraumatic stress disorder (n=4), eating disorders (n=4), alcohol misuse disorders (n=3), grief (n=2), body image issues (n=1), bipolar disorder (n=1), and unspecified or nondefined “public health mental disorders” (n=1). Some studies covered more than one mental health condition. Reviews focused on individuals with clinically diagnosed disorders [ , - ], experiencing psychological symptoms [ , ], or both [ , , ]. We refer throughout the review, where possible, to whether findings relate to mental disorders or symptoms. Individuals with physical conditions or other populations (eg, cancer caregivers, survivors of natural disasters) were the focus of 9 studies, and details about the populations in 5 studies were not provided (see ).
We examined 178 cCBT interventions across the reviews, with 23 studies examining more than one cCBT condition. Two reviews were not focused only on cCBT and included a small number of non-cCBT interventions (n=1) or did not specify the type of therapeutic intervention (n=2). Detailed descriptions about interventions including their content, format, and delivery were reported infrequently. It was possible to discern, where this information was reported, that there were 53 guided and 50 unguided interventions; guidance included email support (n=11), telephone support (n=9), face-to-face contact (n=9), and a discussion forum (n=3). Technical support was provided in 25 studies (see).
Uptake or Refusal
Data on the proportion of people in each study who availed of the use of a cCBT intervention were provided in 5 reviews. Definitions of uptake varied widely and referred to qualitatively different types of individuals; for example, 1 review included the proportion of eligible individuals who were randomized and spontaneous users who accessed cCBT websites . A very wide range of uptake proportions (1%-97%) was reported across reviews [ - , ]. Only 1 review reported the proportion of people who refused (9%-24%) [ ]. In 2 reviews, the proportions who enrolled to receive cCBT and take part in its evaluation but then did not take up cCBT varied widely from 1% to 63% [ , ] (see and ). Rates of uptake, refusal, or not starting were not separated by type of cCBT (eg, self-guided or guided) in reviews, and often this information was not reported by reviews for individual cCBT programs. All 5 reviews included studies of people with depression or anxiety.
|Study||Adherence definition||Rates of uptake||Proportion completing treatmenta||Drop-outs|
|Kaltenthaler et al (2008) b||Patient recruitment; number of patients who accessed website and agreed to take part in study; drop-outs; number of patients who dropped out of the study||25% agreed to take part||Type of program (% participants did not complete components, number of studies): Beating the Blues (26%-45%, 5 studies); MoodGym (17%-75%, 2 studies) website mass recruitment; ODIN (34%, 2 studies); COPE (29%-32%, 2 studies); Recovery Road (32%, 1 study); Five Areas Approach (30%, 1 study); BALANCE (11%, 1 study); two unnamed interventions (0%, 37%)||Mean 31.75% (SD 16.52%); range 0%-75%|
|Waller and Gilbody (2009) c||Invited: those who received information about the study; recruited: those specifically asked to take part in the study; finished study: % of those who started and completed the study; finished modules: % of those who completed all modules||Median 38%; range 4%-84%||Started and subsequently completed: median 83%, range 26%-100%; overall completion: median 56%, range 12%-100%||Individuals in cCBT were twice as likely to drop-out than control groups (OR 2.03, 95% CI 0.81-5.09)|
|Melville et al (2010) d||Drop-out: “…leaving treatment before its completion…at one of a number of points throughout treatment”; pretreatment drop-out: before beginning of treatment; treatment dropout: prior to completion of treatment sessions; follow-up drop-out: prior to completing follow-up assessments; rates of drop-out; number of eligible participants (denominator) and the number of individuals who terminated at any point from registering the treatment and completing follow-up questionnaires (numerator)||Not reported||Not reported||Pretreatment drop-out: 4%-52% (median 10%; weighted mediane 21%); treatment dropout: 0%-78% (median 10%; weighted median 21%); follow-up drop-out: 0%-18% (median 4%; weighted median 8%)|
|Vallury et al (2015) f||Rates of uptake and rates of adherence||Uptake rate: 44%-86%; refusal rate: 9%-24%; did not start: 7%||Completed treatment: 33%-100%; completed follow-up at 3 months: 63%||Not reported|
|Beatty and Binnion (2016) g||Quantitative studies: adherers were those who completed program or completed the posttreatment assessment; nonadherers were those who did not complete program or did not complete the posttreatment assessment; numbers of modules/sessions/assessments completed, duration of logins, time using program, number of logins, number of homework assignments completed, accessing the program; qualitative studies: those who do not complete treatment, barriers to adherence||Uptake rate: 41.3%; did not start: 1%-63%||Completed all sessions: 16.9%-83.0%, 35.8%-66.0% for unguided or unsupported interventions, 58.0%-74.4% for guided or supported interventions; completed only one session or module: 27%-90%, 94% for guided, 95% for unguided; completed some but not all sessions or modules: 10%-99%, 67%-84% for guided interventions, 54.4%-93.0% for unguided interventions; completed all assessments: 26.0%-90.8%||Early drop-out (during first sessions): 10.0%-56.9%, 37.1% for the treatment group, 32.1% for the control group; drop-outs at end of study: 4.25%-38.00%, 56.3%-75.0% for the treatment group, 29%-48% for the control group, 35.9% for guided, 33.7% for unguided, 56% of those who immediately accessed, 80% of those who delayed access|
|Rost et al (2017) h||Uptake, drop-out, or completion rates as a means of assessing user acceptance||39%-97%||Mean 67.17% (SD 20.29%), range 26.7%-100%; 8.1%, 56% guided intervention; 16.28%, 36% unguided intervention||Mean 31.5% (SD 19.49%), range 0%-63%|
|Twomey and O’Reilly 2017 i||Proportion of participants withdrawing before final data collection, proportion of individuals who completed intervention||Not reported||10%-100%||Withdrew before posttreatment data collection: 0%-64%|
aThis was defined differently across reviews; in some cases, this included proportions of those who started and subsequently completed treatment or follow-up assessments.
bOf the 16 studies, 10 were overlapping.
cOf the 36 studies, 15 were overlapping.
dOf the 19 studies, 6 were overlapping.
eWeighted for study sample size.
fOf the 11 studies, 1 was overlapping; 4 studies included adolescent populations.
gOf the 36 studies, 6 were overlapping; 6 studies included populations with physical health or other conditions and thus were excluded.
hOf the 29 studies, 6 were overlapping; 2 studies included adolescents.
iOf the 11 studies, 5 were overlapping.
|Factor||Intervention (population)||Number of studies or participants||Results or findings (number of studies)||Heterogeneitya|
|Demographic variables (not specified)|
|Internet-based cognitive behavioral therapy (CBT; subthreshold depression)||8 studies||No difference between completers and noncompleters b||No association|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||2 studies||No association (1); men more likely to drop-out (1) c||Mixed results|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||15 studies||Women more likely to adhere (7), men more likely to adhere (1), no association with gender (7) b||Not reported|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||2 studies||No association (1)d, younger age associated with drop-out (1) c||Mixed results|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||14 studies||No association (6)d, older age associated with adherence (4), younger age associated with adherence (3), mixed findings (1) b||Not reported|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||2 studies||No association (2 d) [ ]c||No association|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||13 studies||No association (8)d, higher education associated with higher adherence (4), lower education associated with higher adherence (1) b||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||1 study||No association b||Not enough evidence|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||1 study||Being partnered associated with drop-out (1) c||No association|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||8 studies||No association with having a partner (6), being partnered associated with adherence (2) b||No association|
|Computerized CBT (cCBT; mild or moderate depression)||1 study||Getting a job provided as reason for dropping out (2) c||No association|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||7 studies||No association (7) b||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||1 study||Oceania or Europe residency associated with higher completion of modules than residency in North America, South America, and Africa (1) b||Not enough evidence|
|cCBT (prevention of anxiety and depression)||2 studies||Adherence and/or attrition among rural compared to urban participants, rurality had no effect (1) or negative effect (1) on retention to cCBT b,c||Not enough evidence|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||2 studies||Within Ireland and Australia, no association (2) b||Not reported|
|cCBT (mild or moderate depression)||2 studies||Family reasons (3) or change in circumstances (15%) or moving house (10%) provided as reasons for dropping out c||Limited evidence|
|cCBT, (common mental health disorders)||9 studies (126 participants)||Personal circumstances were stated as a reason for decliningd c||Not reported|
|cCBT (mild or moderate depression)||2 studies||Being too busy (8) provided as reason for dropping out c||Limited evidence|
|cCBT (depression)||6 studies||Lack of time reported (6) as reason for dropping out c||Not reported|
|cCBT (mild or moderate depression)||1 study||Ill-health (15%) provided as reason for dropping out of study c||Not enough evidence|
|Mental health status variables (unspecified)|
|Internet-based CBT (subthreshold depression)||8 studies||No difference between completers and noncompleters b||Not enough evidence|
|cCBT (mild or moderate depression)||1 study||Improvement in symptoms was reported as a reason for dropping out (2)e c||Mixed evidence|
|cCBT (common mental health disorders)||2 studies (2 participants)||Perceived increased risk (potential increase in symptoms) reported as reasons for drop-out d c||Mixed evidence|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||3 studies||Lower symptom severity associated with dropping out of study (3 d) [ ]c||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||20 studies||At baseline: no association (10), lower symptom severity associated with increased adherence (6) and increased module completion (1); higher symptom severity associated with higher adherence (3) b||Not reported|
|Duration of problem|
|cCBT (mild or moderate depression)||1 study||Improvement in condition (10%) provided as reason for dropping out of study c||Mixed evidence|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||2 studies||No association (2) c||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||3 studies||Longer duration associated with higher adherence (2), no association (1) b||Not reported|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||2 studies||Comorbid depression and anxiety no association (2) c||Not enough evidence|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||4 studies||Diagnosis of anxiety or depression associated with higher adherence (3), and alcohol dependency associated with higher adherence among waitlist control group (1) b||Not enough evidence|
|Medication or alcohol use|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||2 studies||No association (2; studies not targeting alcohol dependency) b||Not enough evidence|
|Treatment credibility or expectation|
|cCBT (mild or moderate depression)||4 studies||Perception that intervention is not useful (11), unhelpful (10), didn’t like treatment (n not reported), inappropriate for needs (1) provided as reason for dropping out of studye c||Mixed evidence|
|cCBT (common mental health disorders)||11 studies (101 participants)||Therapy was reported as a reason for dropping out, but it is not clear what this meant or which group it referred to c||Not reported|
|Internet-based treatment (psychological dysfunction or distress related to psychiatric conditions)||2 studies||No association (2) c||Not reported|
|cCBT (prevention of anxiety and depression)||1 study||Treatment preference fulfilment was associated with adherence to the study for rural residing participants b||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||9 studies||Positively associated with higher adherence (7), no association (2) b||Not reported|
|cCBT (depression)||6 studies||Treatment being perceived as inconvenient (4) was reported as a reason to drop out c||Not reported|
|Motivation and readiness to change|
|cCBT (mild or moderate depression)||2 studies||Low motivation (8), inability to commit (n not reported), and no desire to continue (n not reported) provided as reasons for dropping out c||Limited evidence|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||4 studies||Treatment readiness associated with higher adherence (2), and intention to complete treatment associated with higher adherence (1); intention to complete treatment no association (1) b||Not reported|
|Self-efficacy or self-confidence|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||3 studies||No association (2), taking responsibility for one’s own choices was associated with higher adherence to a bulimia self-guided program (1) b||Not enough evidence|
|cCBT (mild or moderate depression)||2 studies||Internet-related issues (5), changed mind about PC delivery (1) provided as reasons for dropping out c||Mixed evidence|
|cCBT (common mental health disorders)||5 studies (14 participants)||Information technology issues were not commonly reported as a reason for dropping out c||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||4 studies||Higher adherence was associated with website usability (1) and a positive attitude to a computerized self-guided format (1), no association between adherence and computer literacy level (2) b||Not reported|
|cCBT (depression)||4 studies||Computer or technical issues (4) were reported as reasons for dropping out of the study c||Not reported|
|Guidance or therapist support|
|cCBT (mild or moderate depression)||1 study||Preference for face-to-face help (8) provided as reason for dropping out c||Guided intervention associated with higher adherence|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||8 studies||Guided interventions were associated with higher adherence than unguided interventions (4), phone support was associated with higher adherence than email support (1), no difference in adherence between guided and unguided interventions (3) b||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||3 studies||General practitioner referral (2) or the media (1) associated with higher adherence b||Not enough evidence|
|cCBT (mild or moderate depression)||2 studies||Other help sought provided as reason for dropping out (2), treatment not demanding (n not reported) c||Not enough evidence|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||2 studies||Gratitude intervention group was twice as likely to complete treatment than a monitoring and restructuring intervention group (1); tailored feedback to increase self-efficacy, personalization of intervention team (eg, photo with “we”) increased adherence for participants who accessed all intervention components (1) b||Not reported|
|cCBT (mild or moderate depression)||2 studies||Hard to attend (13) or journey too long (3) provided as reasons for dropping out c||Not enough evidence|
|Duration of intervention|
|cCBT (mild or moderate depression)||16 studies||Duration influenced drop-out (direction of effect not reported in review); duration of interventions ranged from 1-33 sessions; the authors note that it is diﬃcult to make comparisons between cCBT programs regarding drop-out rates because of diﬀerences in study design, populations, and methods for deﬁning drop-outs and level of detail provided in a study c||Duration associated with adherence|
|cCBT (mild or moderate depression)||1 study||Intervention group had higher drop-outs compared to participants in the information website group c||Control groups more likely to adhere than intervention groups|
|cCBT (common mental health disorders)||9 studies||Participants in cCBT were twice as likely to drop out than participants in the control group (OR 2.03, 95% CI 0.81-5.09) c||Not reported|
|Self-directed psychological intervention (psychological outcomes for mental and physical disorders)||4 studies||Control (waitlist) group membership predicted higher adherence than intervention group membership (1), intervention group membership associated with adherence (1), no association with group membership (2) b||Not reported|
aAt least 50% of studies needed to provide evidence of or absence of an association; there was not enough evidence if <5 studies available.
bAssociation with adherence.
cAssociation with drop-out.
eNumber of participants (where reported in reviews).
Adherence to or Completion of One or More Intervention Sessions or Components
The definitions of adherence varied between and within reviews. For example, some reviews included studies that defined adherence in terms of “completers” or individuals who completed an outcome assessment at the end of receipt of cCBT while other studies defined “treatment completers” as individuals who completed all “modules” or a specified “dose” of cCBT even if they did not undertake a research outcome assessment . The proportion of participants who completed an entire cCBT program ranged from 10% to 100% [ - , ]. Only 1 review [ ] provided more detailed data about adherence — the proportion of individuals who discontinued after 1 session or module fluctuated between 27% and 90%, and completion of more than 1 session or module ranged from 10% to 99%.
Heterogeneity in terms of the proportion of individuals who completed cCBT was present for guided and unguided cCBT (guided cCBT: completed full program, 8%-74%; completed 1 module only, 94%; completed some modules, 67%-84%; and unguided cCBT: completed full program, 16%-66%; completed 1 module only, 95%; completed some modules, 54%-93%) [, ]. The proportions of cCBT users who remained in studies after completion of cCBT and until follow-up assessments were infrequently reported [ , ] and probably provide more evidence for the acceptability of research participation rather than cCBT acceptability.
Dropping Out or Discontinuing Use of cCBT
Two reviews reported (without providing details about timing) that, on average, around one-third of users “dropped out” [, ]. According to 2 reviews, service users dropped out pretreatment, before cCBT started (range 4%-52%) [ ], during cCBT treatment (range 0%-78%) [ ], “early” within the first few sessions (10%-57%) [ ], and during posttreatment (0%-38%) [ , ]. A wide range of cCBT participants (0%-64%) did not remain in cCBT studies for posttreatment data collection [ ].
The proportion of service users who dropped out of guided (36%) and unguided (34%) cCBT programs was similar . One review reported that cCBT participants in a treatment group were twice as likely to drop out than active attention control participants [ ]. However, this finding was neither significant nor supported by a second review [ ], which compared “early” drop-outs within the first few sessions (treatment group, 37% vs control group, 32%) to individuals who dropped out at the end of cCBT (treatment group, range 56%-75% vs control group, range 29%-48%).
Sociodemographic Factors and Adherence
Across 2 reviews, 7 of 16 studies (and 1 overlapping study) reported the absence of an association between age and adherence [, ]. The remaining 9 studies in the 2 reviews found mixed evidence (1/16) or that older age (5/16) or younger age (3/16) was associated with cCBT adherence or dropping out. Regarding gender across the same 2 reviews [ , ], there was a positive association between being female and adherence (8/16 studies) or no association (8/16 studies), and 1 study concluded that men were more likely to adhere than female participants. Overall, the evidence from the reviews indicated that there was no association between education [ , ], having a significant other [ , ], or employment status [ , ] with adherence or dropping out. One review [ ] stated (without providing supporting data) that there were no sociodemographic differences between people who completed or did not complete cCBT. The limited available evidence did not indicate that ethnicity [ ], geography [ ], or urban or rural residency [ , ] played a role in adherence (see ).
Mental Health Status–Related Factors and Adherence
The association between mental health symptom severity and adherence was unclear [, , , ]. Across 4 reviews, 10 of 26 studies reported no association between symptom severity and adherence; 9 studies observed that adherence was sustained when fewer and less severe symptoms were experienced while 7 studies reported an association between experiencing more symptoms and adherence. The relationship between duration of symptoms and adherence across 3 reviews [ , , ] was inconsistent (no association: 3/6 studies; shorter duration association: 1/6 studies; and longer duration association: 2/6 studies). In 1 review [ ], 2 studies found no association between adherence and medication or alcohol use, respectively; an examination of attrition bias (as part of risk of bias assessment) indicated that there were no mental health differences between people who completed and did not complete cCBT [ ]. The relationship between physical health status and adherence is unknown (see ).
cCBT Features and Adherence
cCBT features and adherence were examined in 4 reviews. Guided compared to unguided cCBT was associated with higher adherence (4/7 studies), or there was no association (3/7) . Adherence was better sustained via telephone compared to email support, though only 1 review addressed the type of support [ ]. A preference for face-to-face therapy compared to cCBT was stated as a reason for dropping out for 8/60 participants (only 39 participants stated reasons) in 1 study [ ]. Internet or computerized delivery problems were cited as reasons for dropping out in 2 of 3 reviews [ , ]. There was no association between computer literacy and adherence (2 studies), though cCBT website competency (1 study) and a positive attitude towards computerized delivery of CBT (1 study) were associated with higher adherence [ ]. There was wide variation in the duration of cCBT programs (range 1-33 sessions). Only 1 review tested the relationship between cCBT duration and adherence, reporting that duration of cCBT was a factor in dropping out [ ]. Higher adherence (or fewer drop-outs) was observed in control groups compared with cCBT programs [ , , ]. A meta-analysis of 9 studies [ ] reported that controls may be twice as likely as cCBT participants to drop out of studies (OR 2.03, 95% CI 0.81-5.09). The reviews did not investigate other factors such as the role of referral source, specific program content, delivery setting, access, and adherence (see ). Most were narrative reviews and did not include or included only minimal statistical results.
Behavioral Factors and Adherence
Behavioral factors and adherence were examined in 6 reviews. Negative perceptions about cCBT (eg, as unhelpful or inappropriate) were associated with dropping out [, ], while positive expectations were associated with good adherence [ , ]. The direction of the relationship between cCBT expectancies and adherence was uncertain or absent in 2 reviews [ , , ]. Low motivation was a reason for dropping out in 1 review (2 studies) [ ], and a second review [ ] found that intending or being ready to participate in cCBT was associated with higher adherence (3/4 studies). Several studies across 3 reviews reported that being too busy or not having enough time (8 studies) [ , ] and “personal circumstances” (10 studies) [ , ] were reasons for dropping out. The role of self-efficacy and adherence was investigated by a limited number of studies [ ] (see ).
Service Users’ Views
Generally, users appeared to be highly satisfied [- , ] and would recommend cCBT [ , ]; mixed reports of satisfaction were infrequently reported by individual review studies [ ]. Rural-dwelling participants were more likely than urban participants to report that cCBT improved their depression or substance misuse and were more likely to report that they were satisfied with cCBT support and liked the autonomy, confidentiality, and privacy that it afforded [ ]. The anonymous nature of remotely delivered cCBT was preferred compared to face-to-face therapies [ ], though some users reported feelings of increased isolation [ ] and uncertainty about the privacy of cCBT [ ]. The capacity to undertake cCBT in your own time was perceived to be beneficial [ - ] but also led to avoidance of cCBT [ , ]. Similarly, being too busy or unable to find the time to undertake cCBT was reported in several studies [ ]. Users experienced difficulties finding a quiet, private space to access a computer to undertake cCBT [ ]. The computerized format (and associated technical aspects) was reported to be easy to use, particularly when training was provided [ , ]. User perceptions of their information technology (IT) skills or openness to using computers impeded adherence [ - ], and the additional time that older participants required due to lower technical competency appeared to be off-putting [ ].
Predominantly based on qualitative data (with the exception of quantitative data in 1 review ), guided cCBT programs appeared to lead to greater adherence [ , , ]. Studies that compared guided and unguided cCBT were mixed or unclear regarding satisfaction and perceived helpfulness [ ]. There was a preference for face-to-face CBT [ , ], and generally users who had experience with both modes perceived cCBT as more beneficial [ ], though some studies reported criticisms by users in terms of cCBT lacking human contact and sufficient guidance [ ]. The role that therapists played in guided cCBT or face-to-face CBT was perceived positively because they had particular skills (eg, promoting understanding of condition or therapy) or characteristics (eg, more tolerant) or were considered more helpful for particular patient groups (eg, complex mental health cases) [ ]. A small number of reviews reported service users’ views about some aspects of cCBT programs. The use of a booklet, multimedia design, and program structure [ ] and program design [ ] were noted positively by users. There was a positive association between adherence and when cCBT programs were viewed as helpful versus when programs were perceived as impersonal or irrelevant [ , ]. One review [ ] reported that particular user groups had different levels of motivation and adherence levels (eg, self-referrals [higher] and mental health referrals [lower]), and delivery of cCBT in a primary care setting was viewed positively (see ).
A qualitative review containing a meta-synthesis identified 2 “core constructs” to explain factors that impeded or facilitated participation in cCBT: “sensitivity of self and identity” and “the dialectical nature of user experience.” First, the preferences, needs, and challenges that an individual with mental ill-health faces (eg, reduced motivation and concentration) need to be considered before deciding that cCBT is an appropriate option or to inform how to tailor the delivery of cCBT. Second, the delivery of cCBT needs to take into account the contradictory presence of perceived benefits and drawbacks (eg, the option to self-select modules may be empowering and burdensome, and cCBT may be perceived as enhancing confidentiality and privacy or as an obstacle to meeting the need for face-to-face support from therapists and peers) . Lack of motivation related to depression was a barrier to cCBT adherence, and feeling that cCBT was not helping was related to poor adherence [ ]. Some users appeared to drop out of cCBT programs because they were perceived to be too general, limited, or intensive or they had negative experiences with particular cCBT components [ ]. Therapists’ views about cCBT were captured by 1 study in 1 review only [ ]. Their views revolved around the following factors: the availability of resources (the need for training, computer suite, and costs); institutional support to provide resources; “culturally tailored” cCBT; cCBT as an adjunct to, rather than a replacement for, face-to-face therapy; concerns about security; and data protection. Finally, therapists rated face-to-face therapy as safer and more effective [ ] (see ).
This umbrella review investigated the acceptability of cCBT, in terms of the proportion of people who availed of, adhered to, or dropped out of cCBT as well as with reference to qualitative reports of the reasons for dropping out and users’ and therapists’ views. Nine reviews provided relevant data, though the aforementioned indicators of acceptability were defined and operationalized heterogeneously across reviews. The extent of this variation made it difficult to “sum up” the results from the 9 reviews and to compare acceptability across populations, conditions, programs, and settings. Several reviews reported that cCBT retention proportions (eg, face-to-face CBT drop-out range, 5%-38% and cCBT mean drop-out, 33%) were comparable and as variable as psychological therapy delivered face-to-face [, , , ], thereby suggesting that acceptability of cCBT may be on par with face-to-face therapies and unrelated to features of computerized delivery.
Often, reasons were not given for dropping out of studies, so it is difficult to draw firm conclusions regarding why adherence to cCBT was difficult for some individuals. It is important to understand the reasons for dropping out and whether they vary at different stages. For example, did individuals withdraw because cCBT was not what they expected, treatment was too intensive, symptoms improved (whether or not improvement was attributed to cCBT), or for other reasons separately or in combination. Included reviews did not present data regarding why individuals chose not to take up cCBT at all or that illuminated a deeper understanding about reasons for dropping out. Developing strategies to increase uptake, adherence, and retention with respect to cCBT requires a clear understanding about the factors associated with adherence or reasons for dropping out. Most reviews gathered quantitative data, and it is likely that a mixed methods research approach would help to advance our understanding about the reasons for problems and challenges. Indeed, the quantitative results were inconsistent regarding sociodemographic variables, health status, cCBT characteristics, and behavioral factors and their association with cCBT adherence.
The qualitative synthesis presented by Knowles et al  stressed the importance of taking into account individuals’ preferences and captured the contradictory nuances related to user acceptability of cCBT. The results from the quantitative studies indicated that guided cCBT was associated with better adherence, whereas qualitative findings indicated that there were favorable and unfavorable aspects to guided and unguided cCBT formats. Very few reviews directly compared guided and unguided formats, and like-with-like comparisons were not prominent. There is a need for future research to consider whether there are issues that are specific to particular types of cCBT. Targeted training may help to ease apprehensions about IT, and improving computer literacy may overcome reluctance to participate in cCBT.
There is a need for further research to improve understanding about “dosage” and dropping out. The review points clearly to the conclusion that “one size does not fit all” and that cCBT even in relatively small “doses” may work well for some patients including patients who drop out at various stages, whereas a “full dose” and perhaps even more again is needed by other patients. For example, a recent randomized controlled trial and intention-to-treat analysis found a beneficial effect for the cCBT treatment of insomnia despite a high drop-out rate and increasingly lower adherence to modules over the course of the study. It is important to note that the study was comprised of a student population and most students experienced subclinical insomnia [, ]. It was not possible to discern from our umbrella review which types of patients benefited from different “doses” of cCBT, and there are similar unanswered questions for “brief” face-to-face CBT and its variants.
Only limited attention was given to the potential for theory to illuminate understanding about the acceptability of cCBT [, ] such as diffusion of innovations theory [ ] and the Technology Acceptance Model [ ], particularly concepts around perceptions about the ease of use and the effectiveness of cCBT. The use of theories of implementation science (eg, the Consolidated Framework for Implementation Research [ ]) may also facilitate understanding about acceptability and the implementation of cCBT within a health care system, particularly if health care professionals or therapists are resistant to delivering cCBT. The limited evidence in the umbrella review regarding therapist acceptability found fewer positive views about cCBT. Acceptability and implementation may be achieved if cCBT is delivered within a “stepped care model” in which individuals begin with face-to-face therapy and then transfer to a cCBT “step,” though there may be complex cases that require ongoing individual face-to-face therapy [ ]. There may be ethical concerns that require consideration (eg, avoiding the transfer of burden from health care professionals to service users [ ] and possibly creating inequalities around internet access [ - ] or due to education level).
Though a protocol was developed for this review, it was not registered on a registry such as Prospero. However, this umbrella review was informed by an established rigorous methodology for the “summing” of the increasing number of systematic reviews about a given topic and involves independent screening, appraisal, and data extraction. There was a consensus among review authors that study or program heterogeneity and variability and inadequate reporting of details in primary studies restricted meaningful analytical comparisons between cCBT programs, and we relied upon authors’ reports of study and data type (eg, qualitative or quantitative) and use of validated outcome measures of acceptability or satisfaction. Moreover, there is a lot of variance in terms of the labels and descriptions used to describe internet or computerized interventions such as cCBT, and this lack of consensus may contribute to difficulties in research synthesis . This limitation of umbrella review methodology was compounded further due to the reliance on reviews as a secondary data source rather than searching through primary studies to try to identify missing details. It is important to remain cognizant that self-selection of participants to primary research studies (eg, more women) may influence adherence and reports of cCBT experience. This review did not apply any language restrictions, though only English-language reviews met the review criteria. Primary studies published in other languages could provide information about the acceptability of translated versions of cCBT or perhaps for migrant populations where language barriers may exist. The use of a reporting guideline such as the Template for Intervention Description and Replication (TIDieR) Checklist [ ] or even use of relevant parts (eg, Part 5 – Interventions) of the CONSORT (Consolidated Standards of Reporting Trials) extension for reporting studies of social and psychological interventions [ ] would be a major advance towards achieving a reliable synthesis and furthering our understanding about the acceptability of cCBT.
Collectively, the results of the reviews indicate that “one size does not fit all” regarding the acceptability of cCBT and that individual patient and service user group tailoring of cCBT are required to increase the population reach and uptake of cCBT and adherence and so, deliver treatment benefits and improve mental health. In turn, tailoring to create as close a patient-treatment match as possible is likely to involve a blending process involving a human guide or therapist.
This study was supported by the Academy of Finland (grants 315343 and 330527) and the Strategic Research Council at the Academy of Finland (grants 327145 and 327148 for the DigiIn Project). MD and AK received funding from the Economic and Social Research Council (ESRC) (grant ES/S00744X/1) during the writing of this article.
Conflicts of Interest
Supplementary table 1: quality appraisal.DOCX File , 16 KB
Supplementary table 2: included review characteristics.DOCX File , 35 KB
Supplementary table 3: satisfaction and acceptability.DOCX File , 29 KB
Supplementary table 4: qualitative findings.DOCX File , 16 KB
- Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980-2013. Int J Epidemiol 2014 Apr;43(2):476-493 [FREE Full text] [CrossRef] [Medline]
- Cuijpers P, Koole SL, van Dijke A, Roca M, Li J, Reynolds CF. Psychotherapy for subclinical depression: meta-analysis. Br J Psychiatry 2014 Oct 02;205(4):268-274 [FREE Full text] [CrossRef] [Medline]
- Cuijpers P, Smit F, van Straten A. Psychological treatments of subthreshold depression: a meta-analytic review. Acta Psychiatr Scand 2007 Jun;115(6):434-441. [CrossRef] [Medline]
- Titov N, Andrews G, Sachdev P. Computer-delivered cognitive behavioural therapy: effective and getting ready for dissemination. F1000 Med Rep 2010 Jul 14;2:1. [CrossRef] [Medline]
- Kaltenthaler E, Parry G, Beverley C, Ferriter M. Computerised cognitive-behavioural therapy for depression: systematic review. Br J Psychiatry 2008 Sep;193(3):181-184. [CrossRef] [Medline]
- Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78. [CrossRef] [Medline]
- Wright JH, Mishkind M, Eells TD, Chan SR. Computer-Assisted Cognitive-Behavior Therapy and Mobile Apps for Depression and Anxiety. Curr Psychiatry Rep 2019 Jun 27;21(7):62. [CrossRef] [Medline]
- Kaltenthaler E, Cavanagh K. Computerised cognitive behavioural therapy and its uses. Prog. Neurol. Psychiatry 2010 Jun 09;14(3):22-29. [CrossRef]
- Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295. [CrossRef] [Medline]
- Andersson G, Topooco N, Havik O, Nordgreen T. Internet-supported versus face-to-face cognitive behavior therapy for depression. Expert Rev Neurother 2016 Dec 15;16(1):55-60. [CrossRef] [Medline]
- Depression in adults: recognition and management (Clinical Guideline 90). National Institute for Health and Clinical Excellence (NICE). 2009 Oct 28. URL: https://www.nice.org.uk/guidance/cg90 [accessed 2021-06-19]
- Hollinghurst S, Peters TJ, Kaur S, Wiles N, Lewis G, Kessler D. Cost-effectiveness of therapist-delivered online cognitive-behavioural therapy for depression: randomised controlled trial. Br J Psychiatry 2010 Oct 02;197(4):297-304. [CrossRef] [Medline]
- Eells TD, Barrett MS, Wright JH, Thase M. Computer-assisted cognitive-behavior therapy for depression. Psychotherapy (Chic) 2014 Jun;51(2):191-197. [CrossRef] [Medline]
- Gega L, Marks I, Mataix-Cols D. Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol 2004 Feb;60(2):147-157. [CrossRef] [Medline]
- Fiorillo A, Gorwood P. The consequences of the COVID-19 pandemic on mental health and implications for clinical practice. Eur Psychiatry 2020 Apr 01;63(1):e32 [FREE Full text] [CrossRef] [Medline]
- Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013 Dec 01;8:139. [CrossRef] [Medline]
- Holmes EA, O'Connor RC, Perry VH, Tracey I, Wessely S, Arseneault L, et al. Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry 2020 Jun;7(6):547-560. [CrossRef] [Medline]
- Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc 2015 Sep;13(3):132-140. [CrossRef] [Medline]
- Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar 19;38(2):65-76 [FREE Full text] [CrossRef] [Medline]
- Lewis C, Roberts N, Bethell A, Robertson L, Bisson J. Internet-based cognitive and behavioural therapies for post-traumatic stress disorder (PTSD) in adults. Cochrane Database Syst Rev 2018 Dec 14;12(12):CD011710 [FREE Full text] [CrossRef] [Medline]
- Joyce S, Modini M, Christensen H, Mykletun A, Bryant R, Mitchell PB, et al. Workplace interventions for common mental disorders: a systematic meta-review. Psychol. Med 2015 Dec 01;46(4):683-697. [CrossRef]
- Scott SD, Rotter T, Flynn R, Brooks HM, Plesuk T, Bannar-Martin KH, et al. Systematic review of the use of process evaluations in knowledge translation research. Syst Rev 2019 Nov 07;8(1):266 [FREE Full text] [CrossRef] [Medline]
- Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017 Sep 21;358:j4008 [FREE Full text] [CrossRef] [Medline]
- Grant M, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009 Jun;26(2):91-108 [FREE Full text] [CrossRef] [Medline]
- Alexandre K, Desrichard O, Burnand B, Peytremann-Bridevaux I. Factors influencing self-management in adults with diabetes: an umbrella review protocol. JBI Database System Rev Implement Rep 2017 Nov;15(11):2630-2637. [CrossRef] [Medline]
- Vallury KD, Jones M, Oosterbroek C. Computerized Cognitive Behavior Therapy for Anxiety and Depression in Rural Areas: A Systematic Review. J Med Internet Res 2015 Jun 05;17(6):e139 [FREE Full text] [CrossRef] [Medline]
- Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [CrossRef] [Medline]
- Rost T, Stein J, Löbner M, Kersting A, Luck-Sikorski C, Riedel-Heller SG. User Acceptance of Computerized Cognitive Behavioral Therapy for Depression: Systematic Review. J Med Internet Res 2017 Sep 13;19(9):e309 [FREE Full text] [CrossRef] [Medline]
- Beatty L, Binnion C. A Systematic Review of Predictors of, and Reasons for, Adherence to Online Psychological Interventions. Int J Behav Med 2016 Dec;23(6):776-794. [CrossRef] [Medline]
- Melville KM, Casey LM, Kavanagh DJ. Dropout from Internet-based treatment for psychological disorders. Br J Clin Psychol 2010 Nov;49(Pt 4):455-471. [CrossRef] [Medline]
- Zhou T, Li X, Pei Y, Gao J, Kong J. Internet-based cognitive behavioural therapy for subthreshold depression: a systematic review and meta-analysis. BMC Psychiatry 2016 Oct 21;16(1):356 [FREE Full text] [CrossRef] [Medline]
- Twomey C, O'Reilly G. Effectiveness of a freely available computerised cognitive behavioural therapy programme (MoodGYM) for depression: Meta-analysis. Aust N Z J Psychiatry 2017 Mar 11;51(3):260-269. [CrossRef] [Medline]
- Knowles SE, Toms G, Sanders C, Bee P, Lovell K, Rennick-Egglestone S, et al. Qualitative meta-synthesis of user experience of computerised therapy for depression and anxiety. PLoS One 2014;9(1):e84323 [FREE Full text] [CrossRef] [Medline]
- Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med 2008 Nov;38(11):1521-1530. [CrossRef] [Medline]
- Freeman D, Sheaves B, Goodwin GM, Yu L, Nickless A, Harrison PJ, et al. The effects of improving sleep on mental health (OASIS): a randomised controlled trial with mediation analysis. Lancet Psychiatry 2017 Oct;4(10):749-758 [FREE Full text] [CrossRef] [Medline]
- Lallukka T, Sivertsen B. Sleep improvement by internet-based cognitive behavioural therapy might help prevent mental health disorders. The Lancet Psychiatry 2017 Oct;4(10):734-735. [CrossRef]
- Rogers E. Diffusion of Innovation. New York, NY: Free Press; 1995.
- Holden RJ, Karsh B. The technology acceptance model: its past and its future in health care. J Biomed Inform 2010 Feb;43(1):159-172 [FREE Full text] [CrossRef] [Medline]
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009 Aug 07;4:50 [FREE Full text] [CrossRef] [Medline]
- Alam K, Mahumud RA, Alam F, Keramat SA, Erdiaw-Kwasie MO, Sarker AR. Determinants of access to eHealth services in regional Australia. Int J Med Inform 2019 Nov;131:103960. [CrossRef] [Medline]
- Latulippe K, Hamel C, Giroux D. Social Health Inequalities and eHealth: A Literature Review With Qualitative Synthesis of Theoretical and Empirical Studies. J Med Internet Res 2017 Apr 27;19(4):e136 [FREE Full text] [CrossRef] [Medline]
- Gilmour JA. Reducing disparities in the access and use of Internet health information. a discussion paper. Int J Nurs Stud 2007 Sep;44(7):1270-1278. [CrossRef] [Medline]
- Smoktunowicz E, Barak A, Andersson G, Banos RM, Berger T, Botella C, et al. Consensus statement on the problem of terminology in psychological interventions using the internet or digital components. Internet Interv 2020 Sep;21:100331 [FREE Full text] [CrossRef] [Medline]
- Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014 Mar 07;348(mar07 3):g1687-g1687. [CrossRef] [Medline]
- Montgomery P, Grant S, Mayo-Wilson E, Macdonald G, Michie S, Hopewell S, CONSORT-SPI Group. Reporting randomised trials of social and psychological interventions: the CONSORT-SPI 2018 Extension. Trials 2018 Jul 31;19(1):407 [FREE Full text] [CrossRef] [Medline]
|AMSTAR: A Measurement Tool to Assess Systematic Reviews|
|CBT: cognitive behavioral therapy|
|cCBT: computerized cognitive behavioral therapy|
|CONSORT: Consolidated Standards of Reporting Trials|
|iCBT: internet-delivered cognitive behavioral therapy|
|IT: information technology|
|NICE: National Institute for Health and Clinical Excellence|
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|TIDiER: Template for Intervention Description and Replication|
Edited by J Torous; submitted 31.07.20; peer-reviewed by J Apolinário-Hagen, K Mathiasen; comments to author 23.09.20; revised version received 26.01.21; accepted 28.04.21; published 06.07.21Copyright
©Charlene J Treanor, Anne Kouvonen, Tea Lallukka, Michael Donnelly. Originally published in JMIR Mental Health (https://mental.jmir.org), 06.07.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.