RefCheck Maintenance Notice

On Monday, December 3, 2018, from 16:00-18:00 EST, RefCheck will be undergoing maintenance. RefCheck is the process where, during copyediting, all references are extracted from the manuscript file, parsed, matched against various databases (eg, PubMed and CrossRef), and automatically corrected. For more information on RefCheck, please visit our Knowledge Base.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 02.10.18 in Vol 5, No 4 (2018): Oct-Dec

Preprints (earlier versions) of this paper are available at, first published Feb 23, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Supported Internet-Delivered Cognitive Behavior Treatment for Adults with Severe Depressive Symptoms: A Secondary Analysis

    1Clinical Research & Innovation, SilverCloud Health, Dublin, Ireland

    2E-Mental Health Group, School of Psychology, Trinity College Dublin, Dublin, Ireland

    Corresponding Author:

    Derek Richards, PhD

    Clinical Research & Innovation

    SilverCloud Health

    One Stephen Street

    Stephen Street Upper

    Dublin, D08 DR9P


    Phone: 353 15549771



    Background: Depression is a highly prevalent mental health issue that exacts significant economic, societal, personal, and interpersonal costs. Innovative internet-delivered interventions have been designed to increase accessibility to and cost-effectiveness of treatments. These treatments have mainly targeted mild to moderate levels of depression. The increased risk associated with severe depression, particularly of suicidal ideation often results in this population being excluded from research studies. As a result, the effectiveness of internet-delivered cognitive behavioral therapy (iCBT) in more severely depressed cohorts is less researched.

    Objective: The aim of this study is to examine the effect of iCBT on symptoms of severe depression, comorbid symptoms of anxiety, and levels of work and social functioning.

    Methods: Retrospective consent was provided by participants with elevated scores (>28 severe depression symptoms) on the Beck Depression Inventory (BDI-II) who accessed an iCBT intervention (Space from Depression) with support for up to 8 weeks. Data were collected at baseline, posttreatment, and 3-month follow-up on the primary outcome (BDI-II), and secondary outcomes (the Generalized Anxiety Disorder-7 and the Work and Social Adjustment Scale).

    Results: A significant change was observed on all measures between pre- and postmeasurement and maintained at 3-month follow-up. Clinical improvement was observed for participants on the BDI-II from pre- to postmeasurement, and suicidal ideation also reduced from pre- to postmeasurement.

    Conclusions: Users of Space from Depression with symptoms of severe depression were found to have decreased symptoms of depression and anxiety and increased levels of work and social functioning. The intervention also demonstrated its potential to decrease suicidal ideation. Further investigation is required to determine why some individuals improve, and others do not. iCBT may have the potential to be used as an adjunct treatment for severe depression symptoms, but participants may require further treatment if they receive iCBT as a standalone intervention. Although promising, further research incorporating control groups is needed to support the utility of Space from Depression for use in or as an adjunct to treatment for severe depression.

    JMIR Ment Health 2018;5(4):e10204





    Depression is a serious public health concern and is predicted to be the leading cause of disability in the world by 2030 [1]. Depression exacts significant economic, societal, personal, and interpersonal costs [2]. Consequently, numerous interventions, both psychological [3] and pharmacological [4], have been implemented to reduce its prevalence.

    Treatment for Depression

    Evidence-based treatments for depression can comprise of pharmacological and psychological interventions [5]. Historically, pharmacological interventions have been given precedence, but psychological interventions have demonstrated their efficacy [6]. Research also suggests that two-thirds of clients with depression hold a preference for psychological interventions over pharmacological solutions [7]. However, accessing evidence-based psychological interventions can be difficult because of the many barriers [8,9], including not having access to treatment due to a lack of health service resources [8]. These barriers can be more pronounced for severe depression [10], and most individuals with severe levels of depression do not receive adequate treatment [11].

    A typical diagnosis of depression as per the International Classification of Diseases, Tenth Revision involves an individual having at least one of 3 key symptoms of depression (depressed mood, loss of interest in daily activities, reduction in energy) for 2 weeks. Depression severity can be mediated by the presence of several other symptoms, including low self-confidence, disturbed sleep, poor concentration or suicidal thoughts [12]. The differentiation between severe depression and a mild/moderate episode is that symptoms are more pronounced and numerous, while also impairing functioning to a greater extent [13].

    Internet-Delivered Interventions

    Innovative attempts to overcome these barriers have led to the development of internet-delivered psychological interventions, primarily based on cognitive behavioral principles for the treatment of depression and other mental health disorders [14]. Internet-delivered interventions can overcome some of the barriers associated with traditional treatment methods [15]. Meta-analyses have demonstrated positive results with internet-delivered cognitive behavioral therapy (iCBT) for depression [16,17], with The National Institute for Health and Care Excellence (NICE) guidelines recommending iCBT as a treatment option in the management of mild to moderate depression [13]. A legacy issue within the field of internet interventions has been that most studies have focused on participants who present within this mild to moderate symptom range, meaning the evidence base for the use of internet-delivered interventions with severe presentations is less established [18].

    Nevertheless, this is changing, and some studies have demonstrated the effectiveness of iCBT in reducing symptoms of severe depression [14,19-22], with some illustrating maintenance of improvements at 6-month follow-up [19]. However, the complexities associated with delivering a successful internet-delivered intervention for this population are naturally more challenging, with some studies suggesting that severely depressed individuals participate less in iCBT [23] and sometimes demonstrate less improvement than the mild to moderately depressed group [24]. Despite this, iCBT programs tailored to each individual have been found to produce similar effects for those with severe depression, as those with mild to moderate levels [25]. Such findings have also been demonstrated in a younger cohort [23]. Progressing implementation of internet-delivered interventions with severely depressed cohorts has been much slower than in mild to moderately depressed groups, particularly because of the increased risk of suicide associated with this population [26]. Research evaluating internet-delivered interventions often exclude suicidal ideation as an ethical consideration. As such, their effects on reducing suicidal ideation and ultimately the risk of suicide are not well documented [27,28]. However, it stands that the literature in this field is slowly growing and has illustrated the benefit of iCBT for individuals with greater severity of depressive symptoms.

    Where some health systems have been efficient in their adoption of iCBT and other internet-delivered interventions (the improving access to psychological therapies program in the United Kingdom), the use of iCBT in natural settings is not well documented, where iCBT may be classified as an inappropriate treatment option for severe cases. For example, the NICE guidelines for depression [13] identify iCBT and computerized therapies as appropriate treatments for mild-moderate depression, but not necessarily for more severe presentations. Despite this caution, a meta-analysis by Bower et al [14] regarding the impact of initial severity of depression on the effectiveness of low-intensity psychological interventions (both written/physical and online), concluded that those with severe presentations of depression benefit equally to those in lower severity categories. Furthermore, the ongoing monitoring of symptoms, that is facilitated seamlessly through technology, ensures accurate care delivery and informs clinical decisions on the care being delivered. These statements are further supported by the work of Watts et al [29], whose analysis of patients prescribed iCBT for depression with comorbid suicidal ideation illustrated significant decreases from pre- to posttreatment. Thus, despite the limited evidence so far, it seems appropriate, providing the intervention is safely delivered, to explore the potential of iCBT as a treatment for more severe depressive symptomatology.

    iCBT can be of benefit to services as a frontline treatment for depression. As a low-intensity intervention, it consumes less clinical resource in its administration. Within a stepped care model, an individual can receive and benefit from an iCBT intervention while they wait for higher intensity treatment resources to become available. However, technology has since progressed, allowing for the delivery of treatments on more robust systems. One such intervention is SilverCloud, but the current utility of this low-intensity treatment for those with severe depression is unknown.

    This study aims to explore the effect of the SilverCloud iCBT intervention on those with severe depression. It analyzes a cohort that did not meet the criteria for inclusion in a randomized controlled trial (RCT) investigating the impact of iCBT on mild to moderate depression. The authors hypothesized the following:

    • The intervention could impact positively on self-reported symptoms of depression, anxiety and work, and social functioning within a cohort with severe depression symptoms.
    • The results of this study offer a unique opportunity to assess the effects on suicidal ideation in the cohort, and the authors hypothesized that suicidal ideation would be lower post intervention.
    • Clinically meaningful change in depressive symptoms will be observed for the group.



    Participants in this secondary analysis consisted of a subsample from a larger RCT [27,28]. The larger RCT recruited participants who were within the mild to moderate symptom severity as determined by a score of between 14 and 28 on the Beck Depression Inventory (BDI-II). Those scoring above these criteria on the BDI-II (ie, within the “severely” depressed range) were excluded from the main RCT study but were offered access to treatment with support and invited to complete posttreatment and follow-up questionnaires. Individuals scoring in this range also received a referral to the general practitioner as per risk protocols. No case-control group was recruited for this cohort as it extended beyond the scope of the main research study.

    Participants and Recruitment

    Table 1 describes the sociodemographic characteristics of the sample. Participants of the present study scored within the “severe” range for depression (>28 on the BDI-II) [30]. As part of the original RCT protocol, participants were recruited via outreach work and information on the website of Aware, a national depression charity in Ireland. Two-hundred and eleven (N=211) participants were excluded from the main RCT due to symptom severity. An ethical amendment was submitted and approved at a later date to contact these individuals via email to request permission for their data to be analyzed as part of the current secondary analysis. Of the 211, 67 (32%) participants replied and consented for their data to be used in the secondary analysis of outcomes. In total, 67 participants provided data at baseline on all measures, with responses on measures from 33 (49%) participants on the BDI-II and Work and Social Adjustment Scale (WSAS), 34 (51%) on the Generalized Anxiety Disorder-7 item inventory (GAD-7) at post treatment, and from 22 (33%) on all measures at 3-month follow-up.


    Participants were assessed at baseline and posttreatment (after 8 weeks of service provision) and again at 3-month follow-up. At baseline, the BDI-II, Sociodemographic and History Questionnaire, GAD-7, and the WSAS were completed for screening purposes. After that, the BDI-II, GAD-7, and WSAS were completed at the end of treatment, week 8, and at 3-month follow-up. The 21-item BDI-II [30] is a widely used questionnaire measuring symptoms and severity of depression based on the criteria for depressive disorder diagnosis as outlined in The American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) [31].

    Table 1. Participant demographics (N=67).
    View this table

    The scale designates levels of severity: minimal (0-13), mild (14-19), moderate (20-28), and severe (29-63). The BDI-II has been found to have excellent internal consistency and test-retest reliability with a diverse range of samples [30,32,33]. It has also demonstrated good convergent validity with other measures of depression among clinical and nonclinical adult samples [34]. Reliability analysis of this measure for the current sample indicated appropriate levels of internal consistency (alpha=.71).

    The GAD-7 [35] comprises 7 items measuring symptoms and severity of anxiety based on the DSM-IV diagnostic criteria. The GAD-7 has good internal consistency (alpha=0.89) and good convergent validity with other anxiety scales [36]. The GAD-7 is increasingly used in large-scale studies as a generic measure of changes in anxiety symptomatology, using a cutoff score of 8 [37,38]. Reliability analysis of this measure for the current sample indicated appropriate levels of internal consistency (alpha=.84).

    The Work and Social Adjustment Scale (WSAS) [39] is a simple, reliable, and valid measure of impaired daily functioning across five dimensions: work, social life, home life, private life, and close relationships. Reliability analysis of this measure for the current sample indicated appropriate levels of internal consistency (alpha=.72).

    The Intervention

    The SilverCloud Space from Depression program is a 7-module, internet-delivered cognitive behavioral program, with an option to unlock further content as the user progresses. Program content is delivered on a web 2.0 platform and features several forms of rich media (videos, animations, and audio) to facilitate the delivery of the intervention. Treatment content consists of cognitive and behavioral strategies common to CBT protocols: behavioral activation, cognitive restructuring, and activity scheduling [28]. For the current trial, the program was delivered in a supported format and participants received brief asynchronous feedback based on their progress each week for a period of up to 8 weeks. A detailed description of the modules is provided in Table 2.

    Trained volunteers from a national depression charity in Ireland provided support for the intervention. The role of the supporter in this trial was to provide motivational support and encouragement to their assigned patient. The cohort undertook several face-to-face training sessions hosted by the charity before commencing in their role, with the content of the training designed to educate on CBT, the program, the role of the supporter, and how best to respond to patients in distress. An assistant psychologist was employed by the depression charity to assist the supporters, as well as to monitor all correspondence between supporters and their patients. This individual reported directly to the Education and Online Services coordinator, who regularly consulted with the clinical director of the charity. Further risk protocols were incorporated at review points, where a client was escalated along service protocols if they shared information with their supporter that indicated a risk to themselves or others. Where patients indicated a response greater than zero on item 9 of the BDI-II, the supporter was sent an email alert, and clients were presented with “get help now” links and local crisis numbers. This protocol was standard for all patients of the charity, and not just those in the severe group. The supporter would then contact the patient at their earliest convenience, and they would discuss the options available to them.

    Table 2. Contents of Space from Depression and brief descriptions of modules.
    View this table

    Ethical Approval

    Original ethics approval was received for the study on November 25, 2013. Posthoc ethical approval was granted on November 16, 2015, to contact those scoring in the severe ranges of the BDI-II. The subsample was provided with information on the secondary analysis and participants were requested to provide informed consent should they wish to participate.

    Data Analysis

    Baseline comparisons across the variables of age, gender, education level, and employment status were conducted, along with baseline comparisons across the measures of BDI-II, GAD-7, and WSAS between data gathered from all clients with “severe” depression (N=211) as per the BDI-II, and those who consented for their data to be used in the follow-up analysis (n=67).

    Multivariate imputation by chained equations was applied to impute missing question scores using the R MICE v2.0 package [40]. Variables were then imputed with 100 imputations for 30 iterations using predictive mean matching. Following imputation, linear mixed models were fitted for each of the three measures (BDI-II, GAD-7, and WSAS) using the R package lme4 v1.1-17 [41], pooling results over the 100 individual datasets. This method was chosen due to its robustness in the handling of missing data, as this study recorded 49% (33/67) missing data at posttreatment and 68% (46/67) missing data at 3-month follow-up. F tests for fixed effects were carried out with the ANOVA (analysis of variance) function of the R package ImerTest [42], using Satterthwaite’s method for denominator degrees-of-freedom approximation. Effect sizes (Cohen d), averaged over all imputed datasets, were calculated using the mean change and standard deviation of change scores which is a recommended method for repeated measures designs [43-45]. Clinically significant change and deterioration were calculated based on a movement of 9 or more points on the BDI-II from pre- to posttreatment measurement [46]. Reliable recovery was calculated based on a movement of 9 more points, as well as having a posttreatment score of less than 10 on the BDI-II [30]. As per this methodology, 4 categories were established: reliable improvement, reliable deterioration, reliable recovery, and no change.


    Baseline Characteristics

    No significant differences were observed between the BDI-II, GAD-7, and WSAS using t tests between the overall cohort with severe depression (N=211) and those who consented (n=67) to having their posttreatment and follow-up data included in the analysis. Across demographic variables, no significant differences in gender, employment status or age were observed at baseline. However, a significant difference was observed on the level of education between the 2 groups (χ24=10.0, P=.04), with the consenting participants having a higher level of education overall.

    Treatment Response

    Participants engagement with the treatment was positive. To begin with, the participants completed a mean of 17.4 sessions (SD 17.3) over the duration of the treatment period. The mean session time per participant was 0.49 hours (SD 0.41), which amounts to a total mean exposure to the active treatment of 9.22 hours (SD 10.57). However, examining the standard deviation of this total time on the platform would suggest a large variance (ie, 10.57 hours). To illustrate this data further, quartiles have been reported in Table 3.

    Multiple Imputation

    The Little MCAR test revealed a nonsignificant result (χ216=10.8, P=.82), indicating that the data were missing completely at random and not contingent on any measured variables within the data set. Further exploration of the data indicated that no other variables (education level, gender, employment, BDI-II baseline score, GAD-7 baseline score, WSAS baseline score) were significantly different between participants who completed exit measures and participants who did not complete exit measures (χ2 and t tests). Similarly, these were not significantly different between participants who completed 3-month measures and participants who did not complete 3-month measures.

    Posttreatment and Follow-Up Effects

    Linear mixed-model ANOVAs were conducted separately on the participant data (N=67) using time as the within-subjects variable for the BDI-II, GAD-7, and WSAS. For the BDI-II, a significant time effect on depression was found. The BDI-II scores significantly decreased from baseline with a mean of 35.94 (SD 6.91), to posttreatment with a mean of 23.76 (SD 9.68), to 3-month follow-up with a mean 14.52 (SD 7.45) measurement points, (F2,132=174.9, P<.001). Suicidality, as per question 9 on the BDI-II, was also found to significantly decrease across the measurement time points (F2,132=18.5, P<.001). Similar trends were observed in regards to the analyses of secondary measures, where significant decreases in scores were also observed for the GAD-7 (F2,132=98.3, P<.001), and significant increases for WSAS (F2,132=33.0, P<.001). Descriptive statistics are presented in Table 4.

    Reliable Change

    Reliable change for the imputed data of the “severely” depressed group was explored using participants that had BDI-II data at both pre- and posttime points. At posttreatment, 43/67 (64%) participants were classified as reliably improved. Of those who were classified as reliably improved, 6 of the 67 (9%) participants met the criteria for recovery. The remaining 18/67 (36%) individuals were classified as unchanged, where scores did not exceed a movement of 9 points in either direction.

    Table 3. Treatment response data (N=67).
    View this table
    Table 4. Descriptive statistics of the sample (n=67).
    View this table


    Principal Findings

    The current paper sought to investigate the impact of an internet-delivered CBT intervention for the treatment of depression, Space from Depression, on symptoms of severe depression. The BDI-II was observed to have significant decreases with large effect sizes from pre- to post and 3-month follow-up timepoints. The results are encouraging, with participants demonstrating solid changes from pre-post assessment, and the changes beyond that time are suggestive of a continued positive change in symptom reduction. Baseline differences between the overall sample (N=211) and those who consented (n=67) found that those who consented had higher education levels, but did not vary across other clinical or sociodemographic variables. This difference in education is indicative of the wider literature; individuals with higher levels of education are more likely to participate in research [47], potentially stemming from increased levels of volunteerism [48] and belief in science [49] in this population. Treatment response rates for the study were also good, with 50/67 (75%) of participants completing 5 or more sessions.

    The results demonstrate comparative effect sizes of improvement in severe depressive symptoms to those experienced in a mild to moderately depressed group, supporting the use of iCBT for severe depression [14,19]. Over half of the participants, 43/67 (64%) who provided pre- and posttreatment data experienced reliable improvement by participating in the intervention, supporting the potential for iCBT as an effective measure in this population. However, mean scores on the BDI-II remained within the moderately depressed range at posttreatment indicating that the majority of participants would still require further psychological treatment, despite improvement in their depressive symptomatology. These results are consistent with the conclusions of Bower et al [14], where although clinically reliable change was observed in the sample, the majority of those improved were still within clinical ranges (37/67, 55%) and would require further treatment. Further, despite no control group being recruited, limiting the statements that can be made in regards to effectiveness, this severe sample was found to have actively used the intervention over their course of treatment. This provides some support for the relationship between the intervention and the outcomes being reported.

    Outcomes on secondary measures of the GAD-7 and WSAS significantly decreased from pre- to post timepoints, with significant differences and large effect sizes observed. These scores did not significantly differ at follow-up points, implying a maintenance of gains across the sample from post-treatment to 3-month follow-up, similar to what was observed by Meyer et al [19]. The decrease in anxiety scores as a result of iCBT treatment for depression was mirrored in the larger RCT this sample was derived from [28]. Reduction of comorbid symptoms in severe depression may be of particular importance to this population given the association that has been found between the severity of depressive symptoms and greater instances of comorbid conditions [50].

    Increasingly in depression research, the need to include data points relevant to quality of life and functional impairment has been recognized [51,52]. The current study utilized the WSAS, and administering measures such as this can provide researchers and practitioners with deeper insight into current levels of experienced impairment and the recovery process. As a multi-domain scale encompassing areas such as work, leisure activities and home life, the improvements observed in the WSAS illustrate the positive consequences of a low-intensity iCBT intervention outside of decreasing depression symptoms. Other studies utilizing iCBT have recorded similar improvements in their samples [53-55], yet qualitative investigations may be necessary to understand the impact of iCBT on improvements in functioning further. Although preliminary, these results provide support for the helpfulness of iCBT, where an effective intervention should demonstrate its ability to produce outcomes not just on disorder symptomatology, but also in the areas of life typically affected by the disorder.

    Clinically significant change is an important concept due to its potential to provide a complete picture of the impact of interventions, which goes beyond the averages of the group and outlines change at an individual level [56]. Conducting reliable change analyses also allows for researchers to be confident that changes in the score are more than fluctuations around a data point and, to an extent, regression to the mean. The approach adopted in this study was conservative and robust, and the reliable change analysis in research completers illustrated that 43/67 (64%) individuals reliably improved, with the remainder being unchanged. Six of the 67 (9%) individuals recovered using the criteria of <10 and a 9-point change on the BDI-II. The results further show potential for the use of internet-delivered interventions for severe populations, where a large proportion of individuals in a cohort with severe presentations achieved meaningful reductions in symptoms, and in some cases transitioning to mild-moderate categories from severe.

    Suicidal ideation was significantly reduced from pre- to postintervention. The authors acknowledge that despite mean scores being less than 1 on this item of the BDI-II (where a score of 1 on the measure indicates the presence of suicidal thoughts without intentions), the results nonetheless lend support to previous trials investigating this area [57]. The results provide preliminary evidence for the feasibility of online interventions as effective means to tackle the increased risk of suicide in depressed populations.

    The current study demonstrated the potential for a supported internet-delivered intervention to alleviate symptoms in individuals with severe presentations. Implementing supported iCBT interventions allows for sufficient monitoring of patients and the capability to intervene and offer further support in the case where risk escalates [19]. It is relatively unknown whether individuals with severe depression benefit more, or less from support than those in the mild to moderate range, though recent analyses have suggested that there is no difference in how populations of severity benefit from low-intensity interventions [14,58]. Further research would be helpful to delineate mechanisms associated with internet-delivered interventions and how they fluctuate depending on the population and severity.


    While the initial results from this study are promising, the original protocol meant that no control group was recruited and therefore caution is advised as we are unable to conclude that the treatment delivered is responsible for the observed effects. We received a response rate of 37.1% (67/211) of the sample, which can be considered a limitation and represents a potential for self-selection bias. At baseline, we found no significant differences between responders and non-responders on all clinical and socio-demographic variables apart from education. Higher levels of education were found for participants who consented for their follow-up data to be used for this secondary analysis. The authors acknowledge that this may bias the data. However, it has been highlighted that differences in education do not impact on treatment response and outcome in CBT-therapies [59]. The result highlights the need for further investigation to discern the mediating and moderating effects of education and other variables on outcome from iCBT treatment for severe depression.

    Further investigations implementing RCT or feasibility designs are warranted to discern the utility of Space from Depression in more severe cohorts, where our findings must also be understood within the context of the relatively small sample which gave consent for their data to be reported. Research regarding the long-term effects of an online intervention on severe depression would also be an important consideration. Participants’ depression and anxiety symptoms were indexed by self-report rather than clinical diagnosis. However, the authors employed standardized self-report measures that are well-established and have previously been used in numerous research studies relating to depression and internet-delivered interventions. Another potential limitation lies in the fact that the intervention was designed for a mild to moderately depressed group, and so, there may be some aspects that are less relevant for those with severe depression. A final limitation of the study is the large amount of missing data that was observed at posttreatment and 3-month follow-up for these individuals, and the authors, therefore, acknowledge that the results and their generalizability should be interpreted with caution.


    The current study demonstrated the potential for an internet-delivered intervention to reduce symptoms of severe depression. The participants demonstrated reliable decreases in anxiety symptoms and improvements in work and social functioning. Furthermore, reliable improvement in depression symptomology was observed. Suicidal ideation was reduced as a result of engaging in the intervention, and these results suggest that internet-delivered interventions may have the potential to provide a robust method of risk assessment and monitoring. Current treatment guidelines, such as NICE [13], recommend iCBT as a low-intensity treatment option for mild to moderate presentations of depression and advise high-intensity treatments for severe forms of depression. The present findings provide preliminary evidence to justify further research into the utilization of iCBT and the SilverCloud intervention as part of a treatment plan for severe depression that could prove beneficial for some individuals.


    The authors greatly acknowledge SilverCloud Health for funding this study. DR and LT conceptualized the original trial from which this secondary analysis was derived. DD and JB wrote the initial draft of the paper, with reviews from DR, LT, and MA contributing to the final draft. SC was the data manager and statistician. DD and SC developed and actioned the data analytic protocol.

    Conflicts of Interest

    DR, DD, JB, MA, and SC are employees of SilverCloud Health, developers of computerized psychological interventions for depression and anxiety, stress and comorbid long-term conditions. LT is a research consultant for SilverCloud Health who is based in Trinity College, Dublin.


    1. Mathers CD, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med 2006 Nov;3(11):e442 [FREE Full text] [CrossRef] [Medline]
    2. Donohue JM, Pincus HA. Reducing the Societal Burden of Depression. PharmacoEconomics 2007;25(1):7-24. [CrossRef]
    3. Linde K, Sigterman K, Kriston L, Rücker G, Jamil S, Meissner K, et al. Effectiveness of psychological treatments for depressive disorders in primary care: Systematic review and meta-analysis. Annals of Family Medicine 2015;13(1):56-68. [CrossRef]
    4. Cipriani A, Furukawa TA, Salanti G, Chaimani A, Atkinson LZ, Ogawa Y, et al. Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. The Lancet 2018 Apr;391(10128):1357-1366. [CrossRef]
    5. Cuijpers P, Dekker J, Hollon SD, Andersson G. Adding psychotherapy to pharmacotherapy in the treatment of depressive disorders in adults: a meta-analysis. J Clin Psychiatry 2009 Sep;70(9):1219-1229. [CrossRef] [Medline]
    6. Cuijpers P, Andersson G, Donker T, van Straten A. Psychological treatment of depression: results of a series of meta-analyses. Nord J Psychiatry 2011 Dec;65(6):354-364. [CrossRef] [Medline]
    7. Bedi N, Chilvers C, Churchill R, Dewey M, Duggan C, Fielding K, et al. Assessing effectiveness of treatment of depression in primary care. Partially randomised preference trial. Br J Psychiatry 2018 Jan 2;177(04):312-318. [CrossRef]
    8. Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bulletin of the World Health Organisation 2004;82(11):858-866. [CrossRef]
    9. Mohr DC, Ho J, Duffecy J, Baron KG, Lehman KA, Jin L, et al. Perceived barriers to psychological treatments and their relationship to depression. J Clin Psychol 2010 Apr;66(4):394-409 [FREE Full text] [CrossRef] [Medline]
    10. Chen L, Crum RM, Martins SS, Kaufmann CN, Strain EC, Mojtabai R. Service use and barriers to mental health care among adults with major depression and comorbid substance dependence. Psychiatr Serv 2013 Sep 01;64(9):863-870 [FREE Full text] [CrossRef] [Medline]
    11. Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, et al. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry 2017 Dec;210(2):119-124 [FREE Full text] [CrossRef] [Medline]
    12. Gonzalez T, Chiodo C. ICD 10: A Primer for the Orthopedic Foot and Ankle Surgeon. Foot Ankle Int 2015 Sep;36(9):1110-1116. [CrossRef] [Medline]
    13. National Collaborating Centre for Mental Health (UK). Depression: The Treatment and Management of Depression in Adults (Updated Edition). Leicester (UK): British Psychological Society; 2010.
    14. Bower P, Kontopantelis E, Sutton A, Kendrick T, Richards DA, Gilbody S, et al. Influence of initial severity of depression on effectiveness of low intensity interventions: meta-analysis of individual patient data. BMJ 2013 Feb 26;346(feb26 2):f540-f540. [CrossRef] [Medline]
    15. Mohr DC, Vella L, Hart S, Heckman T, Simon G. The Effect of Telephone-Administered Psychotherapy on Symptoms of Depression and Attrition: A Meta-Analysis. Clin Psychol (New York) 2008;15(3):243-253 [FREE Full text] [CrossRef] [Medline]
    16. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther 2009 Jun;38(2):66-82. [CrossRef] [Medline]
    17. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
    18. de Graaf LE, Gerhards SAH, Arntz A, Riper H, Metsemakers JFM, Evers SMAA, et al. Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry 2009 Jul;195(1):73-80 [FREE Full text] [CrossRef] [Medline]
    19. Meyer B, Bierbrodt J, Schröder J, Berger T, Beevers CG, Weiss M, et al. Effects of an Internet intervention (Deprexis) on severe depression symptoms: Randomized controlled trial. Internet Interventions 2015 Mar;2(1):48-59. [CrossRef]
    20. Clarke G, Reid E, Eubanks D, O'Connor E, DeBar LL, Kelleher C, et al. Overcoming depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program. J Med Internet Res 2002 Dec;4(3):E14 [FREE Full text] [CrossRef] [Medline]
    21. Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, DeBar LL, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005 Jun;7(2):e16 [FREE Full text] [CrossRef] [Medline]
    22. Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C. Randomized effectiveness trial of an Internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther 2009 May;38(4):222-234 [FREE Full text] [CrossRef] [Medline]
    23. O'Kearney R, Kang K, Christensen H, Griffiths K. A controlled trial of a school-based Internet program for reducing depressive symptoms in adolescent girls. Depress Anxiety 2009 Jan;26(1):65-72. [CrossRef] [Medline]
    24. Moritz S, Schilling L, Hauschildt M, Schröder J, Treszl A. A randomized controlled trial of internet-based therapy in depression. Behav Res Ther 2012 Aug;50(7-8):513-521. [CrossRef] [Medline]
    25. Johansson R, Sjöberg E, Sjögren M, Johnsson E, Carlbring P, Andersson T, et al. Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS One 2012 May;7(5):e36905 [FREE Full text] [CrossRef] [Medline]
    26. Hawton K, Casañas i Comabella C, Haw C, Saunders K. Risk factors for suicide in individuals with depression: a systematic review. J Affect Disord 2013 May;147(1-3):17-28. [CrossRef] [Medline]
    27. Richards D, Timulak L, Doherty G, Sharry J, Colla A, Joyce C, et al. Internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression: protocol for a randomized controlled trial. BMC Psychiatry 2014 May 21;14:147 [FREE Full text] [CrossRef] [Medline]
    28. Richards D, Timulak L, O'Brien E, Hayes C, Vigano N, Sharry J, et al. A randomized controlled trial of an internet-delivered treatment: Its potential as a low-intensity community intervention for adults with symptoms of depression. Behaviour Research and Therapy 2015 Dec;75:20-31. [CrossRef] [Medline]
    29. Watts S, Newby JM, Mewton L, Andrews G. A clinical audit of changes in suicide ideas with internet treatment for depression. BMJ Open 2012 Sep;2(5) [FREE Full text] [CrossRef] [Medline]
    30. Beck AT, Steer RA, Brown GK. BDI-II, Beck Depression Inventory: Manual. San Antonio, TX: Psychological Corporation; 1996.
    31. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, Fourth Edition, Text Revision (DSM-IV-TR). Washington, DC: American Psychiatric Association; 2000.
    32. Dozois DJA, Dobson KS, Ahnberg JL. A psychometric evaluation of the Beck Depression Inventory-II. Psychological Assessment 1998;10(2):83-89. [CrossRef]
    33. Steer RA, Rissmiller DJ, Beck AT. Use of the Beck Depression Inventory-II with depressed geriatric inpatients. Behaviour Research and Therapy 2000 Mar;38(3):311-318. [CrossRef]
    34. Beck AT, Steer RA, Carbin MG. Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review 1988 Jan;8(1):77-100. [CrossRef]
    35. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
    36. Kroenke K, Spitzer RL, Williams JBW, Löwe B. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. Gen Hosp Psychiatry 2010 Jul;32(4):345-359. [CrossRef] [Medline]
    37. Clark DM, Layard R, Smithies R, Richards DA, Suckling R, Wright B. Improving access to psychological therapy: Initial evaluation of two UK demonstration sites. Behav Res Ther 2009 Nov;47(11):910-920 [FREE Full text] [CrossRef] [Medline]
    38. Richards DA, Suckling R. Improving access to psychological therapies: phase IV prospective cohort study. Br J Clin Psychol 2009 Nov;48(Pt 4):377-396. [CrossRef] [Medline]
    39. Mundt JC, Marks IM, Shear MK, Greist JM. The Work and Social Adjustment Scale: a simple measure of impairment in functioning. Br J Psychiatry 2018 Jan 2;180(05):461-464. [CrossRef]
    40. Buuren SV, Groothuis-Oudshoorn K. : Multivariate Imputation by Chained Equations in. J. Stat. Soft 2011;45(3). [CrossRef]
    41. Bates D, Mächler M, Bolker B, Walker S. Fitting Linear Mixed-Effects Models Using. J Stat Soft 2015;67(1). [CrossRef]
    42. Kuznetsova A, Brockhoff PB, Christensen RHB. Package: Tests in Linear Mixed Effects Models. J. Stat. Soft 2017;82(13). [CrossRef]
    43. Boriani G, Biffi M, Martignani C. Painless shock therapy: more than just an attempt to square the circle? J Cardiovasc Electrophysiol 2007 Sep;18(10):1101-1103. [CrossRef] [Medline]
    44. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol 2013 Nov 26;4:863 [FREE Full text] [CrossRef] [Medline]
    45. Gibbons RD, Hedeker DR, Davis JM. Estimation of Effect Size from a Series of Experiments Involving Paired Comparisons. Journal of Educational Statistics 1993;18(3):271. [CrossRef]
    46. Seggar LB, Lambert MJ, Hansen NB. Assessing clinical significance: Application to the beck depression inventory. Behavior Therapy 2002;33(2):253-269. [CrossRef]
    47. Galea S, Tracy M. Participation Rates in Epidemiologic Studies. Annals of Epidemiology 2007 Sep;17(9):643-653. [CrossRef]
    48. Putnam RD. Tuning In, Tuning Out: The Strange Disappearance of Social Capital in America. PS: Political Science and Politics 1995 Dec;28(4):664. [CrossRef]
    49. Bak H. Education and Public Attitudes toward Science: Implications for the &quot;Deficit Model&quot; of Education and Support for Science and Technology. Social Science Q 2001 Dec;82(4):779-795. [CrossRef]
    50. Kessler RC, Gruber M, Hettema JM, Hwang I, Sampson N, Yonkers KA. Co-morbid major depression and generalized anxiety disorders in the National Comorbidity Survey follow-up. Psychol Med 2008 Mar;38(3):365-374 [FREE Full text] [CrossRef] [Medline]
    51. Kolovos S, Kleiboer A, Cuijpers P. Effect of psychotherapy for depression on quality of life: meta-analysis. Br J Psychiatry 2016 Dec;209(6):460-468. [CrossRef] [Medline]
    52. Papakostas G, Petersen T, Mahal Y, Mischoulon D, Nierenberg AA, Fava M. Quality of life assessments in major depressive disorder: a review of the literature. General Hospital Psychiatry 2004;26(1):13-17. [Medline]
    53. Hadjistavropoulos HD, Pugh NE, Nugent MM, Hesser H, Andersson G, Ivanov M, et al. Therapist-assisted Internet-delivered cognitive behavior therapy for depression and anxiety: translating evidence into clinical practice. J Anxiety Disord 2014 Dec;28(8):884-893 [FREE Full text] [CrossRef] [Medline]
    54. Proudfoot J. Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. The British Journal of Psychiatry 2004 Jul 01;185(1):46-54. [CrossRef]
    55. Cavanagh K, Shapiro DA, Van Den Berg S, Swain S, Barkham M, Proudfoot J. The effectiveness of computerized cognitive behavioural therapy in routine care. Br J Clin Psychol 2006 Nov;45(Pt 4):499-514. [CrossRef] [Medline]
    56. Bauer S, Lambert MJ, Nielsen SL. Clinical significance methods: a comparison of statistical techniques. J Pers Assess 2004 Feb;82(1):60-70. [CrossRef] [Medline]
    57. van Spijker BAJ, Majo MC, Smit F, van Straten A, Kerkhof AJFM. Reducing suicidal ideation: cost-effectiveness analysis of a randomized controlled trial of unguided web-based self-help. J Med Internet Res 2012 Oct 26;14(5):e141 [FREE Full text] [CrossRef] [Medline]
    58. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of Self-guided Internet-Based Cognitive Behavioral Therapy in the Treatment of Depressive Symptoms: A Meta-analysis of Individual Participant Data. JAMA Psychiatry 2017 Apr 01;74(4):351-359. [CrossRef] [Medline]
    59. Wiles N, Thomas L, Abel A, Barnes M, Carroll F, Ridgway N, et al. Clinical effectiveness and cost-effectiveness of cognitive behavioural therapy as an adjunct to pharmacotherapy for treatment-resistant depression in primary care: the CoBalT randomised controlled trial. Health Technol Assess 2014 May;18(31):1-167, vii [FREE Full text] [CrossRef] [Medline]


    ANOVA: analysis of variance
    BDI-II: Beck Depression Inventory
    CBT: cognitive behavioral therapy
    DSM-IV: The American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition)
    GAD-7: Generalized Anxiety Disorder-7 item inventory
    iCBT: internet-delivered cognitive behavioral therapy
    NICE: The National Institute for Health and Care Excellence
    RCT: randomized controlled trial
    WSAS: Work and Social Adjustment Scale

    Edited by J Torous; submitted 23.02.18; peer-reviewed by L Farrer, N Forand, I Choi; comments to author 11.03.18; revised version received 12.06.18; accepted 28.08.18; published 02.10.18

    ©Derek Richards, Daniel Duffy, John Burke, Melissa Anderson, Sarah Connell, Ladislav Timulak. Originally published in JMIR Mental Health (, 02.10.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.