Reducing Depression Through an Online Intervention: Benefits From a User Perspective

Introduction

Growing numbers of people are turning to the Internet as a preferred method for obtaining health-related information, or as an option for accessing therapeutic interventions [1]. In the area of mental health, Internet-based interventions for depression have undergone significant development [2]. Importantly, as use and service options have expanded, research has demonstrated the efficacy of online programs in reducing depressive symptoms [3-10], improving depression literacy [11,12], and decreasing stigma [11-13]. However, while increased interest in online participation appears indicative of the acceptance of Internet-based interventions, the benefits of these programs from the perspective of the participants is often not reported.

Consumer satisfaction is integral in assessing the quality of any health service delivery. While the effectiveness of interventions in reducing symptoms is critical, the experience and satisfaction of participants has important implications for treatment outcomes, continuation of treatment (adherence), and re-connection with a service at a latter point of need [14,15]. Consumer satisfaction reports are increasingly being used to assess the quality of mental health services [14,16-19]. Some research has also investigated the relationship between consumer satisfaction and objective mental health outcome measures [20]. However, to date, consumer feedback has primarily been utilized in the development and evaluation of community-based mental health services [21].

Some researchers have investigated consumer satisfaction with e-mental health programs. Proudfoot and colleagues [22] investigated community attitudes and the acceptability of mobile phones for self-monitoring of symptom severity and obtaining self-help for depression, anxiety, and stress, and obtained positive feedback. High levels of general satisfaction with Internet-based treatment has also been reported in the context of randomized controlled trials (RCTs) conducted for various mental health conditions [23-25]. One study [25] investigating an Internet based intervention for depression found that, on average, participants reported feeling very confident that the “treatment would be successful at teaching them techniques for managing their symptoms”; and a high level of confidence in “recommending the treatment to a friend with depression” (p21). However, largely this is an area within e-mental health services research that has not undergone detailed investigation.

Evaluations of the WellBeing trial [26,27] found that exposure to a 12-week automated training program comprising psycho-education and cognitive behavioral and interpersonal therapy (e-couch) used both alone and in combination with an Internet support group (ISG) produced significant reductions in depressive symptoms at 6- and 12-month follow-ups [27]. In addition, the automated training program was associated with immediate improvements in self-esteem and empowerment relative to control participants, and when combined with the ISG, 6-month improvements in perceived quality of life [28]. However, the subjective experience of the intervention participants in the WellBeing trial was not reported.

The aims of the present study are to (1) explore the level of consumer confidence in online e-mental health programs both prior to and following participation in the trial; (2) investigate the benefits and changes in behavior reported by participants following participation in the trial; and (3) identify any problems encountered by participants. Further, given that publicly available online mental health (depression) interventions are most likely to be accessed by individuals self-identifying as being depressed, comparisons were made between those reporting that they currently suffer from depression (at baseline) with those that did not.

Methods

This study was approved by The Australian National University Human Research Ethics Committee (Protocol 2007/2259) and the WellBeing trial described in this paper was registered with the Controlled Clinical Trials registry (ISRCTN65657330). The current paper contains only a brief description of the methodology specific to the present study as the complete WellBeing trial protocol has been published previously [26,27].

Participants

The study comprised 298 adults aged 18 to 65 years recruited to the WellBeing trial between August 2008 and May 2009. Participants were recruited through a screening survey posted to 70,000 adults randomly selected from the electoral rolls of 8 Australian electoral divisions (4 rural and 4 metropolitan). Participants were informed that the trial was designed to investigate “the usefulness of self-help Internet programs for improving emotional well-being and preventing or reducing the symptoms of depression” [26]. To be eligible for the trial participants required a Kessler Psychological Distress (K10) score greater than 22, and access to the Internet. Respondents were excluded if they were currently (1) receiving treatment from a mental health professional or participating in a mutual support group; (2) participating in another research project at the lead investigator’s (KG) research center; and (3) reported current or past experience with or diagnosis of psychosis, schizophrenia, or bipolar disorder.

Procedure

The study employed a longitudinal RCT design. Data pertaining to participant confidence in online interventions and satisfaction were collected via self-report questionnaires administered at baseline (one week prior to commencement of the intervention), and at the conclusion of the 12-week intervention period.

Interventions

Participants were randomly allocated to receive one of four conditions: an ISG, which utilized a moderated bulletin board format to facilitate discussions between participants on topics primarily related to depression; a depression Internet training program (e-couch) comprising a depression literacy component and online versions of cognitive behavior therapy, interpersonal therapy, applied relaxation, and physical activity programs; a combination of the two (e-couch and ISG); or an attention control website (HealthWatch). HealthWatch comprised 12 online modules, each containing the following components: (1) a series of questions participants were asked to consider on topics potentially related to depression and well-being (eg, nutrition), and (2) online health information about a topic that may be related to well-being but did not address depression specifically (eg, environmental health). The content of each intervention has been described in further detail elsewhere [26,27].

Randomization was conducted by the trial statistician using a stratified block design procedure, with a fixed block size of 4. Stratification variables were level of psychological distress, gender, age, and location of residence [27]. The number of participants completing the baseline questionnaire and commencing the intervention following randomization was as follows: ISG (n=77), e-couch (n=74), e-couch and ISG (n=73), and HealthWatch (n=74).

Measures

This study explores participant confidence in the effectiveness of online interventions (collected prior to and after the intervention), and problems encountered, self-reported benefits, and behavior change following the intervention.

Intervention Benefits

Immediately following the intervention trial period, participants were asked to respond to 14 items reporting the extent to which the website helped them in areas such as “being productive at work” and “reducing the emotional pain they were experiencing”; and the extent to which the website helped them to “discuss subjects that I felt unable to discuss before”, “feel encouraged and supported emotionally”, “feel less isolated and lonely”, “feel proud of myself for helping others”, “learn more about depression and its treatment”, or “seek professional help for my depression”. Items were developed for the purpose of the study based on items developed in a UK study of depression ISG users [29], and the Consumer Reports Effectiveness Scale (CRES-4) [17,30]. Participants responded to each statement on a 5-point scale from 1(strongly agree/made things a lot better) to 5(strongly disagree/made things a lot worse). The scale was then dichotomized as 0 (neither/no difference, disagree/made things somewhat worse, strongly disagree/made things a lot worse), 1 (agree/made things somewhat better, strongly agree/ made things a lot better) to assess the proportion of participants reporting benefit in each area.

User Confidence

At each assessment point participants rated their confidence that a website could both “help people understand depression better” and “help people learn skills for preventing depression”. Participants responded 0 (No) or 1 (Yes) to each statement.

Behavior Change

Participant self-reported behavior change was assessed across the sample using 4 items. Participants responded 0 (No) or 1(Yes) to indicate if they had “given advice about depression to someone else”, “sought help from a health professional”, “sought more information”, or “tried a self-help treatment”.

Problems Encountered

Following the intervention trial period, participants were asked to indicate if they had encountered any problems as a result of using the website or participating in the trial. Responses were recorded as 0 (No) or 1 (Yes). Items were developed for the purpose of the study. The survey was tailored to the types of problems that may have been encountered in each trial arm. Items in the survey of participants exposed to the HealthWatch (control) or e-couch (including combined condition) websites were “Feeling bored”, “Feeling frustrated”, “Feeling more anxious”, and “Finding the program too impersonal”. Items received by ISG participants (including combined condition) were “Feeling annoyed or upset by the comments made by other members on the board”, “Feeling frustrated that I could not meet other members of the board in person”, “Feeling upset that I couldn't help other board members more”, and “Feeling very anxious about other members on the board”.

Statistical Analysis

All primary comparisons were conducted using chi-square analyses. Post-hoc comparisons identifying significant differences between the four intervention groups were based on the standardized residual for each cell. Values greater than ±1.96 indicate a difference at a P value of .05, and values greater than ±2.58 indicate a difference at a P value of .01.

Results

The numbers of participants in each condition at each wave of data collection (baseline and post-test) are shown in Table 1. At post-test, drop-out was significantly lower in the HealthWatch control condition compared to the other conditions. Few significant differences were found between completers and non-completers on measures of depression or demographic characteristics [27].

Confidence in Online Interventions

Across the sample it appeared that confidence in a website as a tool to improve depression literacy was high (Table 2). At baseline (prior to interacting with the website), 83.6% (249/298) of participants were confident that a website could help people understand depression better. Confidence in the ability of a website to help people learn skills for preventing depression was less certain across the sample at baseline, with only 48.3% (144/298) of people making the endorsement.

Following the intervention period this indicator of confidence remained relatively stable with no significant change in the proportion of respondents making these endorsements. No significant differences were found based on intervention group. There were significant differences in the responses of participants who did and did not report depression at baseline. At baseline, participants reporting current depression were significantly less likely than those not reporting depression to be confident that a website could either improve people’s understanding of depression (P=.019) or teach people skills for preventing it (P=.026). However, this significant difference was not maintained following the intervention. Following the intervention participants were asked to report on the usefulness of the website. Of the 226 respondents to the question, 76.5% (173/226) of people indicated they found the website useful or very useful. There were no significant differences between any of the conditions, or on the basis of self-reported depression at baseline (Table 2).

User-Reported Benefits

The most frequently endorsed benefit of participating in the trial by participants across the sample was one of increasing depression literacy (Table 3). Across the total sample, 76.2% (170/223) of respondents indicated that the website helped them to “learn more about depression and its treatment”. In particular, this benefit was strongly endorsed by participants in the e-couch and combined e-couch and ISG condition; participants in the ISG condition were significantly less likely to report that the website helped them to learn more about depression and its treatment when compared to the e-couch and combined ISG in combination with e-couch conditions (P<.05). No significant differences were found between those participants self-reporting current depression at baseline and those who did not.

When asked how much they felt that the website had helped them in areas such as coping with stress and reducing symptoms of depression, the benefit endorsed most by participants was that of promoting personal growth and understanding, reported by 57.8% (125/216) of respondents (Table 3). Comparing benefits obtained across the different intervention groups it was found that the e-couch and combined e-couch and ISG condition participants were again significantly more likely to report that the website helped reduce symptoms of depression and increase productivity at work, the ability to cope with everyday stress, and enjoy life more, compared to the control and ISG alone conditions (significant at P<.05). Compared to the control condition, participants in the e-couch and combined condition were more likely to report that the website helped their ability to relate to others, and reduced their emotional pain (P<.05). The e-couch participants were also more likely to report that the website helped with self-esteem and confidence compared to the control group (significant at P<.05). Again, no significant differences were found between those participants self-reporting current depression at baseline and those who did not. See Multimedia Appendix 1 for each comparison.

When asked if they had done something different (eg, sought more information or treatment) as a result of the website, approximately 47.8% (109/228) of respondents indicated they had (Table 4). This was significantly higher amongst participants in the e-couch and combined e-couch and ISG conditions. Respondents in both conditions were more likely to report have done something different because of the website compared to the control and ISG conditions (P<.01). Specifically, when asked what they had done, participants in both the e-couch and the combined condition were more likely to have tried a self-help treatment compared to control and ISG conditions (significant at P<.01), as was the combined condition (significant at P<.05).

Overall, those reporting depression at baseline were not more likely than their counterparts not reporting depression to have done something different because of participation in the program or website. However, when asked about what they had done, participants self-reporting current depression at baseline who had taken action were significantly more likely to have sought help from a professional (P=.022), and were more likely to have sought more information (P=.055). For tables representing each comparison, see Multimedia Appendices 1-3.

User Problems

The investigation of concerns or problems encountered by participants indicated few stressors resulted from participation in the interventions (Table 5). When asked about problems resulting from the use of the e-couch or control program, 27.6% (45/162) of respondents overall reported finding the program too impersonal. While no significant difference was found between conditions, the highest endorsement figure was for the HealthWatch condition which used impersonal content, followed by e-couch and the conditions involving an ISG. Problems of “Feeling frustrated and bored” were endorsed by less than 20% of the sample; 11% (18/165) of the participants reported feeling anxious as a result of using a program. Examining differences between conditions, respondents in the combined condition were more likely to report feeling more anxious or feeling frustrated compared to the control condition (significant at P<.05).

When asked about problems resulting from participation in the ISG, 29% (27/91) of respondents overall reported feeling upset that they could not help the other members of the ISG more, and 17% (16/92) reported frustration that they could not meet the other members of the ISG in person (anonymity was a rule governing participation in the ISG).

No significant differences in concerns or problems encountered were found between those participants self-reporting current depression at baseline and those who did not (see Multimedia Appendices 1-3 for tables presenting each comparison). Few participants reported being upset or annoyed by the comments of others in the ISG.

Discussion

To our knowledge, this is the most comprehensive quantitative investigation of consumer perspectives on Internet-based depression interventions. Examining confidence and satisfaction with the online services, the benefits and changes in behavior reported by participants following participation in the trial, and problems encountered by participants, the study provides evidence to support the benefits of online psycho-education and cognitive behavior therapy programs from a consumer perspective. Overall, participants reported a high level of confidence in the ability of the online interventions to improve people’s understanding of depression, and following participation in the program a clear majority reported the interventions were useful. Participants engaged in the e-couch intervention indicated that the website helped them to learn more about depression and its treatment and helped reduce symptoms of depression, increase productivity at work, improve their ability to cope with everyday stress, and enjoy life, compared to the control and ISG conditions. A minority of participants reported concerns or problems resulting from participation in the interventions.

Confidence in Online Interventions

The successful development and implementation of online interventions for mental health conditions, such as depression, depend on consumers’ confidence in the program and willingness to engage in this alternative treatment. As such, the present study first examined participant confidence that a website could help people understand and learn skills for preventing depression. Results found that overall confidence in the ability of a website to increase understanding of depression was high; however, only 61.3% (144/235) of participants at baseline reported confidence in a website (online intervention) as a tool to help people learn skills for preventing depression. The high level of confidence in the ability of a website to increase knowledge or literacy, particularly at baseline, is perhaps unsurprising given that the Internet is largely an information resource and the participants of this study had signed up for an online intervention trial. However, the comparatively lower confidence in websites as prevention strategies or tools is of importance since such perceptions may serve as a barrier to the use of the Internet for prevention programs for depression. Moreover, participants reporting current depression were significantly less confident in the ability of a website to either improve people’s understanding of depression or teach people skills for preventing it, compared to those not reporting depression. Despite this scepticism, the willingness of participants to participate in the study (promoted as a trial designed to improve well-being) and to try new interventions is positive. Importantly, the differences (albeit small effects) in confidence that a website could help people understand depression or learn skills for preventing depression identified at baseline between participants self-reporting depression and those that did not, were not found following the intervention. While no significant increase was found across the sample in overall confidence in the ability of an online intervention, the results reflect small increases in confidence after having completed the intervention amongst those participants reporting current depression at baseline. Expectations are likely to play a significant role in whether a consumer seeks help from an online program. Accordingly, the continued monitoring of the acceptability and perception of e-mental health initiatives and programs should be central to the continued expansion and implementation of e-mental health services. This monitoring also should include items which determine consumer perspectives on the effectiveness of online programs for treating depression as well as preventing it. It is possible that perceived treatment effectiveness would be higher than perceived preventive effectiveness among consumers. However, this requires further consideration.

Reported Benefits of Participation

Participants reported benefits and behavior changes as a result of their participation in the trial. An increase in depression literacy was reported by over half of participants, with this benefit most strongly endorsed by those engaged with e-couch (either alone or in combination with the ISG). This is consistent with the fact that e-couch incorporates a specific depression literacy and educational component and might therefore reasonably be expected to deliver greater improvements in literacy than either an ISG or the control condition. One question of interest is whether self-reported improvements in depression literacy should be greater among the ISG participants than the control group given that depression ISGs are often used as a means of communicating information between members [31]. Here they were not. In fact, a surprising large percentage of the HealthWatch control participants perceived an improvement in their knowledge about depression and treatments after completing the condition. This may be explained by the fact that in the quiz sections of the HealthWatch condition participants were asked to think about the role of different lifestyle issues in depression (eg, humor in depression). Participants may have interpreted this as information about depression and its treatment. Further, the ISG used was an experimental group that was established for the purpose of the research trial. Research is required to investigate the perceived benefits of well-established Internet groups with large numbers of participants including those with strong depression literacy.

Participants allocated to e-couch (more than other intervention groups) reported the website as helping to reduce symptoms of depression, increase productivity at work, improve ability to cope with everyday stress and enjoy life more. Further, they reported an improved ability to relate to others and reduced emotional pain compared to control participants. These findings provide corroboration that e-couch is a helpful online intervention for depression. Specifically, these reports complement the initial evaluation of the WellBeing trial that found a significant objective reduction in depressive symptoms at 6 and 12 months following the combined e-couch and ISG intervention [27]. Moreover, it provides support that there are additional benefits from participating in such an automated training program [28]. The lack of significant difference in findings between participants self-identifying as currently depressed versus those that did not is important. It suggests that these benefits may be conferred despite a lack of individual awareness and self-identification of depression. Finally, while it may be anticipated that participation in an ISG should offer stronger perceived benefits for loneliness and isolation, and improved ability to relate to others and cope with everyday stress this was not evidenced in the present study. However, further research examining established ISGs is needed to provide a more ecologically valid consumer evaluation of the benefits.

Perceived Difficulties

Difficulties or problems were reported by some participants. The primary concern of those participating in the automated training program (e-couch) was that it was too impersonal. Although this represented a minority of participants (27.3%, 45/162) and the intervention aims to incorporate engaging graphical and written content, the findings suggests the need to focus on innovative strategies to increase the personalization of automated online interventions. Participants engaged with the ISG reported that they were upset and anxious that they could not help other members more. This may stem in part from rules of the board which required anonymity and are not a characteristic of all support groups. However, it may also be an intrinsic aspect of online support groups where participants lack face-to-face contact, or the asynchronous nature of the bulletin board format. Further, the concerns about not being able to assist more may arise regardless of the communication format (face-to-face or online). A peer supporter may not have the resources to solve all the problems of their peer with depression; this may be an inherent disadvantage of participation in ISGs for some potential participants.

Limitations and Conclusions

Complementary to other more objective outcome evaluations [8-10,27], this study reports the experience and outcomes of this type of online intervention from the perspective of the participants. However, the findings must be considered in the context of several limitations. The analyses did not include participants who failed to provide follow-up data. They may have been less satisfied and less positive than those who were retained in the trial.

In addition, items used to assess the problems encountered and user confidence was adapted for the purpose of the study and as such has not been externally validated. While responses were accepted based on how the participant interpreted the meaning of the items, it is possible that individuals may differ in their understanding of terms such as prevention. Moreover, while tailoring the types of problems participants were asked about to the different interventions reduced the number of perceived irrelevant items being completed by participants, this approach offers limitations in that it makes comparisons between interventions difficult. Further research should consider assessing more generic issues that may be associated with all online interventions.

While further investigation of consumer reports across online mental health interventions is required, the results of the present study offer an important consumer perspective on program effectiveness which complements existing objective and empirical evaluations [27,28]. This enables a more complete understanding of the experiences of participants which may assist in the development and evaluation of future programs.