Published on 10.04.17 in Vol 4, No 2 (2017): Apr-Jun
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/6809, first published Oct 14, 2016.
A Mixed-Methods Study Using a Nonclinical Sample to Measure Feasibility of Ostrich Community: A Web-Based Cognitive Behavioral Therapy Program for Individuals With Debt and Associated Stress
Background: There are increasing concerns about the health and well-being of individuals facing financial troubles. For instance, in the United Kingdom, the relationship between debt and mental health difficulties is becoming more evident due to the economic downturn and welfare reform. Access to debt counseling services is limited and individuals may be reluctant to access services due to stigma. In addition, most of these services may not be appropriately resourced to address the psychological impact of debt. This study describes outcomes from an Internet-based cognitive behavioral therapy (ICBT) program, Ostrich Community (OC), which was developed to provide support to those struggling with debt and associated psychological distress.
Objective: The aim of this feasibility study was to assess the suitability and acceptability of the OC program in a nonclinical sample and examine mental health and well-being outcomes from using the program.
Methods: A total of 15 participants (who were not suffering from severe financial difficulty) were assisted in working through the 8-week ICBT program. Participants rated usability and satisfaction with the program, and after completion 7 participants took part in a semistructured interview to provide further feedback. Before the first session and after the final session all participants completed questionnaires to measure well-being and levels of depression, stress, and anxiety and pre- and postscores were compared.
Results: Satisfaction was high and themes emerging from the interviews indicate that the program has the potential to promote effective financial behaviors and improve financial and global psychosocial well-being. When postcompletion scores were compared with those taken before the program, significant improvements were identified on psychometric measures of well-being, stress, and anxiety.
Conclusions: The OC program is the first ICBT program that targets poor mental health associated with financial difficulty. This feasibility study indicates that OC may be an effective intervention for increasing financial resilience, supporting individuals to become financially independent, and promoting positive financial and global well-being. Further work with individuals suffering from debt and associated emotional difficulties will help to examine clinical effectiveness more closely.
JMIR Ment Health 2017;4(2):e12
- cognitive behavioral therapy;
- computer-assisted therapy;
- psychological stress;
- economic recession;
- mental health
Internet-Based Cognitive Behavioral Therapy
Cognitive behavioral therapy (CBT) [, ] is an effective treatment for depression and anxiety recommended by the National Institute for Health and Care Excellence (NICE). It is based on the assumption that most behavioral and emotional reactions are learnt and focuses on the interrelated components of emotions, behaviors, and thoughts [ ]. A key advantage of CBT is that it adapts to computerization [ ], with Internet-based cognitive behavioral therapy (ICBT) programs tailored to the needs of the individual. There are a range of ICBT programs adapted for different health issues, psychological disorders, and lifestyle choices such as substance misuse [ - ], insomnia [ , ], mild to moderate depression [ , ], pathological gambling [ ], and perfectionist-related issues [ ]. This feasibility study explores the acceptability and usability of an ICBT program that has been designed for people experiencing mental health problems due to difficult financial circumstances.
In the midst of a recession and with the continuing move by the UK government to implement austerity measures, vulnerable members of society are suffering the consequences of an economic contraction . One of the adverse outcomes of the recession is the so-called “credit crunch” whereby individuals are struggling to cope with debt and the consequences of this on mental health and well-being [ - ]. Individuals in financial distress (especially those experiencing unemployment and impoverishment) manifest a range of illnesses including depression, anxiety, and alcohol use disorders [ , ], with greater austerity measures increasing the severity of such mental health issues [ , ]. Indeed, a survey by Rethink [ ] revealed that almost 9 out of 10 of those in debt felt that their financial difficulties had made their mental health problems worse.
Mental Health Problems and Debt
A further negative consequence of the imposed austerity measures is the continuing erosion of access to mental health services [, ]. Financial institutions, such as the Citizens Advice Bureau, are not able to support those experiencing financial difficulties and associated mental health problems [ ]. Due to limited access to appropriate support and intervention, individuals may therefore be less able to resolve their financial problems or access help, and subsequently, may find themselves sinking deeper into debt.
In light of the detrimental effects that debt may have on well-being, and the lack of evidence-based services that offer both financial and emotional advice, government and financial institutions now recognize that considerable work needs to be done to address this gap in services [, ]. However, although the optimal treatment for debt and mental health is still being sought, there is good reason to assume that ICBT can be adapted to the specific issues associated with the burden of debt. ICBT may be appealing in that it may reduce concerns an individual has about stigma associated with seeking face-to-face help for debt and mental health issues [ ]. Indeed, most of those who experience financial difficulty also experience feelings of shame, self-disgust, and secretiveness [ ]. This means that individuals in debt can suffer in silence when confidential services that assure anonymity are lacking [ ].
Therefore, this study was conducted to assess the usability and acceptability of a novel ICBT treatment program, Ostrich Community (OC), which is designed to help individuals with their financial difficulties while supporting them to overcome any associated mental health problems. The program aims to promote development of positive behaviors, increase financial knowledge, and improve self-efficacy with regards to financial management. The study adopted a mixed-methods approach as recommended by the Medical Research Council (MRC) guidance around the development and evaluation of complex, multicomponent psychosocial interventions [, ]. This MRC guidance recommends that before examining clinical effectiveness of complex interventions via approaches such as randomized controlled trials (RCTs), feasibility and piloting work should be conducted in order to ascertain initial acceptability and usability of any novel intervention. This may be particularly important when developing digital interventions that can be perceived as “disruptive” to the status quo within an existing health or social care system [ , ].
In line with the MRC guidance, it was important to conduct a feasibility study to explore acceptability and usability in a group of participants experiencing only moderate financial difficulties, before examining effectiveness in the target population, that is, service users accessing support for financial and mental health difficulties. This approach is important to gain insights into improvements that could be made to the program before evaluating it more formally with a group who may have more complex needs . Small-scale feasibility studies can also reveal insights into acceptability of novel treatment approaches, which can be used to facilitate engagement during formal piloting and effectiveness studies [ - ].
This was a mixed-methods feasibility study, incorporating semistructured qualitative interviews and quantitative satisfaction measures, alongside psychometric measures of stress and well-being. These measures were taken to explore acceptability and usability of the OC program.
Description of the OC Program
The OC ICBT program provides practical advice about management of finances and psychological well-being and clinically validated evidence-based psychosocial intervention strategies that are grounded in the theoretical underpinnings of the CBT model . Each module provides psychoeducation, cognitive-behavioral worksheets or strategies, and money management stories and quotes to help increase motivation. The program moves beyond information-giving and is the very first intervention program that is evidence-based and tailored to address issues surrounding financial and associated psychological distress.
The program comprises a 2-min introductory video, followed by 8 Web-based sessions at weekly intervals. Each session lasts approximately 30 min, with approximately 30 additional min per week spent practicing away from the computer (eg, keeping problem diaries, thought records, and completing behavioral experiments) in order to encourage the user to practice the skills taught throughout the program. The program contains information delivered by video and audio to enhance engagement and accessibility. Seefor more detail of the content of individual modules and topics covered by the program.
The OC program is best suited to users who are experiencing mild to moderate stress or depression and anxiety symptoms, and who need information and guidance about money management. Participants with more serious mental health symptoms and debt issues are introduced to content on the home page and there is a toolbox that provides information on more intensive mental health and debt advice services that are more appropriate to their needs.
|Module||Topic||Summary of activities|
|1||What are stress, fear, and anxiety?||Provides information to give awareness and understanding of stress and how it can be prevented and managed. The module introduces the purpose of the program (ie, to support users to develop skills to cope with financial stress), and helps the user to understand the links between financial difficulties and stress ().|
|2||The Ostrich model— therapeutic interventions||Introduces cognitive behavioral therapy and self-help techniques to help the user understand why they feel as they do, and learn strategies for changing the way they think, feel, and act. It uses videos, text, and exercises that focus on generic knowledge and skills to help the individual apply these methods in their own lives, with a specific focus on applying these skills to coping with difficult financial situations.|
|3||Thinking patterns||Provides information, videos, and exercises to help the user identify and challenge negative thoughts and learn to think in a more accurate and realistic way toward their financial situation (). Uses cognitive restructuring and other cognitive change techniques to encourage accurate appraisal of financial difficulties.|
|4||Problem solving||Builds on the cognitive techniques provided in session 3 to support users to utilize their new, realistic insights into their financial situation to solve identified difficulties. The problem solving approaches are underpinned by a goal-setting strategy, to encourage the user to set realistic, achievable goals. Mind mapping exercises are used to identify possible solutions to barriers to goal attainment. Promotes help seeking behaviors (seeking debt advice, talking to creditors).|
|5||Incorporating positive actions||Helps to identify unhelpful and unhealthy behaviors, particularly around finances, and provides guidance on changing these into healthier behaviors. Provides techniques to increase coping skills and effective communication skills to assist the user in talking to other agencies about their debt, for example. This session also includes completion of an activity monitoring and scheduling planner, to enable the user to organize days and times that they will carry out problem-solving activities related to their debt.|
|6||Taking action||Provides practical information and resources directly related to financial issues, such as budgeting exercises, and guides the user through a step-by-step exercise to become more financially competent.|
|7||Stress management and relaxation methods||Provides more general exercises, information, and videos of wider stress management techniques, including relaxation training, progressive muscle relaxation, visualization, and gentle exercise, to help promote relaxation and enhance ability to cope with financial stress.|
|8||Recap||This module recaps on prior learning and information covered in the program to help consolidate and reinforce learning and the practical skills taught.|
The sample consisted of 17 students from the University of Salford, who were not in significant financial distress, but were coping with the demands of a further education course of study, alongside the low income and financial difficulties many students experience. This group was selected as it was considered important to explore acceptability and usability of the program in a group without significant financial and emotional difficulties, before testing individuals with more complex needs that are facing real financial and emotional difficulties. All participants were studying full-time and some were also working to supplement their income.provides demographic information on the sample.
|Demographic characteristics||n (%)|
|Age range (years)|
|Employment status (alongside being a student)|
|Working (full-time)||4 (24)|
|Working (part-time)||3 (18)|
|Student only (no full- or part-time work)||10 (59)|
|Home owner||3 (18)|
|Renting a property||9 (53)|
|Living with parents (not paying rent)||5 (29)|
Participants who consented to take part subscribed to the program by activating a Web-based account, providing an email address, and creating a password. They received a welcome message together with login details via email. Of the 17 consenting participants, 15 activated an account.
Participants worked through the modules sequentially, with a new module released each week for 8 weeks together with an email reminder. Following completion of all modules, participants were invited to take part in a semistructured interview exploring acceptability and usability of the program (7 took part). Before module 1 and after module 8, participants completed two standardized psychometric assessments, measuring stress and well-being (with a total of 13 participants providing these data). The flowchart depicted indemonstrates the sequence of events during the study, from initial recruitment to completion of qualitative interviews, and shows attrition rates at each stage.
Three separate methods were used to examine (1) acceptability and usability of the program, (2) participant satisfaction, and (3) stress and well-being outcomes.
Feasibility Study: Qualitative Interviews to Assess Acceptability and Usability
On completion of the program, participants were offered the opportunity to take part in an interview. Interviews were conducted on a one-to-one basis with the lead author (DS) and explored participants’ views of the OC program. Open-ended questions allowed participants to elaborate on their answers. Interviews lasted approximately 30 min and were recorded using a digital Dictaphone.
The interviews were transcribed and entered into NVivo qualitative data analysis software version 10 (QSR International Pty, Ltd), and thematic analysis (TA) was used to analyze the transcripts. Published guidelines around the process of TA in psychological research were followed , with transcripts initially being read by the lead author (DS) to identify quotes relevant to the study aims. Connections were made between quotes and were used to develop overarching themes, which were examined and refined to ensure that they accurately described themes common to all transcripts. These were reviewed by the second author (SE), to increase reliability, and any discrepancies were discussed until an agreement was reached. The themes were named and defined to ensure that they were discrete.
Feasibility Study: Quantitative Measures to Assess Satisfaction
This stage of the study measured satisfaction with the program using the 8-item Client Satisfaction Questionnaire (CSQ-8) . The CSQ-8 total score ranges from 8 to 32, with higher scores indicating higher satisfaction. Participants completed this at the end of every session and at the end of the program. A short questionnaire developed by the lead author was also used to gain additional information around satisfaction and usability specific to the OC program, and this was completed after each module. This was a 5-item questionnaire with a Likert scale ranging from 0 (“not at all”) to 7 (“very much”) and participants were asked to state how much they agreed with statements such as “How useful did you find the new topic introduced today?” Data were analyzed using Microsoft Excel and Statistical Package for the Social Sciences (SPSS).
Clinical Outcomes: Quantitative Psychometric Measures of Stress and Well-Being
Participants completed the Warwick and Edinburgh Mental Well-Being Scale (WEMWBS)  and the short version of the Depression Anxiety Stress Scale (DASS-21) [ ] to monitor any changes in well-being and stress as a result of engaging with the program. The WEMWBS is a scale of 14 positively worded items for assessing mental well-being. Each statement asks for a response on a Likert scale that ranges from 1 (“None of the time”) to 5 (“All of the time”). The scores on the questionnaire range from 14 to 70 with higher scores indicating greater well-being. The DASS is a 21-item self-report questionnaire designed to measure the severity of a range of symptoms common to depression, anxiety, and stress with 7 items for each scale. Each item is scored on a 4-point scale from 0 (“Did not apply to me at all”) to 3 (“Applied to me very much or most of the time”) indicating how often the respondent experienced each state in the past week. The questionnaire provides 3 scores (one for each state) with higher scores indicating that respondents had suffered from a particular state in the last week.
In order to determine stress and well-being outcomes, pre- and postintervention scores on the DASS  and WEMWBS [ ] were subjected to statistical analyses. Nonparametric Wilcoxon Signed-Ranks tests were used to examine changes in scores at the beginning and end of the program.
As three methods were used to address three independent aims, findings from each of these are reported separately.
Feasibility Study: Qualitative Interviews to Assess Acceptability and Usability
Three themes were identified relating to (1) enhancing engagement with the OC program, (2) acceptability of program content, and (3) the programs’ potential to benefit the target population.
Theme 1: Enhancing Engagement
Participants reported a number of features of the program that enhanced engagement, including presentation of modules in a sequential and progressive manner with each one building on knowledge gained from the previous session:
it was organized in a way that you can do it, in a step-by-step process.
Participants also liked that the program content was delivered via interactive Web-based multimedia including video and audio, although one participant did suggest that a purely Web-based program may not be suitable for everyone:
I think...the way the website was very visual, and you did a blend of different ways (to) suit people’s needs in terms of their learning style (but) I think it needs to be maybe classroom based...there’s a tutor or someone taking them through it bit by bit then taking them back to apply it.
Theme 2: Acceptability and Accessibility of Content
In general, participants were positive about the content of the program, and while there was acknowledgment from participants that the content was informed by academic theory, this did not make it difficult to access or understand:
I liked that there was a good balance between the theory, the information provided, and activities...it was nothing too strenuous.
Positive reinforcement within the program was also perceived as helpful and played an important part in supporting users to achieve the goals they set:
I like the positive quotes and encouraging words that were throughout the program, I liked how rewarding it was...which made you feel like you’ve done some progress.
Theme 3: Potential to Benefit the Target Population
On the whole, participants reported that they understood the concept of the program, even though a Web-based financial capability program was a new idea:
it was a new concept that I haven’t explored in other programs. It’s a financial capability program so its specialized and it’s good that its specialized to help people with financial difficulties in that sense.
The opinion among participants was that the program offered practical advice and solutions to financial difficulties and would support individuals to develop the tools necessary to overcome their financial difficulties:
it was practical, there was so many things you provided in the tool box, which is a good thing because the program didn’t just talk about stuff it actually provided the tools to help people.
Feasibility Study: Quantitative Measures to Assess Satisfaction
All 13 participants completed the usability questionnaire after each module and the CSQ-8  at the end of the program. Data from the usability questionnaire ( ) demonstrated mean experience across the 8 sessions was favorable, with participants rating the usefulness of each topic and the benefits of each session as high. Following each session participants felt less stressed compared with how they felt at the beginning, with none reporting distress from any of the sessions.
|How stressed were you at the beginning of the session?||3.63 (0.74)|
|How useful did you find the new topic introduced today?||6.13 (0.64)|
|How relaxed do you feel right now?||5.88 (0.35)|
|How beneficial did you find the session?||6 (0.53)|
|How tense do you feel right now?||2.25 (0.46)|
aSD: standard deviation.
From the CSQ-8 data (), 92% (12/13) of participants rated the quality as good or excellent. All participants received the service they wanted, and 92% (12/13) reported that the program met most or all of their needs and that they were satisfied with the amount of help provided. All would recommend the program to a friend in need of financial help and 92% (12/13) said that they would come back to the program if they needed help in the future. When asked how satisfied they were with the program overall, 69% (9/13) were “very satisfied” and 30% (4/13) were “mostly satisfied.”
|How would you rate the quality of service you have received?||Excellent||6 (46)|
|Did you get the kind of service you wanted?||Yes, definitely||8 (62)|
|Yes, generally||5 (38)|
|No, not really||0 (0)|
|No, definitely not||0 (0)|
|To what extent has the program met your needs?||Almost all of my needs have been met||7 (54)|
|Most of my needs have been met||5 (38)|
|Only a few of my needs have been met||1 (8)|
|None of my needs have been met||0 (0)|
|If a friend were in need of similar help would you recommend the program?||Yes, definitely||9 (69)|
|Yes, I think so||4 (31)|
|No, I don’t think so||0 (0)|
|No, definitely not||0 (0)|
|How satisfied are you with the amount of help you have received?||Very satisfied||11 (84)|
|Mostly satisfied||1 (8)|
|Indifferent or mildly dissatisfied||1 (8)|
|Quite dissatisfied||0 (0)|
|Have the services you have received helped you to deal more effectively with your problems?||Yes, they helped a great deal||6 (46)|
|Yes, they helped||6 (46)|
|No, they seemed to make things worse||0 (0)|
|No, they really did not||1 (8)|
|In an overall, general sense, how satisfied are you with the service you have received?||Very satisfied||9 (69)|
|Mostly satisfied||4 (31)|
|Indifferent or mildly dissatisfied||0 (0)|
|Quite dissatisfied||0 (0)|
|If you were to seek help in the future would you come back to the program?||Yes, definitely||8 (62)|
|Yes, I think so||4 (31)|
|No, I do not think so||1 (8)|
|No, definitely not||0 (0)|
Clinical Outcomes: Quantitative Measures of Stress and Well-Being
Participants reported higher levels of anxiety before the program (median 24) than at the end (median 16) and this difference was significant, Z (13)=−2.006, P<.05 There was also a significant difference in stress scores before and after (median scores of 32 and 26 respectively) with participants reporting lower levels of stress at the end of the program, Z (13)=−2.814, P<.01. There was no difference between levels of depression at the end of the program (median 22) compared with the beginning (median 26), Z (13)=−0.393, P=.69.
Scores on the WEMWBS were lower at the beginning of the program (median 45.00) compared with the end (median 51). Participants reported a significant increase in well-being from pre- to postprogram, Z (13)=−2.275, P<.05.
The continuing government austerity measures, coupled with a bleaker outlook on jobs, have resulted in more people experiencing debt . This often takes a toll on a person’s health and psychological well-being [ ] and is exacerbated by a shortage of services available to assist people experiencing financial hardship and associated mental health difficulties. Financial and other governmental institutions have limited resources to assist people in this situation and the waiting list of people needing help is expanding [ ].
The OC program is an innovative ICBT program that aims to support individuals who are having financial difficulties that are affecting their emotional well-being. The focus of the program is to relieve psychological stress, anxiety, and depression while teaching positive financial behaviors and resilience. This initial feasibility study was conducted to explore acceptability and usability of the program.
Many participants said that they appreciated taking part in the study because it helped them to learn how to budget effectively and how to be prepared in terms of dealing with sudden financial changes. One of the benefits of this program is therefore that individuals can use it as a preventative tool regardless of their financial circumstances and in the absence of significant levels of debt [, ]. Indeed, participants noted that the CBT principles in the program could be beneficial for anyone, even for those without any significant financial or psychological difficulties. In addition, after completing the program, participants reported a significant reduction in anxiety and stress, and improvements in well-being.
Participants found the program to be helpful and easy to use and there was high satisfaction with the financial information and emotional help provided. These findings support the high satisfaction with ICBT when addressing gambling , insomnia [ ], mild-to-moderate depression [ ], and drug and alcohol use [ , ]. Participants liked the interactive and multimedia, digital format of the program, reporting that they found it to be user-friendly, and the information easy to understand. Some observed that although the program allows for people to work independently, it may be too much for some who may benefit from “computer-assisted therapy” (CAT) rather than “self-help therapy.”
CAT integrates therapeutic content delivered by an ICBT program with a human practitioner who may provide screening, supervision, and other support to the user , whereas self-help programs do not use practitioner monitoring or involvement [ ]. Evidence suggests that people with higher levels of motivation, such as students, have improved outcomes from participation in this latter type of program. Students are generally driven by a desire to achieve [ ] and are encouraged to be self-motivated autonomous learners [ ]. This is a drawback with this sample as participants may have been more motivated than those in the general population suffering with debt problems. This is supported by the relatively low dropout rate compared with other studies of ICBT programs, some of which report high attrition rates [ ]. In addition, as the students who participated in this study were not necessarily in debt, and instead were coping with a relatively low income that is common among students, the sample is not representative of the target user groups.
A “blended” approach to facilitation of ICBT programs has been demonstrated to be helpful for those individuals experiencing mental health difficulties . Therefore, further work will explore delivery of the program not as self-help, but as CAT. By delivering the program as CAT with practitioner support, this may lessen the attrition rate in the target population, and may provide vital additional therapeutic support for individuals experiencing significant financial hardship and mental health difficulties [ ].
This study intended to explore initial feasibility and acceptability of the OC program with a small cohort of participants experiencing some financial difficulties. This was to gain insights into improvements that could be made to the program before evaluating it with a population of service users with more complex needs. Multiple digital health studies take this approach, including studies of mental health intervention programs  and alcohol misuse programs [ ]. The sample size was limited, yet when examining feasibility of novel digital interventions such as the OC program, small-scale studies can reveal important insights regarding acceptability and clinical content. This can facilitate optimal engagement during formal piloting and effectiveness studies [ - ]. Future work will include a larger pilot study with the target population, that is, service users accessing support for their financial and mental health difficulties.
The aims of this feasibility study were to assess the acceptability and usability of a novel ICBT program for people experiencing poor mental health due to financial difficulties. A total of 15 participants worked through the program over a period of 8 weeks, and they reported satisfaction with the program and noted that it was easy to use. They also suggested that it made them consider their financial situation and gave them skills they could use beyond the immediate focus of finances. There was also a significant improvement in well-being, anxiety, and stress on completion of the program.
Some participants reported that it was difficult to work through the modules individually and so future work will explore the addition of support from a practitioner and potentially a moderated peer support forum. These implementation approaches may help to increase impact , increase engagement, and support completion. The sample was also limited to participants not in serious financial difficulties. Therefore, further research will include a larger sample of participants who come from the intended targeted population for the OC program; individuals experiencing significant financial difficulties and associated mental health difficulties.
DS created the OC program, conceived and designed the study, analyzed the qualitative data, and drafted the manuscript. SE analyzed the qualitative data and drafted the manuscript. LDM provided advice on study design and ethical approval. CT designed the study, analyzed the quantitative data, and drafted the manuscript.
Conflicts of Interest
- Beck AT. Cognitive therapy: past, present, and future. J Consult Clin Psychol 1993 Apr;61(2):194-198. [Medline]
- Beck JS. Cognitive Behavioural Therapy: Basics and Beyond, 2nd edition. New York: The Guilford Press; Aug 18, 2011.
- Williams C, Garland A. A cognitive–behavioural therapy assessment model for use in everyday clinical practice. Adv Psychiatr Treat 2002;8(3):172-179. [CrossRef]
- Peck DF. The therapist-client relationship, computerized self-help and active therapy ingredients. Clin Psychol Psychother 2010;17(2):147-153. [CrossRef] [Medline]
- Elison S, Davies G, Ward J. An outcomes evaluation of computerized treatment for problem drinking using breaking free online. Alcohol Treat Q 2015 May 05;33(2):185-196. [CrossRef]
- Elison S, Davies G, Ward J. Effectiveness of computer-assisted therapy for substance dependence using breaking free online: subgroup analyses of a heterogeneous sample of service users. JMIR Mental Health 2015 Apr 23;2(2):e13. [CrossRef] [Medline]
- Carroll K, Ball S, Martino S, Nich C, Babuscio T, Nuro K, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry 2008 Jul;165(7):881-888 [FREE Full text] [CrossRef] [Medline]
- Kay-Lambkin F, Baker A, Lewin T, Carr VJ. Computer-based psychological treatment for comorbid depression and problematic alcohol and/or cannabis use: a randomized controlled trial of clinical efficacy. Addiction 2009 Mar;104(3):378-388. [Medline]
- Espie C, Hames P, McKinstry B. Use of the internet and mobile media for delivery of cognitive behavioral insomnia therapy. Sleep Med Clin 2013 Sep;8(3):407-419. [CrossRef]
- Espie C, Kyle S, Miller C, Ong J, Hames P. Attribution, cognition and psychopathology in persistent insomnia disorder: outcome and mediation analysis from a randomized placebo-controlled trial of online cognitive behavioural therapy. Sleep Med 2014 Mar 12;15(8):913-917. [Medline]
- Richards D, Richards A, Barkham M, Cahill J. PHASE: a 'health technology' approach to psychological treatment in primary mental health care. Prim Health Care Res Dev 2002;3:159-168. [CrossRef]
- Williams AL, Merten MJ. A review of online social networking profiles by adolescents: implications for future research and intervention. Adolescence 2008;43(170):253-274. [Medline]
- Carlbring P, Smit F. Randomized trial of internet-delivered self-help with telephone support for pathological gamblers. J Consult Clin Psychol 2008 Dec;76(6):1090-1094. [CrossRef] [Medline]
- Radhu N, Daskalakis Z, Arpin-Cribbie C, Irvine J, Ritvo P. Evaluating a Web-based cognitive-behavioral therapy for maladaptive perfectionism in university students. J Am Coll Health 2012;60(5):357-366. [CrossRef] [Medline]
- Step Change. 2015. An action plan on problem debt URL: https://www.stepchange.org/policy-and-research/problem-debt-action-plan.aspx [accessed 2017-03-24] [WebCite Cache]
- Dooley D, Catalano R, Wilson G. Depression and unemployment: panel findings from the epidemiologic catchment area study. Am J Community Psychol 1994 Dec;22(6):745-765. [Medline]
- Ford E, Clark C, McManus S, Harris J, Jenkins R, Bebbington P, et al. Common mental disorders, unemployment and welfare benefits in England. Public Health 2010 Dec;124(12):675-681. [CrossRef] [Medline]
- Jefferis B, Nazareth I, Marston L, Moreno-Kustner B, Bellón J, Svab I, et al. Associations between unemployment and major depressive disordervidence from an international, prospective study (the predict cohort). Soc Sci Med 2011;73(11):1627-1634. [CrossRef] [Medline]
- McKee-Ryan F, Song Z, Wanberg C, Kinicki AJ. Psychological and physical well-being during unemployment: a meta-analytic study. J Appl Psychol 2005 Jan;90(1):53-76. [CrossRef] [Medline]
- Compton W, Gfroerer J, Conway K, Finger MS. Unemployment and substance outcomes in the United States 2002-2010. Drug Alcohol Depend 2014 Sep 01;142:350-353 [FREE Full text] [CrossRef] [Medline]
- Murphy R, Zemore S, Mulia N. Housing instability and alcohol problems during the 2007–2009 US recession: the moderating role of perceived family support. J Urban Health 2013 Jul 30;91(1):17-32. [CrossRef]
- Marmot M. Institute of Health Equity. 2010. Fair society, healthy lives: the Marmot review URL: http://www.instituteofhealthequity.org/projects/fair-society-healthy-lives-the-marmot-review [accessed 2017-03-15] [WebCite Cache]
- Wahlbeck K, McDaid D. Actions to alleviate the mental health impact of the economic crisis. World Psychiatry 2012;11(3):139-145. [Medline]
- Rethink Mental Illness. 2014. Lost generation: why young people with psychosis are being left behind, and what needs to change URL: https://www.rethink.org/media/973932/LOST%20GENERATION%20-%20Rethink%20Mental%20Illness%20report.pdf [accessed 2017-03-15] [WebCite Cache]
- Royal College of Nursing. 2014 Oct. Frontline first: turning back the clock? RCN report on mental health services in the UK URL: https://www2.rcn.org.uk/__data/assets/pdf_file/0004/600628/004772.pdf [accessed 2017-03-15] [WebCite Cache]
- Money Advice Liaison Group. URL: http://www.malg.org.uk/debt-and-mental-health/ [accessed 2017-03-15] [WebCite Cache]
- Financial Conduct Authority. 2014 Apr 01. Consumer credit and consumers in vulnerable circumstances URL: https://www.fca.org.uk/publications/research/consumer-credit-and-consumers-vulnerable-circumstances [accessed 2017-03-15] [WebCite Cache]
- Department of Health, UK. 2011 Feb 02. No Health Without Mental Health: a cross-Government mental health outcomes strategy for people of all ages - a call to action URL: https://www.gov.uk/government/publications/no-health-without-mental-health-a-cross-government-mental-health-outcomes-strategy-for-people-of-all-ages-a-call-to-action [accessed 2017-03-15] [WebCite Cache]
- Department of Health, UK. 2011 Feb 02. No health without mental health: a cross-government outcomes strategy URL: https://www.gov.uk/government/publications/no-health-without-mental-health-a-cross-government-outcomes-strategy [accessed 2017-03-15] [WebCite Cache]
- Fitch C, Hamilton S, Bassett P, Davey R. The relationship between personal debt and mental health: a systematic review. Ment Health Rev J 2011 Dec 09;16(4):153-166. [CrossRef]
- Zetterqvist K, Maanmies J, Ström L, Andersson G. Randomized controlled trial of internet-based stress management. Cogn Behav Ther 2003;32(3):151-160. [CrossRef] [Medline]
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008 Sep 29;337:a1655 [FREE Full text] [CrossRef] [Medline]
- Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015 Mar 19;350:h1258. [CrossRef]
- Krebs P, Prochaska J, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med 2010;51(3-4):214-221 [FREE Full text] [CrossRef] [Medline]
- Hwang J, Christensen CM. Disruptive innovation in health care delivery: a framework for business-model innovation. Health Aff (Millwood) 2008;27(5):1329-1335 [FREE Full text] [CrossRef] [Medline]
- Fraeyman J, Van RP, Vriesacker B, De ML, Van HG. How is an electronic screening and brief intervention tool on alcohol use received in a student population? A qualitative and quantitative evaluation. J Med Internet Res 2012 Apr 23;14(2):e56 [FREE Full text] [CrossRef] [Medline]
- Ammerlaan J, Scholtus W, Drossaert H, van OH, Prakken B, Kruize A, et al. Feasibility of a website and a hospital-based online portal for young adults with juvenile idiopathic arthritis: views and experiences of patients. JMIR Res Protoc 2015 Aug 14;4(3):e102 [FREE Full text] [CrossRef] [Medline]
- Steele GC, Gill A, Khan I, Hans K, Kuluski K, Cott C. The electronic patient reported outcome tool: testing usability and feasibility of a mobile app and portal to support care for patients with complex chronic disease and disability in primary care settings. JMIR Mhealth Uhealth 2016 Jun 02;4(2):e58 [FREE Full text] [CrossRef] [Medline]
- Babson A, Ramo E, Baldini L, Vandrey R, Bonn-Miller MO. Mobile app-delivered cognitive behavioral therapy for insomnia: feasibility and initial efficacy among veterans with cannabis use disorders. JMIR Res Protoc 2015 Jul 17;4(3):e87 [FREE Full text] [CrossRef] [Medline]
- Schrader G, Bidargaddi N, Harris M, Newman L, Lynn S, Peterson L, et al. an eHealth intervention for patients in rural areas: preliminary findings from a pilot feasibility study. JMIR Res Protoc 2014 Jun 12;3(2):e27 [FREE Full text] [CrossRef] [Medline]
- Braun V, Clarke V. Qualitative Research in Psychology. 2006 Jan 21. Using thematic analysis in psychology URL: http://eprints.uwe.ac.uk/11735/2/thematic_analysis_revised_-_final.pdf [accessed 2016-11-02] [WebCite Cache]
- Larsen D, Attkisson C, Hargreaves W, Nguyen TD. Assessment of client/patient satisfaction: Development of a general scale. Eval Program Plann 1979 Jan;2(3):197-207. [CrossRef]
- Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes 2007 Nov 27;5:63 [FREE Full text] [CrossRef] [Medline]
- Lovibond PF, Lovibond SH. Manual for the Depression Anxiety Stress Scales. Sydney: Psychology Foundation; 1995.
- We Need to Talk. Mind. 2015 Sep. An urgent need: We Need to Talk’s manifesto for better talking therapies for all URL: https://www.mind.org.uk/media/1178860/we-need-to-talk-briefing-september-2014.pdf [WebCite Cache]
- National Institute for Health and Care Excellence. Computerised cognitive behaviour therapy for depression and anxiety URL: https://www.nice.org.uk/guidance/ta97 [WebCite Cache]
- Wright J, Wright A, Albano A, Basco M, Goldsmith L, Raffield T, et al. 2005 Jun;162(6):1158-1164. [CrossRef] [Medline]
- Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
- Sass E. Motivation in the College Classroom: What Students Tell Us. Teaching of Psychology 1989 Apr 01;16(2):86-88 [FREE Full text] [CrossRef]
- Lowman J. Mastering the techniques of teaching. San Francisco: Jossey-Bass; 1995.
- Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011 Aug 05;13(3):e52 [FREE Full text] [CrossRef] [Medline]
- Kooistra L, Wiersma J, Ruwaard J, van Oppen P, Smit F, Lokkerbol J, et al. Blended vs face-to-face cognitive behavioral treatment for major depression in specialized mental health care: study protocol of a randomized controlled cost-effectiveness trial. BMC Psychiatry 2014 Oct 18;14. [CrossRef]
- Farrer L, Gulliver A, Chan J, Bennett K, Griffiths KM. A Virtual Mental Health Clinic for University Students: A Qualitative Study of End-User Service Needs and Priorities. JMIR Ment Health 2015;2(1):e2 [FREE Full text] [CrossRef] [Medline]
|CAT: computer-assisted therapy|
|CBT: cognitive behavioral therapy|
|CSQ: Client Satisfaction Questionnaire|
|DASS: Depression Anxiety Stress Scale|
|ICBT: Internet-based cognitive behavioral therapy|
|MRC: Medical Research Council|
|NHS: National Health Service|
|NICE: National Institute for Health and Care Excellence|
|OC: Ostrich Community|
|RCT: randomized controlled trial|
|SPSS: Statistical Package for the Social Sciences|
|TA: thematic analysis|
|WEMWBS: Warwick and Edinburgh Mental Well-Being Scale|
Edited by J Torous; submitted 14.10.16; peer-reviewed by K Easton, G Mourad; comments to author 16.11.16; revised version received 21.12.16; accepted 29.01.17; published 10.04.17
©Dawn Smail, Sarah Elison, Linda Dubrow-Marshall, Catherine Thompson. Originally published in JMIR Mental Health (http://mental.jmir.org), 10.04.2017.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.