Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 17.07.20 in Vol 7, No 7 (2020): July

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/14536, first published Aug 25, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    A Digital Intervention for Adolescent Depression (MoodHwb): Mixed Methods Feasibility Evaluation

    1Division of Psychological Medicine and Clinical Neurosciences, Medical Research Council Centre for Neuropsychiatric Genetics and Genomics, Cardiff University, Cardiff, Wales, United Kingdom

    2National Centre for Mental Health, Cardiff University, Cardiff, Wales, United Kingdom

    3Cwm Taf Morgannwg University Health Board, Wales, United Kingdom

    4Population Health Sciences, University of Bristol, Bristol, England, United Kingdom

    5Institute of Health and Wellbeing, University of Glasgow, Glasgow, Scotland, United Kingdom

    6Faculty of Medical and Health Sciences, School of Medicine, University of Auckland, Auckland, New Zealand

    7Department for Health, University of Bath, Bath, England, United Kingdom

    8Medical Research Council/Chief Scientist Office Social and Public Health Sciences Unit, Institute of Health and Wellbeing, University of Glasgow, Glasgow, Scotland, United Kingdom

    Corresponding Author:

    Rhys Bevan Jones, PhD, MRCPsych

    Division of Psychological Medicine and Clinical Neurosciences

    Medical Research Council Centre for Neuropsychiatric Genetics and Genomics

    Cardiff University

    Hadyn Ellis Building

    Maindy Road

    Cardiff, Wales, CF24 4HQ

    United Kingdom

    Phone: 44 02920688451

    Email: bevanjonesr1@cardiff.ac.uk


    ABSTRACT

    Background: Treatment and prevention guidelines highlight the key role of health information and evidence-based psychosocial interventions for adolescent depression. Digital health technologies and psychoeducational interventions have been recommended to help engage young people and to provide accurate health information, enhance self-management skills, and promote social support. However, few digital psychoeducational interventions for adolescent depression have been robustly developed and evaluated in line with research guidance.

    Objective: We aimed to evaluate the feasibility, acceptability, and potential impact of a theory-informed, co-designed digital intervention program, MoodHwb.

    Methods: We used a mixed methods (quantitative and qualitative) approach to evaluate the program and the assessment process. Adolescents with or at elevated risk of depression and their parents and carers were recruited from mental health services, school counselors and nurses, and participants from a previous study. They completed a range of questionnaires before and after the program (related to the feasibility and acceptability of the program and evaluation process, and changes in mood, knowledge, attitudes, and behavior), and their Web usage was monitored. A subsample was also interviewed. A focus group was conducted with professionals from health, education, social, and youth services and charities. Interview and focus group transcripts were analyzed using thematic analysis with NVivo 10 (QSR International Pty Ltd).

    Results: A total of 44 young people and 31 parents or carers were recruited, of which 36 (82%) young people and 21 (68%) parents or carers completed follow-up questionnaires. In all, 19 young people and 12 parents or carers were interviewed. Overall, 13 professionals from a range of disciplines participated in the focus group. The key themes from the interviews and groups related to the design features, sections and content, and integration and context of the program in the young person’s life. Overall, the participants found the intervention engaging, clear, user-friendly, and comprehensive, and stated that it could be integrated into existing services. Young people found the “Self help” section and “Mood monitor” particularly helpful. The findings provided initial support for the intervention program theory, for example, depression literacy improved after using the intervention (difference in mean literacy score: 1.7, 95% CI 0.8 to 2.6; P<.001 for young people; 1.3, 95% CI 0.4 to 2.2; P=.006 for parents and carers).

    Conclusions: Findings from this early stage evaluation suggest that MoodHwb and the assessment process were feasible and acceptable, and that the intervention has the potential to be helpful for young people, families and carers as an early intervention program in health, education, social, and youth services and charities. A randomized controlled trial is needed to further evaluate the digital program.

    JMIR Ment Health 2020;7(7):e14536

    doi:10.2196/14536

    KEYWORDS



    Introduction

    Prevention and Management of Adolescent Depression

    Adolescent depression is common and is associated with social and educational impairments, deliberate self-harm, and suicide. It can also mark the beginning of long-term mental health difficulties [1]. Early treatment and prevention of adolescent depression is therefore a major public health concern [2]. However, many adolescents with depression do not access interventions, and engaging young people in prevention and early intervention programs is challenging [1,3,4].

    Guidelines for the prevention and management of depression in young people [5,6] stress the need for good information and evidence-based psychosocial interventions for the young person, family, and carer. There has been growing interest in psychoeducational interventions, which deliver accurate information to individuals, families, and carers about mental health or a specific diagnosis, management and prognosis, and relapse prevention strategies [6-8]. The American Academy of Child and Adolescent Psychiatry (2007) “Practice parameter for the assessment and treatment of children and adolescents with depressive disorders” describes psychoeducation as the “education of family members and the patient about the causes, symptoms, course, and different treatments of depression and the risks associated with these treatments as well as no treatment at all. Education should make the treatment and decision-making process transparent and should enlist parent and patient as collaborators in their own care.” [6] Although the risk factors and possible causes of adolescent depression are complex, a family history of depression, psychosocial stress, and a previous history of depression increase individual risk, and these groups could be targeted for such strategies [9].

    Findings from a systematic review concluded that psychoeducational interventions were effective in improving the clinical course, treatment adherence, and psychosocial functioning of adults with depression [10]. However, a systematic review of such interventions for adolescent depression [11] showed that there were few existing programs. This is an important gap in the literature because depression is common in young people, and its presentation and management are different from those of adults [1].

    Digital Mental Health

    Digital health or electronic health (eHealth) has been identified as a key area of future clinical practice and research in adolescent depression, especially to improve reach and access to therapies at relatively low cost [8,12]. There is evidence to support the use of some digital interventions for adolescent depression, and they have been recommended in treatment and prevention guidelines [5,13-16]. However, to our knowledge, there is no digital psychoeducational intervention that has been co-designed and specifically developed for adolescents with depression, or those at elevated risk, or developed and evaluated in line with key guidance of digital and complex interventions [17-20].

    Here, we describe an early stage evaluation of MoodHwb, a digital psychosocial intervention co-designed with, and for young people with, or at elevated risk of depression, and their families and carers [21,22]. The main aim of this analysis was to examine whether the program and assessments were feasible and acceptable. This is a crucial step to support the refinement of the prototype and design of a randomized controlled trial [20]. This work could also help to inform the development and evaluation of other digital technologies in this field.


    Methods

    The Digital Intervention: MoodHwb

    MoodHwb was designed to engage young people (and families and carers) by using developmentally appropriate language, illustrations, animations, and interactive components. It also aims to promote self help, help-seeking where appropriate, and social support. Although it was founded primarily on psychoeducation, it also includes elements of cognitive behavioral therapy, positive psychology, and interpersonal, family systems and behavioral change theories. It is multi-platform and is available as an app, which includes the interactive components and a mobile-friendly version of the main site.

    The program was co-designed using a series of interviews and focus groups with potential users: young people (with depressive symptoms or at high risk), parents, carers, and professionals from health, education, social, and youth services and charities. It was also informed by the systematic review noted earlier [11]; person-centered guidelines for developing digital interventions [17-19]; design, educational, and psychological theory; a logic model; and consultations with a digital media company and clinical/research experts.

    The program includes sections on mood and depression, possible reasons for low mood and depression, self-management (including planning, problem solving, and lifestyle approaches), finding help, and other issues commonly experienced alongside depression (eg, anxiety; Figures 1 and 2). There is also a section for families, carers, friends, and professionals, in part to promote social support.

    On the welcome screen, there are separate user pathways for the young person (“I’m here for myself”) and for the parent, carer or another person concerned about a young person (“I’m here for someone else;” Figure 1). The user is then asked questions (eg, regarding their mood and anxiety), and the answers (1) are stored in the “My profile” section, and (2) help to signpost the relevant subsections on the subsequent dashboard screen. Along with this “Mood monitor,” there are other interactive elements: a goal setting element (“My goals”) and a section to save links to helpful resources (“Stuff I like;” Figure 3).

    MoodHwb is available in English and Welsh (hwb is the Welsh translation for hub, and can also mean a lift or boost—HwbHwyliau is the Welsh name for the program). The program and the design and development process are described further by Bevan Jones et al [21,22].

    Figure 1. MoodHwb welcome screen (main image/left) and open menu (right).
    View this figure
    Figure 2. Examples of subsections from the “Possible reasons (for low mood)” and “Where to get help” sections.
    View this figure
    Figure 3. Interactive elements: mood monitor and diary (left); “Stuff I like” (top center); goal setting (bottom center); dashboard/profile section (right).
    View this figure

    Study Design

    A mixed methods (quantitative and qualitative) approach was taken to assess the feasibility, acceptability, and potential effectiveness of the program. Data were collected from the following: (1) semistructured interviews with young people, parents and carers who used the intervention, (2) a focus group with professionals, (3) Web usage data, and (4) pre intervention and post intervention questionnaires.

    Ethical and Health Board Approval

    The Dyfed-Powys (Wales, the United Kingdom) and Cardiff University School of Medicine Research Ethics Committees gave a favorable opinion for the research project. Research and development approval was granted by Cwm Taf, Cardiff and Vale, Abertawe Bro Morgannwg, Aneurin Bevan, Hywel Dda, and Powys University Health Boards (UHBs), 6 of the 7 UHBs in Wales. The authors of the questionnaires were contacted where approval was required.

    Participants

    We planned to approach around 40 young people, as this sample size was considered sufficient to allow assessment of acceptability and feasibility in a relatively broad sample of the target group. Young people were recruited from specialist Child and Adolescent Mental Health Services (CAMHS), primary mental health teams, school counselors and nurses, and from the Cardiff Early Prediction of Adolescent Depression (EPAD) study [23]. Individuals (recruited from CAMHS and EPAD) who participated in the design and development phase of MoodHwb were also invited to participate.

    Eligibility Criteria

    Young people had to be at least 13 years of age, and to have either (1) a current or past history of depression or (2) be at elevated risk of depression due to a family history of recurrent depression in a parent. The parents and carers of these young people were also invited to take part. Participants were not eligible if they were unable to understand the intervention or questions/discussions (eg, because of insufficient understanding of English).

    Professionals from health, education, social, and youth services and charities were approached if they worked with young people with mental health difficulties.

    Recruitment and Consent

    Eligible young people were identified by their CAMHS and primary mental health practitioners and school counselors and nurses, and provided with letters and information sheets. Individuals from the EPAD study (who had consented to being contacted about other research) were sent a letter and information sheet inviting them to participate. Interested young people and their parents or carers sent a completed reply card to the study team, who then telephoned the young person and parent or carer to discuss the project further. Participants were given the choice of meeting the study team to discuss the project and complete the consent forms in person, or to do this via post. Signed consent was obtained from young people above the age of 16 years, and signed parent and guardian consent and child assent from those under 16 years. Professionals from the appropriate services were identified and sent a letter and information sheet, inviting them to participate, with a reply card as above.

    Procedures

    Consenting participants were provided a link to the program via email, and they created their own passwords. Parents and carers could create separate accounts to their children. Participants were informed that they could use the program as they wished (MoodHwb includes a section and animation to introduce the program to the user). Young people continued to attend sessions with their practitioner or counselor, if they were seeing someone. They were given access for a minimum of 2 months to allow them sufficient time to work through the program. Professionals were provided with a link to the program 2 weeks before their focus group.

    Interviews and Focus Group
    Semistructured Interviews With Young People, Parents, and Carers

    Young people were asked whether they would like to be interviewed alone or with a parent or carer. The parent or carer was also asked whether they would like to be interviewed separately. Interviews were held either at Cardiff University or a location convenient for the participant (eg, home and school) and lasted up to 90 min. All interviews were completed by RBJ. The interviews were informal, and interviewees had access to MoodHwb on a mobile device.

    Sampling Frame

    Adolescents were selected for interview based on their usage of MoodHwb (ie, frequent or occasional), age, gender, service or charity involvement, and primary language (English, Welsh). The range of characteristics and number of interviews completed ensured a broad spread of viewpoints.

    Focus Group With Professionals

    The group session was held at Cardiff University and was facilitated by RBJ and a colleague. The session lasted approximately 120 min. The program was projected on a screen during the group session. We aimed for a balance of professionals in terms of services and charities, gender, and primary language (English, Welsh).

    The interviews and focus group were all audio recorded digitally and transcribed; participants could also write or draw their thoughts.

    Web Usage Data

    Individual user interactions with the website were captured from bespoke analytics developed by the digital media company.

    Pre Questionnaires and Post Questionnaires

    Adolescent participants and their parents or carers were asked to complete questionnaires before and after having access to MoodHwb (after approximately 2 months). As with the consent forms, participants were given the choice of completing the questionnaires in person or via post (with phone or email support if required). Parents and carers completed questions about their child and about themselves.

    Measures

    The main outcomes of the evaluation related to the feasibility and acceptability of the digital program and of the evaluation process. Secondary outcomes included the potential effect on mood, knowledge, attitudes, and behavior after using the program.

    Feasibility and Acceptability of the Program

    The feasibility of the program was assessed in part through information on usage. This consisted of (1) Web data, including the number of visits to each section, language used on each visit (English/Welsh), and user pathway (for self or another person); and (2) self-reported questionnaire data, including the frequency of use, sections looked at the most often, language option chosen (English, Welsh, both), average time spent on the site each time, and use with others (eg, parent, carer, friend, professional). Participants were also asked about technical issues they experienced in the interviews and focus group.

    The acceptability of the program was assessed in the interviews, focus group, and feedback questionnaire. The interview topic guide covered (1) general views of the program, (2) specific strengths, (3) areas to develop further regarding the design and content of each section, and (4) integration into services and charities. The interviewing was iterative; where new themes emerged, they were incorporated into the subsequent interviews. The focus group discussion outline evolved from this guide.

    Feasibility and Acceptability of the Evaluation Process

    Assessment of the feasibility of the evaluation process included information related to the recruitment (number and range of participants) and retention of participants, as well as the completeness of data. Participants were also asked about the acceptability of the process in the interviews and via the pre intervention and post intervention questionnaires (eg, regarding their views on the number of questions).

    Potential Impact

    Standardized questionnaires were used to explore changes in depression literacy and stigma, help-seeking behavior, self-efficacy, behavioral activation, depression and anxiety symptoms, and general behavior (see Table 1 for the list of measures [24-32]).

    Table 1. Content of standardized questionnaires (pre intervention and post intervention).
    View this table

    Data Analysis

    Qualitative Analysis

    The interview and focus group transcripts were analyzed using a thematic analysis approach [33]. To ensure reliability of coding, all transcripts were coded by RBJ, and 40% of transcripts were double coded independently by SSA. Coding a proportion of interviews (from around 10% of transcripts) is a standard approach to improve coding reliability [34,35], and we agreed this higher proportion in advance as it was likely to increase the rigor of coding. We did not measure concordance in a quantitative way (eg, using kappa statistic), as this is controversial in qualitative research [36]. Agreement on concepts and coding was also sought with other authors (SS, AT, and BM). Initial ideas on the coding framework were discussed among the team; the draft framework was applied to some of the data and refined as coding proceeded. Codes were applied to broad themes, which were then broken down further into subcodes. Transcripts were closely examined to identify the key themes and associated subthemes. Thematic analysis was supported by the computer-assisted NVivo qualitative data analysis software (version 10, QSR International Pty Ltd).

    Quantitative Analysis

    Quantitative data from the questionnaires and Web usage were analyzed and presented descriptively with summary statistics (means, percentages). Paired sample t tests were used to explore changes in outcome scores between the pre intervention and post intervention questionnaires (the prescores were subtracted from the postscores). Findings from the Shapiro-Wilk test did not show statistical evidence to suggest a deviation from normality for any of the reported outcomes. Mean differences and confidence intervals were presented for these analyses. When generating sum scores for the individual pre questionnaires and post questionnaires, missing values were replaced with the mean value for that individual, provided there was less than 10% of missing data. All analyses were conducted using Stata statistical software (version 14, StataCorp LP).


    Results

    Participants

    Figure 4 shows the flow of participants in the quantitative evaluation phase. In total, 59 young people expressed an interest in participating in the evaluation phase, and 44 consented to participate (75%). Of those who consented to participate, 36 young people (82%) completed either the post intervention or feedback questionnaire (ie, 2 of the young people completed only one questionnaire). In all, 31 parents or carers provided consent to participate, of whom 21 (68%) completed either the post intervention or feedback questionnaire (ie, all but one of the parents or carers completed both questionnaires).

    Figure 4. Flowchart of participants in the quantitative evaluation phase.
    View this figure

    The characteristics of participating young people are presented in Table 2. The ratio of females to males was approximately 4:1. The majority (29/31, 94%) of the parents and carers who participated were mothers. In all, 45% (14/31) of the parents and carers were receiving treatment for depression, and 72% (21/29) had been treated for depression during their lifetime. Of those who participated in the previous development phase, 17 young people and 6 parents or carers consented to participate in the evaluation (ie, 39% of young people and 19% of parents and carers who consented to participate). The levels of missing data are shown in the footnotes of the tables below, where relevant.

    Table 2. Characteristics of young people participating in the study at baseline (N=43).
    View this table

    Descriptive information about interview and focus group participants is provided in Table 3. A total of 19 young people were interviewed; the ratio of females to males was approximately 3:1. Approximately two-thirds of the young people were interviewed along with their parents or carers (11 mothers, 1 father). In all, 9 of those interviewed spoke Welsh fluently. A total of 13 professionals from a range of disciplines participated in the focus group.

    Table 3. Characteristics of young people interviewed and professionals in the focus group.
    View this table

    Use of the Program

    Web Usage Data

    Analytics from the digital company revealed that the most common section accessed by young people, parents and carers was “What are mood and depression?” (27% of total use). The other sections were accessed approximately half as often (range 12%-17%). Overall, 16% of usage was in Welsh and 84% was in English. Most participants used the program for themselves (78% of use) and 22% for another person.

    Questionnaire Data

    The questionnaire findings on usage are presented in Table 4. In total, 21% (7/34) of young people used it once or twice a week, 44% (15/34) used it once or twice a month, and 26% (9/34) used it once or twice overall. Parents and carers used it less frequently, with most (11/20, 55%) using it only once or twice. A total of 30% (6/20) of parents and carers used it once or twice a month, and 10% (2/20) used it once or twice a week. Most young people (19/35, 54%), parents and carers (12/20, 60%) used it for half an hour at a time and most young people used it alone (29/33, 88%). Parents and carers were more likely to use it with others (7/19, 37%).

    Both groups reported looking the most at the “Self help,” “What are mood and depression?,” and “Possible reasons” sections. Young people also reported they looked at the “Mood monitor,” and parents and carers looked at the “Families, carers, friends, professionals” section.

    Interviews and Focus Group

    There were some difficulties related to the compatibility of the program with certain devices and operating systems, especially NHS computers. Some experienced difficulties with internet access at home or on mobile devices.

    Table 4. Use of program (questionnaire data).
    View this table

    Views/Acceptability of the Program (Results of Interviews and Focus Group)

    The overarching themes in the interviews and later in the focus group were as follows: (1) design features, (2) sections and content, and (3) integration and context. These themes were influenced by the primary issues the research team wished to explore, to help refine the content and design of MoodHwb. The overall feedback from participants is presented later, with verbatim examples listed in Table 5. Specific suggestions on changes to the program are described in Multimedia Appendix 1. Many of those who had participated in the development phase commented that they were pleased to see that suggestions made in the codevelopment group sessions had been adopted in the prototype.

    Key Theme 1: Design Features
    Overall Design, Navigation, and Ease of Use

    In general, all interview and focus group participants made favorable comments about MoodHwb and stated that it seemed helpful for young people, families and carers. Some noted they were surprised by the high quality, and that they had reservations about it previously because it might be “academic,” “dry,” or “overwhelming.” Most found the overall design attractive, and more interesting and appropriate than designs for existing resources (quote 1).

    Overall, participants felt that the program was clearly and consistently structured, easily navigated and user-friendly. They noted that the design and layout made it easy to identify what was relevant. Participants found the color coding, progress bar (see Figure 2), and quizzes particularly engaging (quote 2), but suggested that some elements could be better signposted.

    Interactive Elements and Personalization

    Figure 3 presents screenshots from the interactive elements of the program.

    Initial Questions and Mood Monitoring

    Nearly all praised the mix of rating scales (and corresponding face icons) and multiple-choice questions at the start of the program (which could also be answered subsequently to monitor mood), particularly for their functionality, ease of use, and “fun” element. Some noted that this was their favorite aspect (quote 3). Several professionals approved the message for the user to seek help, if they indicated they had thoughts of self-harm.

    “Stuff I Like”

    Some young people had used this to add songs and images, and felt that it helped to personalize the program. However, others had not done so, because they already bookmarked sites they found interesting.

    Goal Setting

    Most found “My goals” to be helpful and motivating. They liked how it was possible to add the number of activities, and to note when they were completed (quote 4).

    A few noted that their use of these interactive elements phased out over time, and that reminders and rewards might increase engagement. They also noted these components could be explained and personalized further.

    Dashboard and Profile Section

    “My profile” was described as helpful, particularly to group together strengths and difficulties and save links to resources. Professionals noted that this section helped users to reflect on and add perspective to their situation (quote 5).

    Illustrations and Animations

    Young people, parents and carers described the illustrations and animations as “friendly” and “sophisticated” without being “patronizing.” The characters were designed to appeal to a diverse audience, with abstract minimal features. All young people stated they preferred the illustrative approach to a more photographic one (quote 6).

    In summary, participants, in particular young people, gave favorable feedback on the design features, including the overall presentation, structure, and interactive and graphic elements. They also suggested the areas to develop further, including the navigation, personalized components, interactive and audiovisual elements, reminders, and rewards, and “native app.”

    Key Theme 2: Sections and Content
    Language, Tone, and Amount of Information

    On the whole, participants agreed that the language used was accessible, simple, and jargon-free, but “not too dumbed down.” Many approved of the general tone, stating it was “sensitive” and “affirming” (quote 7).

    In general, participants were pleased with the content, and found it comprehensive. They liked the flow of information, and how text was broken up (eg, by illustrations), “bite size,” and “short and snappy.”

    Professionals praised the “extensive,” “authoritative,” and “reliable” information. There were mixed views regarding the amount of text—some felt there was too much, others that it was appropriate (quote 8).

    Personal Stories

    All participants made favorable comments about the personal stories, and how they were from a range of perspectives (quote 9). Young people, parents, carers, and a small number of the professionals suggested adding stories from “celebrities,” and including photos, animations, videos, or comic strips.

    Table 5. Themes, subthemes, and quotes from interviews and focus group.
    View this table
    Sections

    Figure 1 shows how the sections are displayed on the welcome screen, and Figure 2 presents screenshots of two of the subsections.

    What Are Mood and Depression?

    Participants approved of the information on the difference between sadness and depression, how to identify difficulties and associated metaphors (quote 10). Some noted that those with depression might be reluctant to engage with the subject matter (quote 11).

    Possible Reasons for Low Mood and Depression

    Professionals were particularly pleased with this “exploratory” section (Figure 2; quote 12).

    Help Sections

    Several stated that the self help section was “motivational” and their favorite section (quote 13), and some asked for more self help approaches in specific situations. Professionals approved of the description of services and professionals (Figure 2), and of signposting to resources.

    Other Health Issues

    Most noted that the information was important and comprehensive, particularly on anxiety, eating and weight issues, and physical health. However, a few asked for more specific information (eg, on panic), while others were concerned that some users might worry unnecessarily (quote 14).

    Families, Carers, Friends, and Professionals

    Parents, carers, and professionals praised the “holistic” and “systemic” approach and highlighted the parental mental health subsection (quote 15). Some suggested adding more information on how to talk to a family member or friend (quote 16).

    Generally, participants approved of most aspects of the content, including the language, tone, personal stories, and the individual sections of the program. Areas to improve upon related to the specific information and amount of text in general.

    Key Theme 3: Integration and Context
    Targeted Versus Universal

    Most participants (and all professionals) thought that MoodHwb would be engaging and helpful for anyone who was interested in learning more about mood and depression (quote 17). Some of those who had not experienced depressive difficulties (from the EPAD study) did not think the program was as relevant to them, while some of those who had completed CAMHS therapy sessions already knew much of the information included in the program. On the whole, participants concluded that MoodHwb was especially appropriate for those starting to experience difficulties. Some stated that the program could help counteract stigma and making it freely available would help with this.

    Use With Families, Carers, and Friends

    Most young people had separate user accounts to their parents and carers; some suggested adding the option to monitor someone else’s mood and share information. Participants felt that the program could help with communication between young people and their family, carers and friends (quote 18).

    Integration With Services

    Many felt that MoodHwb could be used in schools, particularly personal, social and health education lessons – and suggested including a “teacher area” within the program. The school-based professionals stated that it could address the “big gap” in resources and training for teachers. However, some young people noted that associating it with schools might make it less appealing.

    Some noted that the use of MoodHwb could complement existing services, for example, by allowing young people to reflect on their entries with counselors, General Practitioners, and primary mental health and specialist CAMHS practitioners (quote 19).

    All health care professionals felt that the intervention could be used at their workplace—especially at the school and primary care level. Some noted that it could be helpful for those no longer in services, waiting for an appointment or who did not meet the criteria for mental health services. An educational psychologist noted that the digital program would be relevant for children in care, especially as “they’re moving around a lot” (quote 20).

    Many professionals highlighted the challenge of keeping the program up to date. One psychiatrist advised that many professionals were not IT-literate and might be reluctant to recommend it.

    In summary, most participants felt that all young people, families and carers should have access to MoodHwb, and it would be especially helpful as an early intervention in a range of settings for those who were starting to experience difficulties.

    Views and Acceptability of the Program (Results of Feedback Questionnaire)

    Overall, young people reported that the program was helpful, especially with finding ways to help themselves (Multimedia Appendix 2). When asked to select all the sections they found particularly helpful, most young people identified “Self help” (26/35, 74%), followed by “Mood monitor” (21/35, 60%), “What are mood and depression?” (19/35, 54%), and “Possible reasons” (19/35, 54%). The sections identified as the least helpful were “Stuff I like” (13/35, 37%), “Families, carers, friends, professionals” (7/35, 20%), and “My goals” (7/35, 20%), although interview feedback about these sections was more favorable. In total, 51% (17/33) downloaded the accompanying app and 14% (5/35) felt this was one of the most useful components.

    Findings were similar for parents and carers; they felt that the young person found it helpful overall and selected “Self help” (9/21, 43%) and the “Mood monitor” (8/21, 38%) as the most useful for the young person. Overall parents and carers found the program helpful for themselves (Multimedia Appendix 2). The sections identified as most helpful were “Possible reasons” (13/21, 62%), “What are mood and depression?” (11/21, 52%), “Where to get help” (9/21, 43%), “Families, carers, friends, professionals” (8/21, 38%), and “Self help” (8/21, 38%).

    Overall, young people, parents, and carers felt that the amount of information included in each section was adequate (Multimedia Appendix 2). Both groups rated the ease of use and clarity highly, and approved of the design elements, although parents and carers were slightly less positive than young people (Multimedia Appendix 2).

    Acceptability of Interviews and Questionnaires

    During the interviews, young people stated they were able to discuss the program openly and appreciated that they could choose the location, and whether they were seen with their parents or carers. They found it helpful to be able to navigate through the program as it was discussed.

    Data on acceptability of questionnaires were available for 43 young people and 30 parents or carers at baseline, and 34 young people and 19 parents or carers at follow-up. Most felt the number of questions was “about right.” Young people at follow-up were more likely to rate the questionnaires as having a “few too many questions” than at baseline, possibly due to the additional burden of the feedback questions.

    Potential Effect of the Program (Comparison of Pre intervention and Post intervention Questionnaires)

    Table 6 summarizes the results from the pre intervention and post intervention standardized questionnaires. It is important to note that these findings are considered exploratory given the small sample size and lack of power.

    Young People

    Levels of depression literacy were higher after using the program (difference in means 1.7, 95% CI 0.8 to 2.6). Although the differences were small, several other scores also improved, particularly regarding self-efficacy and depressive symptoms.

    Parents and carers

    Parents’ and carers’ own depression literacy improved after using the program (difference in means 1.3, 95% CI 0.4 to 2.2). Parent-rated scores of their children’s mood and behavior were similar or slightly worse after using the intervention. Overall, parent-rated scores of their child’s depression, anxiety, and behavior were lower than children’s self-rated scores.

    Table 6. Comparison of pre intervention and post intervention questionnaires.
    View this table

    Discussion

    We aimed to conduct an early-stage evaluation of a novel digital psychosocial intervention for adolescent depression and to assess the feasibility and acceptability of the assessment process that will be used in a future trial.

    The Digital Intervention: MoodHwb

    Most participants noted that the program was engaging, clear, easy to navigate, and well structured, and praised the graphic and animated approach. The content was described as comprehensive, motivational, holistic, and accessible to young people—and also relevant to families, carers and professionals in a range of services. Young people found it easier to use overall and were better able to understand the design elements, although parents, carers and professionals found it acceptable.

    Nearly all who completed the follow-up used the program, with most young people using it alone and for at least half an hour at a time over the course of the study (minimum of 2 months), which is favorable in terms of adherence in eHealth [12]. Participants did not use it frequently; however, they found it helpful overall, and its use might vary according to the context and needs of the user (eg, the stage and severity of their difficulties) [37].

    There were reports of improved depression literacy, self-efficacy (especially regarding self help strategies and knowing where to get help), and depressive symptoms—giving preliminary support for the intervention program theory [22]. There was no clear evidence of potential negative effects from using the program.

    These results were consistent with a recent systematic review of psychoeducational interventions in adolescent depression [11], which found that these programs can be acceptable and feasible and can influence a range of outcomes. Overall, the development and initial evaluation of MoodHwb fills a knowledge gap identified in the review (particularly of digital programs) and meets the need for good health information and evidence-based psychosocial interventions [5].

    MoodHwb might be particularly helpful when a young person starts to experience difficulties and first presents to services—for example to school counseling or nursing, primary care/mental health, youth, or social services. However, given that the majority of young people with depression do not access services [38], digital health interventions such as this could help improve reach and access to therapies. The program could also complement other approaches, for example, in the management of severe or chronic difficulties. It fits with the guided self help approach (as it can be used either independently or with another person) and the increasing interest in personalized medicine [39]. 

    This evaluation will be used to inform the refinement of MoodHwb, with the aim of improving the user engagement with it. The suggestions for improvement made by participants were discussed with the digital media company and research team. The issues were prioritized according to the level of importance given to them by the participants, and whether they would improve the acceptability, feasibility, and ease of use of the program. Other considerations were the program aims, underlying theory and evidence, technical difficulty, and time, resources, and funds required to make the changes.

    The main revisions considered included improving the navigation and audiovisual/interactive elements, making it more personalized, developing the app version of the program, and expanding the self help and anxiety sections (Multimedia Appendix 1). Ahead of a large trial, we would also need to ensure the program has the required technical specifications (eg, compatibility with devices and operating systems, adequate database capacity, and easy access to the app).

    The Evaluation Process

    Strengths

    One of the main strengths of the project relates to the rigorous mixed methodology, which followed guidance for complex interventions [20] for both the development and initial evaluation reported here. There was also a collaborative, person-based, and iterative approach [17] for the intervention development.

    Another strength was the diverse range of participants from recruitment centers in urban and rural areas, including in areas with several ethnicities and areas of deprivation. Efforts were made to engage young people, families, and carers by offering to see them where convenient. There was also contact afterwards to promote retention and adherence (eg, email, text, phone). The recruitment target of 40 young people was met, and there was a good retention rate (82% of young people). The digital media team was available to address technical issues during the study.

    Limitations

    The research and digital media teams were involved throughout the project, and it is possible that their personal views could have influenced the development and evaluation process. Considering the number of centers visited, only a small number of potential participants were referred, which may have led to a biased sample. Several of those who initially showed interest did not go on to participate, and often no reason was given. This might reflect the difficulty in engaging young people in mental health research, or that the program might be more useful for certain subgroups or contexts. More females than males participated, although this might reflect in part that depression is more common in females in adolescence [1].

    Participants might have reported more favorable responses because they were seen by, or in regular contact with, the researchers. Some had participated in the development phase, which may have influenced their opinion of the program (those who had been involved in its development generally gave favorable feedback on the prototype). Outcomes were assessed via self-report and response bias could be an issue, for example, where participants provide socially desirable answers. Young people might have been less likely to give candid responses in the presence of parents or carers, although they were given the option of being seen alone.

    The results of the questionnaires must be interpreted with caution given the small sample size and limited power. It would have been interesting to explore how factors (eg, age, gender, severity/history of difficulties, family history, and involvement of parent and carer) might have influenced the results; however, the exploration of subgroups was not feasible given the sample size. It is also possible that some of those whose symptoms or behavior improved may have done so in any case over time, or due to other interventions or factors.

    Conclusions

    This early mixed methods evaluation found that the co-designed prototype of MoodHwb and assessment process were feasible and acceptable. The findings will be used to refine the intervention and inform the design of a randomized controlled trial [20]. If proven to be effective, MoodHwb could ultimately be rolled out as an early intervention program in health, education, social, and youth services and charities to help young people, families, carers, friends, and professionals.

    Acknowledgments

    The authors would like to thank all the young people, parents, carers, professionals, and researchers who participated in the study. Will Richards and Andrew Price (Made by Moon digital media company), Mike Clague (Carbon Studio animation company), and Dyfan Williams (graphic designer) helped to develop the program and accompanying literature, and the authors thank them for their work. The authors would like to thank Emma Rengasamy (medical student) for helping to facilitate the professionals’ focus group. This work was supported by the Welsh Government through Health and Care Research Wales, and the authors thank them for their support. RBJ was supported by a National Institute for Health Research/Health and Care Research Wales Programme Fellowship (NIHR-FS-2012). SS was supported by a Medical Research Council Strategic Award (MC-PC-13027, MC_UU_12017_14, MC_UU_12017_1, SPHSU11, and SPHSU14). BM was supported by the NIHR Biomedical Research Centre at the University Hospitals Bristol NHS Foundation Trust and the University of Bristol. DS is supported by a Lister Institute Prize Fellowship (2016-2021). IJ is Director of the National Centre for Mental Health, and the authors thank the center for its support. The funders had no role in the study design or conduct of the study or in the writing of the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Table 1: Suggestions for the further development of the program; Table 2: Further quotes from the interviews and focus group.

    DOCX File , 22 KB

    Multimedia Appendix 2

    Questionnaire feedback on program.

    DOCX File , 342 KB

    References

    1. Thapar A, Collishaw S, Pine DS, Thapar AK. Depression in adolescence. Lancet 2012 Mar 17;379(9820):1056-1067 [FREE Full text] [CrossRef] [Medline]
    2. North West Clinical Senates. London: Department of Health; 2009. New Horizons: Towards a Shared Vision for Mental Health   URL: https:/​/webarchive.​nationalarchives.gov.uk/​20130104235147/​http:/​/www.​dh.gov.uk/​en/​Publicationsandstatistics/​Publications/​PublicationsPolicyAndGuidance/​DH_109705 [accessed 2020-03-23]
    3. Thapar A, Collishaw S, Potter R, Thapar AK. Managing and preventing depression in adolescents. Br Med J 2010 Jan 22;340:c209. [CrossRef] [Medline]
    4. Bennett C, Bevan Jones R, Smith D. Prevention strategies for adolescent depression. Adv Psychiatr Treat 2014 Mar;20(2):116-124 [FREE Full text] [CrossRef]
    5. National Institute for Health and Clinical Excellence (NICE). Depression in children and young people: Identification and management. London: NICE Clinical Guideline; 2019.
    6. Birmaher B, Brent D, AACAP Work Group on Quality Issues, Bernet W, Bukstein O, Walter H, et al. Practice parameter for the assessment and treatment of children and adolescents with depressive disorders. J Am Acad Child Adolesc Psychiatry 2007 Nov;46(11):1503-1526. [CrossRef] [Medline]
    7. Colom F. Keeping therapies simple: psychoeducation in the prevention of relapse in affective disorders. Br J Psychiatry 2011 May;198(5):338-340. [CrossRef] [Medline]
    8. Brent D, Maalouf F. Depressive disorders in childhood and adolescence. In: Thapar A, Pine DS, Leckman JF, Scott S, Snowling MJ, Taylor E, editors. Rutter's Child and Adolescent Psychiatry. Sixth Edition. New Jersey: Wiley-Blackwell; 2015:873-892.
    9. Stice E, Shaw H, Bohon C, Marti, Rohde P. A meta-analytic review of depression prevention programs for children and adolescents: factors that predict magnitude of intervention effects. J Consult Clin Psychol 2009 Jun;77(3):486-503 [FREE Full text] [CrossRef] [Medline]
    10. Tursi M, Baes CV, Camacho FR, Tofoli SM, Juruena MF. Effectiveness of psychoeducation for depression: a systematic review. Aust NZ J Psychiatry 2013 Nov;47(11):1019-1031. [CrossRef] [Medline]
    11. Bevan Jones R, Thapar A, Stone Z, Thapar A, Jones I, Smith D, et al. Psychoeducational interventions in adolescent depression: A systematic review. Patient Educ Couns 2018 May;101(5):804-816 [FREE Full text] [CrossRef] [Medline]
    12. Hollis C, Morriss R, Martin J, Amani S, Cotton R, Denis M, et al. Technological innovations in mental healthcare: harnessing the digital revolution. Br J Psychiatry 2015 Apr;206(4):263-265. [CrossRef] [Medline]
    13. Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MF. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. Br Med J 2012 Apr 18;344:e2598 [FREE Full text] [CrossRef] [Medline]
    14. Reyes-Portillo JA, Mufson L, Greenhill LL, Gould MS, Fisher PW, Tarlow N, et al. Web-based interventions for youth internalizing problems: a systematic review. J Am Acad Child Adolesc Psychiatry 2014 Dec;53(12):1254-70.e5. [CrossRef] [Medline]
    15. Rice SM, Goodall J, Hetrick SE, Parker AG, Gilbertson T, Amminger GP, et al. Online and social networking interventions for the treatment of depression in young people: a systematic review. J Med Internet Res 2014 Sep 16;16(9):e206 [FREE Full text] [CrossRef] [Medline]
    16. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual Research Review: Digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry 2017 Apr;58(4):474-503. [CrossRef] [Medline]
    17. Yardley L, Morrison L, Bradbury K, Muller I. The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res 2015 Jan 30;17(1):e30 [FREE Full text] [CrossRef] [Medline]
    18. Mohr DC, Schueller SM, Montague E, Burns MN, Rashidi P. The behavioral intervention technology model: an integrated conceptual and technological framework for eHealth and mHealth interventions. J Med Internet Res 2014 Jun 5;16(6):e146 [FREE Full text] [CrossRef] [Medline]
    19. Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: Key issues, process model, and system features. Commun Assoc Inf Syst 2009;24:485-500. [CrossRef]
    20. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J 2008 Sep 29;337:a1655 [FREE Full text] [CrossRef] [Medline]
    21. Bevan Jones R, Thomas J, Lewis J, Read S, Jones I. Translation: From Bench to Brain – Using the visual arts and metaphors to engage and educate. Research All 2017;1(2):265-283 [FREE Full text] [CrossRef]
    22. Bevan Jones R, Thapar A, Rice F, Beeching H, Cichosz R, Mars B, Smith, et al. A web-based psychoeducational intervention for adolescent depression: design and development of MoodHwb. JMIR Ment Health 2018 Feb 15;5(1):e13 [FREE Full text] [CrossRef] [Medline]
    23. Mars B, Collishaw S, Smith D, Thapar A, Potter R, Sellers R, et al. Offspring of parents with recurrent depression: which features of parent depression index risk for offspring psychopathology? J Affect Disord 2012 Jan;136(1-2):44-53. [CrossRef] [Medline]
    24. Hart SR, Kastelic EA, Wilcox HC, Beaudry MB, Musci RJ, Heley KM, et al. Achieving Depression Literacy: The Adolescent Depression Knowledge Questionnaire (ADKQ). School Ment Health 2014 Sep 1;6(3):213-223 [FREE Full text] [CrossRef] [Medline]
    25. Griffiths, Christensen H, Jorm AF, Evans K, Groves C. Effect of web-based depression literacy and cognitive-behavioural therapy interventions on stigmatising attitudes to depression: randomised controlled trial. Br J Psychiatry 2004 Oct;185:342-349. [CrossRef] [Medline]
    26. Wilson CJ, Deane FP, Ciarrochi J, Rickwood D. Measuring help-seeking intentions: properties of the general help-seeking questionnaire. Can J Couns 2005;39(1):15-28 [FREE Full text]
    27. Tonge B, King N, Klimkeit E, Melvin G, Heyne D, Gordon M. The Self-Efficacy Questionnaire for Depression in Adolescents (SEQ-DA). Development and psychometric evaluation. Eur Child Adolesc Psychiatry 2005 Oct;14(7):357-363. [CrossRef] [Medline]
    28. Kanter JW, Mulick PS, Busch AM, Berlin KS, Martell CR. The Behavioral Activation for Depression Scale (BADS): Psychometric properties and factor structure. J Psychopathol Behav Assess 2007;29(3):191-202 [FREE Full text] [CrossRef]
    29. Angold A, Costello EJ, Messer SC, Pickles A. Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. Int J Methods Psychiatr Res 1995;5(4):237-249 [FREE Full text]
    30. Birmaher B, Brent DA, Chiappetta L, Bridge J, Monga S, Baugher M. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): a replication study. J Am Acad Child Adolesc Psychiatry 1999 Oct;38(10):1230-1236. [CrossRef] [Medline]
    31. Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry 1997 Jul;38(5):581-586. [CrossRef] [Medline]
    32. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983 Jun;67(6):361-370. [CrossRef] [Medline]
    33. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101 [FREE Full text] [CrossRef]
    34. Lombard M, Snyder-Duch J, Bracken CC. Content analysis in mass communication: assessment and reporting of intercoder reliability. Human Comm Res 2002 Oct;28(4):587-604. [CrossRef]
    35. Ranney ML, Meisel ZF, Choo EK, Garro AC, Sasson C, Guthrie KM. Interview-based qualitative research in emergency care part II: data collection, analysis and results reporting. Acad Emerg Med 2015 Sep;22(9):1103-1112 [FREE Full text] [CrossRef] [Medline]
    36. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? Br Med J 2001 May 5;322(7294):1115-1117 [FREE Full text] [CrossRef] [Medline]
    37. Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res 2017 Jun 29;19(6):e232 [FREE Full text] [CrossRef] [Medline]
    38. Potter R, Mars B, Eyre O, Legge S, Ford T, Sellers R, et al. Missed opportunities: mental disorder in children of parents with depression. Br J Gen Pract 2012 Jul;62(600):e487-e493 [FREE Full text] [CrossRef] [Medline]
    39. Bahcall O. Precision medicine. Nature 2015 Oct 15;526(7573):335. [CrossRef] [Medline]


    Abbreviations

    CAMHS: Child and Adolescent Mental Health Services
    eHealth: electronic health
    EPAD: Early Prediction of Adolescent Depression
    UHBs: University Health Boards


    Edited by G Doron; submitted 25.08.19; peer-reviewed by E Lattie, A Lundervold; comments to author 22.10.19; revised version received 17.12.19; accepted 02.02.20; published 17.07.20

    ©Rhys Bevan Jones, Anita Thapar, Frances Rice, Becky Mars, Sharifah Shameem Agha, Daniel Smith, Sally Merry, Paul Stallard, Ajay K Thapar, Ian Jones, Sharon A Simpson. Originally published in JMIR Mental Health (http://mental.jmir.org), 17.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.