This paper is in the following e-collection/theme issue:

Technology in Psychiatry/Clinical Psychology Training and Education (26) e-Learning and Medical Education (645) Human Factors and Usability Case Studies (739) Methods and Feasibility Studies (367)

Published on in Vol 5, No 3 (2018): Jul-Sept

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10517, first published .
A New Online Mental Health Training Program for Workplace Managers: Pre-Post Pilot Study Assessing Feasibility, Usability, and Possible Effectiveness

A New Online Mental Health Training Program for Workplace Managers: Pre-Post Pilot Study Assessing Feasibility, Usability, and Possible Effectiveness

A New Online Mental Health Training Program for Workplace Managers: Pre-Post Pilot Study Assessing Feasibility, Usability, and Possible Effectiveness

Authors of this article:

Aimée Gayed1 Author Orcid Image ;   Anthony D. LaMontagne2, 3 Author Orcid Image ;   Allison Milner2 Author Orcid Image ;   Mark Deady4 Author Orcid Image ;   Rafael A. Calvo5 Author Orcid Image ;   Helen Christensen4 Author Orcid Image ;   Arnstein Mykletun1, 6, 7, 8, 9 Author Orcid Image ;   Nick Glozier10 Author Orcid Image ;   Samuel B Harvey4 Author Orcid Image
Article Authors Cited by (12) Tweetations (18) Metrics

    Original Paper

    1School of Psychiatry, Faculty of Medicine, University of New South Wales, Randwick, Australia

    2School of Population and Global Health, The University of Melbourne, Melbourne, Australia

    3Centre for Population Health Research, Deakin University, Geelong, Australia

    4Black Dog Institute, Faculty of Medicine, University of New South Wales, Sydney, Australia

    5School of Electrical and Information Engineering, University of Sydney, Sydney, Australia

    6Department of Mental Health and Suicide, Norwegian Institute of Public Health, Oslo, Norway

    7Department of Community Medicine,, University of Tromsø, Tromsø, Norway

    8Centre for Work and Mental Health, Nordland Hospital Trust, Bodø, Norway

    9Centre for Research and Education in Forensic Psychiatry and Psychology, Haukeland University Hospital, Bergen, Norway

    10Brain and Mind Centre and Central Clinical School, School of Medicine, Faculty of Medicine and Health, University of Sydney, Sydney, Australia

    Corresponding Author:

    Aimée Gayed, BPsych (Hons), MCrim

    School of Psychiatry

    Faculty of Medicine

    University of New South Wales

    Black Dog Institute Building

    Hospital Road

    Randwick, 2031

    Australia

    Phone: 61 +61293824517

    Fax:61 +61293828208

    Email: a.gayed@unsw.edu.au


    Background: Mental health has become the leading cause of sickness absence in high-income countries. Managers can play an important role in establishing mentally healthy workplaces and coordinating their organization’s response to a mentally ill worker.

    Objective: This pilot study aims to evaluate the feasibility, usability, and likely effectiveness of a newly developed online training program for managers called HeadCoach. HeadCoach aims to build managers’ confidence in supporting the mental health needs of staff and promote managerial behavior most likely to result in a more mentally healthy workplace.

    Methods: In total, 66 managers from two organizations were invited to participate in this pre-post pilot study of HeadCoach, which was made available to managers to complete at their own pace over a 4-week period. Data were collected at baseline and post intervention via an online research platform. The difference in mean scores for each outcome between these two time points was calculated using paired samples t tests.

    Results: Of all the invited managers, 59.1% (39/66) participated in the trial, with complete pre–post data available for 56.4% (22/39) of the participants. The majority of respondents reported positive engagement with the program. During the study period, managers’ knowledge regarding their role in managing mental health issues (P=.01) and their confidence in communicating with employees regarding mental illness (P<.001) significantly increased. In addition, a significant increase was observed from the baseline in managers’ self-reported actions to use strategies to prevent and decrease stress among their team members (P=.02).

    Conclusions: Although caution is needed due to the absence of a control group, preliminary results of this study suggest that HeadCoach could be a feasible, acceptable, and efficient method of training managers in best workplace practices to help support the mental health needs of their staff.

    JMIR Ment Health 2018;5(3):e10517

    doi:10.2196/10517

    Keywords

    manager; supervisor training; workplace mental health; mental health education; online intervention; knowledge; attitudes; behaviour; eHealth 


    In several high-income countries, mental health conditions have become the leading cause of long-term sickness absence and occupational incapacity [1-4]. The development or persistence of mental ill health for some workers might, in part, be related to their workplace [5], a link that has now been acknowledged as a major public health concern [4]. Anxiety and mood disorders are the most common mental illnesses reported in the working population [6-8]. Although treatable and often preventable, the rates of functional impairment due to psychiatric conditions within the working age group have increased over recent decades [9], which comes at a substantial cost to individuals, their workplaces, and, eventually, the economy [4,7,10-14]. Thus, there is a growing focus on elucidating how work can affect mental health and how it can be addressed through workplace-based mental health and well-being interventions [5,15,16].

    Workplace mental health programs could provide an opportunity to alter modifiable risk and protective factors for mental health and a chance to aid the identification, treatment, and rehabilitation of workers with mental health problems. Some psychosocial working conditions have been recognized as primary sources of work-related stress, which, if not managed effectively, can adversely affect workers’ well-being and productivity; these include conflicting and excessive work demands, a lack of job control, organizational failure to effectively communicate with staff, and poor collegial relationships and support [15,17]. Many of these workplace risk factors can be modified through decisions and adjustments, which managers are often in a position to make [18]. The degree to which the managers set a positive example of accepting attitudes and supportive behaviors toward the mental health of their staff can act as a protective factor for their workers. In addition, managers can react to mental ill health episodes in a way that could benefit the recovery process for workers [19]. Such strategies include, but are not limited to, facilitating regular conversations with an employee, maintaining a focus on an employee’s well-being, and developing an appropriate return to work plan if a worker is on long-term sickness absence for a mental health issue, regardless of the underlying cause [20,21]. Overall, these various preventative and responsive strategies delivered across individual, team, and organizational levels can create a mentally healthy workplace that enhances the mental health of its employees [4]. Despite the availability of best practice guidelines detailing these primary, secondary, and tertiary approaches to managing workplace mental health, managers often report uncertainty with regard to how to best support employees experiencing or at risk of mental illness [7,21].

    In order to address these concerns, many organizations are introducing training for managers in how to decrease work-based mental health risk factors for their employees, support their recovery, and facilitate successful return to work following a period of sickness absence for mental ill health. There is some evidence suggesting the value of specialized training delivered to managers to promote an understanding of the mental health needs of their workers and help increase managers’ confidence in discussing mental health matters with their staff [2,3,22-24]. Further evidence supports that such manager training is effective in shifting stigmatizing attitudes regarding mental illness [24-27] and promoting the implementation of positive managerial behaviors to address mental health issues within their team [22,24,26] with an overall positive effect for manager training found across these outcomes [28]. Yet, evaluations of a selection of workplace-based mental health training programs have been unable to determine the beneficial effects on managers’ attitude toward mental illness [18] or managerial behaviors of mental health issues either reported by managers themselves or objectively by their direct reports [18,29,30]. This disparity in outcomes may be due to the selection of components included in the training. It is becoming increasingly recognized that an integrated approach is considered the best practice in workplace mental health interventions [15]. An integrated approach incorporates strategies to prevent harm, promote positive mental health, and address mental health at the workplace irrespective of the cause of the illness [15]. However, the integration of these key components has yet to become standard in manager training, resulting in the dissemination of a series of uncoordinated educational programs.

    This pilot study comprises the feasibility stage of the Medical Research Council (MRC) Framework for complex interventions [31] and elucidates the development and initial testing of the delivery of a comprehensive online training intervention for managers called HeadCoach. HeadCoach integrates the components recognized as key to mental health training and aims to build managers’ confidence and ability to best support the mental health needs of the staff they supervise. The content for this online program has been derived from two separate face-to-face programs developed for managers with a focus on the mental health of their employees [22,32], which, when combined, encompass the recommended reactive and preventative components. We acknowledge that e-learning cannot offer some benefits that face-to-face contact with an educator may provide, and discussion with other participants in the course is often limited or not available. However, the modification of the delivery format from face-to-face to a mobile-responsive website offers a more flexible, time-effective, and economical means of training a large number of staff members [33]. Participants have the opportunity to schedule training around the demands of their job and may also revisit the content within a standardized learning environment, providing a better opportunity for the consolidation of the course material.

    This pilot study aims to test the feasibility and usability of the HeadCoach program with a small group of managers prior to evaluating the program as part of a larger randomized controlled trial (RCT). The objective data on program engagement, matched with participants’ self-reported program rating scores and free text feedback, would provide information on the rates of adherence and user experience. Besides evaluating the uptake of and interaction with the program, this study aims to investigate the possible effectiveness of HeadCoach as a workplace mental health intervention for managers. We hypothesized that the HeadCoach program would help improve managers’ self-reported confidence to respond to the needs of staff experiencing mental health issues and promote their implementation of managerial techniques that would create a more mentally healthy workplace. Although the results of this study would not be capable of determining the true effectiveness of the intervention due to the absence of a control group, findings from this pre–post design may provide an initial insight into participants’ responses to and acceptability of the program and be a valuable first step in examining any impact that may be found from this type of training.


    Development of Intervention

    In accordance with the MRC Framework for complex interventions [31], we reviewed the relevant literature and a meta-analysis regarding manager training [28] to establish a theoretical basis for the development of the intervention. Following this, the HeadCoach online training intervention [34] was developed to improve workplace mental health by providing a flexible, easily accessible, and engaging training program for managers that informed them how to best aid the mental health needs of all their employees. Consistent with the concepts of the self-efficacy theory [35], which indicates that people are more likely to engage in particular behaviors when they feel more capable of attaining the desired behavior, this program aimed to build managers’ confidence to effectively respond to the needs of employees experiencing mental health issues and implement evidence-based managerial styles that promote a more mentally healthy workplace environment.

    Previously, collaborators on this trial have conducted cluster RCTs of two different face-to-face manager training programs. The “RESPECT” Manager Training Program [22] is a 4-hour training package delivered by a clinical psychologist or a consultant psychiatrist to small groups of managers; this package combined mental health knowledge and communication training to promote more appropriate reactive responses from managers when mental health issues arise in staff they supervise. A previous RCT of this manager training showed that it resulted in changes to managers’ confidence and reactive behavior that lasted for six months at least [22]. The second existing face-to-face program, which was more preventative in its approach, aimed to improve psychosocial working conditions within an organization by changing the managers’ behavior to best promote a mentally healthy workplace [36]; the principles within this program were based on the management competencies for preventing and reducing stress at work developed by the Health and Safety Executive (HSE) in the UK [37]. HeadCoach was developed by combining the content from these two face-to-face programs and transforming the material into a format compatible with online delivery. Educators, designers, and IT developers with experience in creating online mental health programs were consulted for the development and appearance of the learning platform. Their primary aim was to create a functional user experience that would appeal to the target audience; this included carefully considering aspects such as the style of language in which the content would be presented in and the visual design features selected for the branding.

    Consultation Groups

    As part of the modeling phase recommended when assessing complex interventions [31], we held meetings ahead of the pilot stage of the trial with representatives from various industries partnering on the evaluation of HeadCoach. Included in these meetings were members from the organizations’ human resources, health and well-being, and media communication teams as well as managers representing potential users of HeadCoach. During these sessions, we consulted participants on the relevance of the vignettes to be included in the program as well as the appropriateness of the language and style. In addition, sections of the program were also shown to determine the feasibility and usability for different industry groups. Where appropriate, adjustments were made to the content and functionality based on the feedback received from the consultation groups.

    HeadCoach Content

    HeadCoach is a self-paced online intervention program that comprises three topics to be completed sequentially. Each topic contains a series of 10-minute modules featuring small sections of text, activities, short videos, practical activities, and topic summary exercises for individuals to complete. Textbox 1 presents the pilot program outline, and Figures 1-3 display the screen shots of various pages within HeadCoach. In addition, Supplementary File 1 details the vignettes included in the summary exercises for topics 2 and 3. The first topic, common mental illnesses, introduces mental health issues commonly found at the workplace. The following topic, topic 2, how to help an employee you are concerned about, outlines the signs managers can look out for to assist in recognizing people within their team who may be at risk of mental health issues, provides useful steps on how to initiate a conversation with employees who may be experiencing mental ill health, and discusses managerial techniques that can be implemented to support employees such as having regular catch ups with staff, creating a meaningful workplace where staff feel connected, and knowing what mental health resources are available for employees. In addition, it specifies ways to assist employees to stay at work if they wish or return to work faster following a period of sickness absence owing to mental illness. Finally, topic 3, minimizing mental health risks at the workplace, aims to up-skill managers with techniques useful in altering a range of workplace mental health risk factors across the individual, team, and organizational levels [21] to create a mentally healthy workplace for their employees. Strategies include how to prevent and manage work-related stress for employees by being a respectful and responsible manager, manage and communicate existing and future work for staff, tailor the management of individuals based on their individual capabilities and needs, and proactively and objectively resolve difficult situations and conflict within the team. These techniques are based on observational data [21] and management standard frameworks provided by agencies such as the HSE in the UK [37].

    Participants were expected to complete HeadCoach in approximately 2.5 hours, although it was designed in a way that users could work through the content at a pace that suits their learning style and job demands across a 4-week period with automated weekly reminders sent to prompt program adherence. In addition, the intervention was accessed online through a mobile-responsive website using a desktop, laptop, tablet, or mobile phone.

    Study Design

    This pilot study was primarily conducted to test the feasibility and usability of HeadCoach and investigate, through self-reported responses, early evidence of its possible effectiveness in altering managers’ level of confidence regarding dealing with mental health matters at the workplace. In order to maximize the opportunity to test the functionality of the product and receive substantial feedback on the processes and quality of training, all participants were allocated to receive the intervention. Participants comprised managers employed by one of the two industry partners collaborating on the pilot study. HeadCoach was delivered to managers individually through their usual computer, tablet, or Web-based mobile phone. Through an online form, the managers provided informed consent to participate in the study.

    Recruitment

    Two organizations in Australia volunteered to participate in the pilot evaluation of HeadCoach. One organization was an equipment and machinery hire company servicing Australia across metropolitan, regional, and remote areas; managers in this organization supervised branches and employees across one or more offices or regions. The second organization provided a statewide fire and rescue service; managers in this organization who were eligible for this pilot study were employed at the duty commander or station officer level and were responsible for one or several different fire stations. For both organizations, the managerial level identified as relevant for this pilot study comprised supervisors who were the primary contact for staff members regarding periods of sickness absence or when workplace issues arose.

    Textbox 1. The course outline for the pilot version of HeadCoach.

    Topic 1: Common Mental Illnesses

    • Module 1: Recognizing mental health issues
    • Module 2: The workplace and its people
    • Module 3: Topic summary exercises

    Topic 2: How to help an employee you are concerned about

    • Module 1: Identifying people at risk
    • Module 2: Providing support
    • Module 3: Having the talk
    • Module 4: Facilitating help seeking
    • Module 5: Modifying work to help recovery
    • Module 6: Returning to work
    • Module 7: Topic summary exercises

    Topic 3: Minimizing mental health risks in the workplace

    • Module 1: How to be a respectful and responsible manager
    • Module 2: Managing and communicating existing and future work
    • Module 3: Managing individuals within a team
    • Module 4: Managing difficult emotions
    • Module 5: Topic summary exercises
    ‎
    Figure 1. Screenshot of the HeadCoach dashboard.
    ‎
    Figure 2. Screen shot of the landing page for a HeadCoach video resource.
    ‎
    Figure 3. Screen shot of an interactive exercise with feedback in HeadCoach.

    Participants met the inclusion criteria if they were currently supervising a team of ≥3 employees, aged ≥18 years, and currently residing in Australia with a good level of English comprehension. We contacted 66 managers via email who were identified by their employer as those fulfilling the inclusion criteria. The email described the purpose of the research, outlined what would be involved, and contained a link to register for the pilot study. In addition, the email emphasized that although participation in the trial was supported by their employer, it was entirely voluntary, and their level of involvement in the study would remain confidential from their employer.

    Procedure

    The trial procedure and stages of assessment for participants are outlined in Figure 4. All eligible to participate in the study received an information email from the researchers, inviting them to participate in the study. A hyperlink provided in the email directed them to the HeadCoach study website, which detailed further information about the program, the online consent form, and the registration page. Once registration was complete, the online baseline assessment was made available, following the completion of which participants were able to commence the online HeadCoach manager training program. The design of the program allowed managers to work through the program at their own pace over a 4-week period. Emails were automatically distributed weekly from sign up until participants had completed the program; these emails informed participants of the time remaining to complete the program and served as a reminder to revisit their account to continue working through the program.

    Follow-up questionnaires were disseminated 4 weeks after completion of the baseline questionnaire. This 4-week period comprised the training period allocated to managers to complete HeadCoach. If all the components of the online program were completed earlier in the 4-week training period, the participant was invited to complete the follow-up questionnaire at that time point. This approach was selected to reduce nonresponse.

    Data Collection

    The baseline and postintervention data were collected electronically via the research platform that hosted the HeadCoach program, allowing a streamlined process between completing the HeadCoach program and completing questionnaires. In the case of nonresponse, two reminders were automatically sent across the subsequent 10 days via the research platform. On the completion of the 4-week postintervention questionnaire, the HeadCoach content was made available again for participants within their account for their reference.

    ‎
    Figure 4. HeadCoach pilot study procedure flow.
    Measures of Usability and Feasibility

    At the 4-week postintervention follow-up, participants were asked whether “This online course was engaging and interesting?” with response options ranging from strongly disagree to strongly agree. In addition, questions were asked evaluating the ease of navigating the program and finding information, whether the course fulfilled their expectations, and how likely would they recommend the program to their colleagues. Participants were also provided with an opportunity to provide detailed feedback through additional free text questions, including suggestions on what should be included in future versions. Finally, we obtained data regarding completion rates, including the duration of time taken to complete each module, from the research platform database.

    Measures of Effectiveness

    We measured managers’ confidence in managing mental health issues and promoting a mentally healthy workplace using a modified version of the previously described supervisor scales [19]. Managers were asked to indicate their current level of confidence on a 5-point Likert scale with options ranging from not at all confident to extremely confident. Online Supplementary File 2 details the questions used for this measure. Knowledge about common mental health was assessed using the first 6 questions of the Mental Health Knowledge Schedule (MAKS) [38]. Nonstigmatizing attitudes toward mental illness was assessed using a modified version of previously published measures of personal stigma [39-41]. Comprehension of their role as a manager in dealing with mental health in the workplace was assessed using a questionnaire developed in accordance with the core competences outlined in a report detailing managers’ role in supporting return to work after ill health [20]. The managers’ application of managerial techniques that promote a mentally healthy workplace was assessed using an adapted version of the HSE Management Standards Indicator Tool [17]. We used a 5-point Likert scale with response options ranging from strongly disagree to strongly agree for questions such as “I provide regular opportunities for my team to speak one to one” and “I give employees the right level of job responsibility.” Online Supplementary File 2 provides the remaining items. All questions were asked at the baseline and at the 4-week postintervention assessment. We converted the participants’ scores to a percentage of the maximum possible score prior to data analysis.

    We collected demographic information including age, gender, job role and length of service in the role, number of employees currently supervising, and previous mental health training.

    Statistical Analysis

    Participants’ use of the training program was described in terms of the total time and number of modules completed. Descriptive statistics were used to demonstrate participants’ responses to the questions on the usability and acceptability of HeadCoach. We assessed the differences in the mean percentage scores of each of the outcomes between baseline and post intervention collected at the 4-week follow-up using paired samples t tests. The parameters included managers’ confidence in managing mental health issues within the workplace, level of mental health literacy, nonstigmatizing attitude toward mental illness, understanding of their role in managing mental health issues within their team, and application of managerial techniques that promote a mentally healthy workplace. All statistical analyses were conducted using SPSS version 23.


    Demographics

    In this study, all managers were enrolled during November 2016 and December 2016 with the postintervention data collected during December 2016 and January 2017. Among the 66 managers who were invited to participate in the HeadCoach pilot study, 59.1% (39/66) registered and completed the baseline assessment. All managers were assigned to receive the intervention. Table 1 outlines the demographics of the study sample, including baseline characteristics of the entire sample and participants included in the final analysis.

    Usability and Feasibility of HeadCoach

    Over half of the study sample (51.3%; 20/39) completed all 15 modules of the HeadCoach program. The time taken to complete the program ranged from 43 minutes to 2 hours 53 minutes. We obtained feedback on the program through the postquestionnaire. Of those who responded, 77% (17/22) found HeadCoach engaging and interesting, 86% (19/22) agreed it was easy to find the information needed, 73% (16/22) reported that the program fulfilled their expectations, 91% (20/22) considered it useful, and 86% (19/22) considered it worth recommending to a friend.

    Table 1. Demographics as reported at the baseline.
    DemographicTotal sample (n=39)Sample with pre- and postintervention data available (n=22)
    Age (years), mean (SD)45.03 (8.74)47.86 (8.44)
    Gender, n (%)


    		Male34 (87.2)20 (90.9)

    		Female5 (12.8)2 (9.1)
    Industry, n (%)


    		Building/Construction12 (30.8)5 (22.7)

    		Emergency Services27 (69.2)17 (77.3)
    Years at current employer, n (%)


    		<13 (7.7)2 (9.1)

    		1-55 (12.8)2 (9.1)

    		5-105 (12.8)2 (9.1)

    		10-154 (10.3)2 (9.1)

    		>1522 (56.4)14 (63.6)
    Years in this level or above, n (%)


    		<16 (15.4)2 (9.1)

    		1-59 (23.1)4 (18.4)

    		5-1014 (35.9)10 (45.5)

    		10-156 (15.4)4 (18.2)

    		>154 (10.3)2 (9.1)
    Modules completed, n (%)


    		07 (17.9%)1 (4.5)

    		1-78 (20.5%)1 (4.5)

    		8-144 (10.3%)3 (13.6)

    		15 (Completed program)20 (51.3%)17 (77.3)

    		Mean (SD)9.08 (6.62)12.95 (4.37)
    ‎
    Figure 5. Baseline and 4-week post intervention mean average scores (%) and standard error of means for outcomes.

    The qualitative feedback received post evaluation was primarily positive. All participants highly valued the practicality of the information and the format it was presented in. Examples of comments reflecting these views included “This topic was useful in giving logical and practical guidance on early intervention and strategies to minimize harm occurring in the workplace,” “Good tips on initiating conversations and advice on where to go for resources,” “It was great to get some practical examples and information that could be used in my workplace,” and “Very well explained and easy to understand.” In addition, comments around improved confidence to manage workplace mental health issues included “I found that it [HeadCoach] consolidated what I had covered previously in training and in doing so gave me more confidence to address an issue should one arise” and “[HeadCoach] gave me a bit more confidence in my approach.” The key negative feedback regarding the content related to the level of detail included about common mental illnesses (eg, “Having a family history with mental illness, I hoped this course may be more informative, it tends to be a little superficial or shallow in details of symptoms to look out for”). This feedback prompted the implementation of more detailed information into the subsequent version of HeadCoach with the inclusion of direct access to additional online resources.

    Effectiveness of HeadCoach

    Figure 5 displays that following their use of HeadCoach, managers exhibited significantly higher levels of self-reported confidence in communicating with employees regarding mental illness (t22=4.180; P<.001) and actions to employ managerial strategies to prevent and reduce stress among their team (t21=2.468; P=.02). In addition, we observed a significant increase in managers’ knowledge regarding their role in managing mental health issues (t23=2.881; P=.01). At the follow-up, no significant change was noted in nonstigmatizing attitudes toward mental illness (t23=–0.268; P=.80), whereas an increase in the levels of mental health knowledge fell just short of statistical significance (t24=1.987; P=.06).


    Principal Findings

    This pilot study assessed the feasibility, usability, and likely effectiveness of HeadCoach, a newly developed online training intervention for workplace managers regarding the mental health needs of employees reporting directly to them. To the best of our knowledge, HeadCoach is the first educational program for managers, which is delivered entirely online and provides an integrated program of preventative and reactive content consistent with recently recommended best practice frameworks for workplace mental health [4,15].

    In this pilot study, we collected relevant data to assess the feasibility of recruitment and participation processes and the usability of the program through self-reported measures and also objectively, as provided through the online research platform. This online platform has the potential to provide valuable information regarding levels of adherence and engagement [42]. The findings suggest that for HeadCoach, the online recruitment and registration processes, as well as methods for gathering data and delivering course material as operated through the current online research platform, were a practical and acceptable means for the dissemination and collection of various forms of information.

    This study also suggests that HeadCoach correlated with significant increases in the managers’ knowledge of their role in managing mental health issues among their staff, their confidence to do so, and their application of management strategies to promote a mentally healthy workplace. These changes in manager outcomes correspond to prior RCTs evaluating face-to-face training for managers promoting both reactive and preventative strategies to manage mental health issues within their team [22,26]. Although this study only evaluated the effect of HeadCoach with a pre–post design, the preliminary findings hold prominence because they suggest that e-learning options could replicate the positive outcomes previously reported with similar face-to-face manager training. This is an encouraging prospect because, if proven, it would allow organizations a practical means of training a large number of managerial staff about mental health issues with minimal time away from their jobs. Online training is a flexible and convenient format of learning because it can be structured to fit around other responsibilities and deadlines that managers are required to meet in their daily role. Although further evaluation through an RCT is warranted, these findings suggest the potential of HeadCoach to provide feasible, acceptable, and effective workplace mental health training for managers.

    Limitations

    Despite the novelty of our findings, there are a number of important limitations. As mentioned previously, the sample size was small with just over half of participants (51.3%; 20/39) completing the online intervention, and 56.4% (22/39) of the sample providing pre- and postintervention data. Although this dropout rate is typical in studies on online training [42,43] and survey responses [44], a higher response rate for both the adherence and follow-up would be ideal to minimize any potential bias caused by nonresponse. Besides the significance of personalized reminders [44], alternative engagement strategies may need to be adopted to help prompt adherence to the various stages of the trial, such as varying the messages contained within the email reminders and clearly conveying the purpose of participation [45]. In addition, strategies to maximize the response rates to the final questionnaire, such as the inclusion of a prize draw, may be valuable to increase the odds of participants responding [45].

    The generalizability of results from this pilot study is limited by the opt-in approach to recruitment. The sample of managers who agreed to participate and who completed the multiple components of the trial might have held a pre-existing awareness of mental health issues with an interest to further develop their skills to best support their employees. This may explain the lack of substantial change found in managers’ mental health knowledge because the sample may have already been well-informed of mental health issues and, therefore, a ceiling effect may have affected the outcomes of this study. Alternatively, the lack of marked effect may have been due to the lack of power, and with a larger sample size, a substantial change may have been observed for these outcomes. The generalizability of results may be limited by the sample of managers participating in this pilot study; the managers were from two specific industries, and nearly two-thirds (63.6%; 14/22) had been with their current employer for >15 years. An investigation within a variety of organizational contexts with a more representative sample is warranted before drawing conclusions about the feasibility and possible effects of this program for managers outside the building construction and emergency services industries.

    The absence of a control group further limits the conclusions that could be drawn regarding the efficacy of the intervention because other factors explaining the observed change cannot be ruled out. Another limitation was the short follow-up period included in this trial. Although we observed a marked change postintervention, conclusions about the persistence of intervention effects over time cannot be drawn because of the lack of an extended follow-up. The short evaluation period included in this study may have also been insufficient to allow managers the opportunity to display a change in attitude within their work environment, yet with the inclusion of a longer follow-up, a change in nonstigmatizing attitudes could have been observed. A prolonged evaluation period would allow for employee-level data to be obtained for assessing the impact of manager training as perceived by the employees, providing valuable objective information about the flow on effects of manager training for the staff they supervise. The final potential limitation to note is the collection of self-reported data, especially regarding confidence levels. Participants may be more likely to report their levels of confidence or behavior favorably, although the use of identical questions at both time points and the anonymity of an online survey should have decreased this risk.

    Conclusions

    Although testing in a more comprehensive study with the inclusion of a control group is required to demonstrate a true effect, the preliminary findings from this pilot study suggest that HeadCoach may provide a practical and efficient method of training managers in best workplace mental health practices. Given the key role managers play in the promotion of mental health and well-being within their teams, there is great potential for this type of training to improve the support provided to employees for their mental health needs in the future.

    Acknowledgments

    This study was funded by beyondblue with donations from the Movember Foundation (Project Code: LK: 7139). Additional funding was provided by the icare foundation and the Mental Health Branch of NSW Health.

    Authors' Contributions

    AG and SBH conceptualized and designed the study. AG, ADL, AM, and SBH designed the intervention. AG acquired the data and with SBH, conducted the statistical analyses. AG and SBH drafted the manuscript. All coauthors contributed to the final manuscript.

    Conflicts of Interest

    HC and SBH are employed by the Black Dog Institute which provides manager training to workplaces.

    ‎

    Multimedia Appendix 1

    Supplementary File 1 Vignettes for Summary Exercises.

    PDF File (Adobe PDF File), 147KB
    ‎

    Multimedia Appendix 2

    Supplementary File 2 Questionnaire.

    PDF File (Adobe PDF File), 62KB
    1. Harvey SB, Henderson M, Lelliott P, Hotopf M. Mental health and employment: much work still to be done. Br J Psychiatry. Mar 2009;194(3):201-203. [CrossRef] [Medline]
    2. McLellan RK, Pransky G, Shaw WS. Disability management training for supervisors: a pilot intervention program. J Occup Rehabil. Mar 2001;11(1):33-41. [Medline]
    3. Tsutsumi A. Development of an evidence-based guideline for supervisor training in promoting mental health: literature review. J Occup Health. 2011;53(1):1-9. [FREE Full text] [Medline]
    4. Petrie K, Joyce S, Tan L, Henderson M, Johnson A, Nguyen H, et al. A framework to create more mentally healthy workplaces: A viewpoint. Aust N Z J Psychiatry. Jan 2018;52(1):15-23. [CrossRef] [Medline]
    5. Harvey SB, Modini M, Joyce S, Milligan-Saville JS, Tan L, Mykletun A, et al. Can work make you mentally ill? A systematic meta-review of work-related risk factors for common mental health problems. Occup Environ Med. Mar 2017;74(4):301-310. [CrossRef] [Medline]
    6. Harvey S, Henderson M. Occupational psychiatry. Psychiatry. May 2009;8(5):174-178. [FREE Full text] [CrossRef]
    7. Henderson M, Harvey SB, Overland S, Mykletun A, Hotopf M. Work and common psychiatric disorders. J R Soc Med. May 2011;104(5):198-207. [FREE Full text] [CrossRef] [Medline]
    8. van Zoonen K, Buntrock C, Ebert DD, Smit F, Reynolds CF, Beekman ATF, et al. Preventing the onset of major depressive disorder: a meta-analytic review of psychological interventions. Int J Epidemiol. Apr 2014;43(2):318-329. [FREE Full text] [CrossRef] [Medline]
    9. Harvey SB, Deady M, Wang M, Mykletun A, Butterworth P, Christensen H, et al. Is the prevalence of mental illness increasing in Australia? Evidence from national health surveys and administrative data, 2001-2014. Med J Aust. Jun 19, 2017;206(11):490-493. [Medline]
    10. Knudsen AK, Øverland S, Aakvaag HF, Harvey SB, Hotopf M, Mykletun A. Common mental disorders and disability pension award: seven year follow-up of the HUSK study. J Psychosom Res. Jul 2010;69(1):59-67. [CrossRef] [Medline]
    11. Knudsen AK, Harvey SB, Mykletun A, Øverland S. Common mental disorders and long-term sickness absence in a general working population. The Hordaland Health Study. Acta Psychiatr Scand. Apr 2013;127(4):287-297. [CrossRef] [Medline]
    12. Harvey SB, Glozier N, Henderson M, Allaway S, Litchfield P, Holland-Elliott K, et al. Depression and work performance: an ecological study using web-based screening. Occup Med (Lond). May 2011;61(3):209-211. [CrossRef] [Medline]
    13. Modini M, Joyce S, Mykletun A, Christensen H, Bryant RA, Mitchell PB, et al. The mental health benefits of employment: Results of a systematic meta-review. Australas Psychiatry. Aug 2016;24(4):331-336. [CrossRef] [Medline]
    14. Cocker F, Sanderson K, LaMontagne AD. Estimating the Economic Benefits of Eliminating Job Strain as a Risk Factor for Depression. J Occup Environ Med. Jan 2017;59(1):12-17. [CrossRef] [Medline]
    15. LaMontagne AD, Martin A, Page KM, Reavley NJ, Noblet AJ, Milner AJ, et al. Workplace mental health: developing an integrated intervention approach. BMC Psychiatry. May 09, 2014;14:131. [FREE Full text] [CrossRef] [Medline]
    16. Joyce S, Modini M, Christensen H, Mykletun A, Bryant R, Mitchell PB, et al. Workplace interventions for common mental disorders: a systematic meta-review. Psychol Med. Mar 2016;46(4):683-697. [CrossRef] [Medline]
    17. Health Safety Executive. How to tackle work-related stress: A guide for employers on making the Management Standards work. www.hse.gov.uk. pdf. Health and Safety Executive, Executive HaS; 2009:430.
    18. Nishiuchi K, Tsutsumi A, Takao S, Mineyama S, Kawakami N. Effects of an education program for stress reduction on supervisor knowledge, attitudes, and behavior in the workplace: a randomized controlled trial. J Occup Health. May 2007;49(3):190-198. [FREE Full text] [Medline]
    19. Nieuwenhuijsen K, Verbeek JHAM, de Boer AGEM, Blonk RWB, van Dijk FJH. Supervisory behaviour as a predictor of return to work in employees absent from work due to mental health problems. Occup Environ Med. Oct 2004;61(10):817-823. [FREE Full text] [CrossRef] [Medline]
    20. Chartered Institute of Personnel and Development. Report No 5244. London. CIPD; Jun 2010. URL: https:/​/www.​cipd.co.uk/​Images/​manager-support-for-return-to-work-following-long-term-sickness-absence_2010_tcm18-10818.​pdf [accessed 2018-06-13] [WebCite Cache]
    21. Harvey S, Joyce S, Tan L, Johnson A, Nguyen H, Modini M, et al. Mental Health Commission, Australian Government. Australia. National Mental Health Commission and the Mentally Healthy Workplace Alliance; Nov 2014. URL: http:/​/www.​mentalhealthcommission.gov.au/​media/​116414/​Developing%20a%20mentally%20healthy%20workplace_Final%20November%202014.​docx [accessed 2018-06-13] [WebCite Cache]
    22. Milligan-Saville J, Tan L, Gayed A, Barnes C, Madan I, Dobson M, et al. Workplace mental health training for managers and its effect on sick leave in employees: a cluster randomised controlled trial. The Lancet Psychiatry. Nov 2017;4(11):850-858. [FREE Full text] [CrossRef]
    23. Takao S, Tsutsumi A, Nishiuchi K, Mineyama S, Kawakami N. Effects of the job stress education for supervisors on psychological distress and job performance among their immediate subordinates: a supervisor-based randomized controlled trial. Journal of occupational health Internet. Nov 2006;48(6):494-503. [FREE Full text]
    24. Tsutsumi A, Takao S, Mineyama S, Nishiuchi K, Komatsu H, Kawakami N. Effects of a supervisory education for positive mental health in the workplace: a quasi-experimental study. J Occup Health. May 2005;47(3):226-235. [FREE Full text] [Medline]
    25. Moffitt J, Bostock J, Cave A. Promoting well-being and reducing stigma about mental health in the fire service. Journal of Public Mental Health. 2014;13(2):103-113. [FREE Full text]
    26. Dimoff JK, Kelloway EK, Burnstein MD. Mental health awareness training (MHAT): The development and evaluation of an intervention for workplace leaders. International Journal of Stress Management. 2016;23(2):167-189. [CrossRef]
    27. Shann C, Martin A, Chester A, Ruddock S. Effectiveness and Application of an Online Leadership Intervention to Promote Mental Health and Reduce Depression-Related Stigma in Organizations. J Occup Health Psychol. Jan 04, 2018. [CrossRef] [Medline]
    28. Gayed A, Milligan-Saville JS, Nicholas J, Bryan BT, LaMontagne AD, Milner A, et al. Effectiveness of training workplace managers to understand and support the mental health needs of employees: a systematic review and meta-analysis. Occup Environ Med. Mar 21, 2018. [CrossRef] [Medline]
    29. Kawakami N, Takao S, Kobayashi Y, Tsutsumi A. Effects of web-based supervisor training on job stressors and psychological distress among workers: a workplace-based randomized controlled trial. Journal of Occupational Health. 2006;48(1):28-34. [FREE Full text]
    30. Stansfeld S, Berney L, Bhui K, Chandola T, Costelloe C, Hounsome N, et al. Pilot study of a randomised trial of a guided e-learning health promotion intervention for managers based on management standards for the improvement of employee well-being and reduction of sickness absence: the GEM (Guided E-learning for Managers) study. Public Health Research. 2015;3(9). [CrossRef]
    31. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. Sep 29, 2008;337(sep29 1):a1655-a1655. [CrossRef]
    32. LaMontagne AD, Milner AJ, Allisey AF, Page KM, Reavley NJ, Martin A, et al. An integrated workplace mental health intervention in a policing context: Protocol for a cluster randomised control trial. BMC Psychiatry. Feb 27, 2016;16:49. [FREE Full text] [CrossRef] [Medline]
    33. Kawakami N, Kobayashi Y, Takao S, Tsutsumi A. Effects of web-based supervisor training on supervisor support and psychological distress among workers: a randomized controlled trial. Prev Med. Aug 2005;41(2):471-478. [FREE Full text] [CrossRef] [Medline]
    34. Gayed A, Bryan BT, Petrie K, Deady M, Milner A, LaMontagne AD, et al. A protocol for the HeadCoach trial: the development and evaluation of an online mental health training program for workplace managers. BMC Psychiatry. Dec 29, 2018;18(1):25. [FREE Full text] [CrossRef] [Medline]
    35. Sweet SN, Fortier MS, Strachan SM, Blanchard CM, Boulay P. Testing a Longitudinal Integrated Self-Efficacy and Self-Determination Theory Model for Physical Activity Post-Cardiac Rehabilitation. Health Psychol Res. Jan 13, 2014;2(1):1008. [FREE Full text] [CrossRef] [Medline]
    36. LaMontagne A, Milner AJ, Allisey AF, Page KM, Reavley NJ, Martin A, et al. An integrated workplace mental health intervention in a policing context: Protocol for a cluster randomised control trial. BMC Psychiatry. Feb 27, 2016;16:49. [FREE Full text] [CrossRef] [Medline]
    37. Yarker J, Lewis R, Donaldson-Feilder E. Management competencies for preventing and reducing stress at work: Identifying and developing the management behaviours necessary to implement the HSE Management Standards: Phase Two. Health and Safety Executive. Jun 2008:1-95. [FREE Full text]
    38. Evans-Lacko S, Little K, Meltzer H, Rose D, Rhydderch D, Henderson C, et al. Development and psychometric properties of the Mental Health Knowledge Schedule. Can J Psychiatry. Jul 2010;55(7):440-448. [CrossRef] [Medline]
    39. Greenberg N, Gould M, Langston V, Brayne M. Journalists' and media professionals' attitudes to PTSD and help-seeking: A descriptive study. Journal of Mental Health. Dec 16, 2009;18(6):543-548. [CrossRef]
    40. Griffiths K, Crisp D, Jorm A, Christensen H. Does stigma predict a belief in dealing with depression alone? J Affect Disord. Aug 2011;132(3):413-417. [CrossRef] [Medline]
    41. Griffiths KM, Christensen H, Jorm AF, Evans K, Groves C. Effect of web-based depression literacy and cognitive-behavioural therapy interventions on stigmatising attitudes to depression: randomised controlled trial. Br J Psychiatry. Oct 2004;185:342-349. [FREE Full text] [CrossRef] [Medline]
    42. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res. 2009;11(2):e13. [FREE Full text] [CrossRef] [Medline]
    43. Fear NT, Jones M, Murphy D, Hull L, Iversen AC, Coker B, et al. What are the consequences of deployment to Iraq and Afghanistan on the mental health of the UK armed forces? A cohort study. Lancet. May 22, 2010;375(9728):1783-1797. [CrossRef] [Medline]
    44. Cook C, Heath F, Thompson RL. A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys. Educational and Psychological Measurement. Dec 01, 2000;60(6):821-836. [CrossRef]
    45. Sauermann H, Roach M. Increasing web survey response rates in innovation research: An experimental study of static and dynamic contact design features. Research Policy. Feb 2013;42(1):273-286. [FREE Full text] [CrossRef]

    ‎
    GEM: Guided e-learning for managers
    MHAT: Mental health awareness training
    MRC: Medical Research Council
    RCT: Randomized controlled trial
    HSE: Health and Safety Executive

    Edited by G Eysenbach; submitted 28.03.18; peer-reviewed by J Wang, Q Yuan, S Pezaro; comments to author 09.05.18; revised version received 22.05.18; accepted 24.05.18; published 03.07.18.

    Copyright

    ©Aimée Gayed, Anthony D. LaMontagne, Allison Milner, Mark Deady, Rafael A. Calvo, Helen Christensen, Arnstein Mykletun, Nick Glozier, Samuel B Harvey. Originally published in JMIR Mental Health (http://mental.jmir.org), 03.07.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.