Published on in Vol 9, No 5 (2022): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/34655, first published .
Acceptability of Web-Based Mental Health Interventions in the Workplace: Systematic Review

Acceptability of Web-Based Mental Health Interventions in the Workplace: Systematic Review

Acceptability of Web-Based Mental Health Interventions in the Workplace: Systematic Review

Review

1Department of Psychosis, Institute of Psychiatry, Psychology and Neuroscience, School of Academic Psychiatry, King’s College London, London, United Kingdom

2Maudsley Learning, South London & Maudsley NHS Foundation Trust, London, United Kingdom

Corresponding Author:

Johanna Scheutzow, BSc, MSc

Department of Psychosis

Institute of Psychiatry, Psychology and Neuroscience, School of Academic Psychiatry

King’s College London

16 De Crespigny Park

London, SE5 8AB

United Kingdom

Phone: 44 020 7848 0002

Email: johanna.scheutzow2512@gmail.com


Background: Web-based interventions have proven to be effective not only in clinical populations but also in the occupational setting. Recent studies conducted in the work environment have focused on the effectiveness of these interventions. However, the role of employees’ acceptability of web-based interventions and programs has not yet enjoyed a similar level of attention.

Objective: The objective of this systematic review was to conduct the first comprehensive study on employees’ level of acceptability of web-based mental health interventions based on direct and indirect measures, outline the utility of different types of web-based interventions for work-related mental health issues, and build a research base in the field.

Methods: The search was conducted between October 2018 and July 2019 and allowed for any study design. The studies used either qualitative or quantitative data sources. The web-based interventions were generally aimed at supporting employees with their mental health issues. The study characteristics were outlined in a table as well as graded based on their quality using a traffic light schema. The level of acceptability was individually rated using commonly applied methods, including percentile quartiles ranging from low to very high.

Results: A total of 1303 studies were identified through multiple database searches and additional resources, from which 28 (2%) were rated as eligible for the synthesis. The results of employees’ acceptability levels were mixed, and the studies were very heterogeneous in design, intervention characteristics, and population. Approximately 79% (22/28) of the studies outlined acceptability measures from high to very high, and 54% (15/28) of the studies reported acceptability levels from low to moderate (overlap when studies reported both quantitative and qualitative results). Qualitative studies also provided insights into barriers and preferences, including simple and tailored application tools as well as the preference for nonstigmatized language. However, there were multiple flaws in the methodology of the studies, such as the blinding of participants and personnel.

Conclusions: The results outline the need for further research with more homogeneous acceptability studies to draw a final conclusion. However, the underlying results show that there is a tendency toward general acceptability of web-based interventions in the workplace, with findings of general applicability to the use of web-based mental health interventions.

JMIR Ment Health 2022;9(5):e34655

doi:10.2196/34655

Keywords



Background

There is an increasing level of awareness regarding the importance of health and well-being in the workplace [1]. Anxiety, stress, and depression are the dominant mental health issues for workers in the United Kingdom, with a prevalence of 1320 cases per 100,000 workers, causing close to 18 million lost working days per year [2]. Employers have a responsibility to take care of their employees and provide support for both their physical and mental health [3,4].

Web-based mental health interventions are increasingly being used in the work environment as they have the advantage of being cost-effective, efficient, anonymous, location-independent, flexible, and empowering. They are regularly used for both prevention and intervention [5-10].

Web-based interventions also have multiple flaws, including technical difficulties, ethical concerns, increased attrition rates, and low engagement in the absence of guidance by professional support [6,11]. Therefore, it is important to understand the barriers that reduce engagement and acceptability of web-based interventions. Multiple systematic studies provide evidence of the effectiveness of web-based mental health interventions at work [12,13]. Importantly, they outline the need to tailor interventions to populations’ needs, which requires greater insight into barriers and the acceptability of web-based mental health interventions in the workplace.

The acceptability of an intervention includes users’ emotional and cognitive responses to the intervention [14], including affective perceptions, burden and barriers, perceived benefits, understanding of the intervention, opportunity costs, and usability. In practice, this takes into account the individuals’ preferences for features and tools, their willingness to use web-based interventions, their engagement (eg, dropout and attrition rate), and users’ perceived utility or satisfaction with the intervention.

Studying users’ acceptability of new treatments has ethical, methodological (validity), and practical applications [15]. Specifically, ethical obligations include the exploration of reasons for acceptable or unacceptable treatments as perceived by the users. It is important to understand potential barriers to intervention engagement before introducing the intervention to employees. Awareness of intervention efficacy alone does not mean that employees accept web-based interventions as a useful tool for self-help.

Sekhon et al [14] outlined studies assessing interventions’ acceptability by using operational definitions in line with measurable acceptability data (dropout rate and satisfaction rating) and qualitative studies focusing on in-depth user experiences. Current research has been limited to studies on clinical populations. Clinical populations differ significantly in symptom severity, level of risks, functionality, and response to treatment; thus, the results might not be generalizable to occupational populations [13]. Therefore, it is relevant to explicitly assess employees’ acceptability of web-based interventions.

Objectives

This systematic review aimed to assess employees’ acceptability of web-based interventions to improve their mental health. The study aimed to inform intervention design and utility by evaluating user experience and barriers and facilitators to using web-based mental health interventions in the workplace.


This systematic review was conducted in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [16] and followed the ENTREQ (Enhancing the Transparency in Reporting the Synthesis of Qualitative Research) guidelines [17,18].

Eligibility Criteria

Eligible studies met the Population, Intervention, Comparison, Outcome criteria and included qualitative interviews, quantitative studies including scale measures of satisfaction and forms of attrition rates, and mixed methods studies. Acceptability was assessed by means of both direct (acceptability, satisfaction, and experience) and indirect (compliance, completion, adherence, attrition, and dropout rate) measures. Studies were included if they were available in English and published after 2005.

Population

The population was narrowed down to people aged ≥18 years as there is a difference between child and adult interventions. The participants could be employed part-time or full-time or self-employed. Studies were included if the participants were >60% employed. This threshold guaranteed that the main outcome could be generalized to the eligible population for the study’s purpose.

Intervention

Following the guidance of the meta-review by Joyce et al [19] on general workplace mental health interventions, web-based interventions were kept broad to include those that were conducted at work, had a work-related component, or aimed to treat work-related risk factors (eg, stress, depression, or anxiety). However, eligible interventions had to be exclusively web-based programs or interventions that targeted employed people or were applied in an occupational setting. Interventions or programs could be delivered via a computer program, app, or website. They could also differ in the device used to deliver the content (computer, laptop, or mobile phone) as well as include various forms of multimedia. All interventions aimed to change employees’ behavior or mental health. They could have the aim of preventing, treating, or rehabilitating mental health issues.

Comparison

This review compared randomized controlled trials (RCTs), nonrandomized comparative trials, noncomparative trials, explorative studies, and qualitative studies published between 2005 and 2019.

Outcome

Studies were included if they measured acceptability directly or indirectly by means of qualitative assessment of acceptability, satisfaction, and experience or the indirect measure of acceptability through compliance, completion, adherence, attrition, or dropout rate. Studies were included that assessed the potential willingness to use interventions or the potential features of interventions that were preferred or addressed as disadvantageous for utility.

Exclusion Criteria

Studies were excluded if they did not meet the Population, Intervention, Comparison, Outcome criteria; that is, if they included guidance through coaches, therapists, or face-to-face interactions and were applied to participants who were retired or unemployed (>40%). In addition, studies were excluded if they did not measure acceptability or willingness of use as an outcome variable or used interventions that were not focused on the users’ mental health.

Data Sources

The search was conducted in July 2019 and included the following electronic databases: PsycINFO (Ovid), Embase (Ovid), MEDLINE (Ovid), Global Health (Ovid), and the Cochrane Library Trials (CENTRAL). Backward searching was used to ensure that no key papers were missed.

Search Strategy

Databases were searched for studies published between 2005 and 2019. Duplicates were removed (Ovid search option). The Boolean system was applied using AND and OR (Textbox 1) to combine different terminologies of 4 key concepts included in a free-text and keyword search. Specific occupations were added to the general search of employees to increase the likelihood of finding studies on high stress–exposed work settings; for example, in the military or firefighter professions. The search terms used were categorized as occupational settings (employee), web-based interventions, mental ill health, and acceptability of interventions. All the key terms considered American and British spelling.

Search terms organized into 4 concepts.

Concepts (combined using AND) and corresponding search terms (combined using OR)

  • Employee: employment, job, work, worker*, workplace, occupation*, employee*, manager*, line-manager*, staff, military, fire-fighter*, police, emergenc*, business, organisational*, work related, personnel
  • Web-based intervention: video-therap*, mobile therap*, computerised cbt, ccbt, digital therap*, e-mental health, e-health, ehealth, computerised* therap*, internet-based intervention, occupational e-therap*, occupational e-mental health, e-therap*, web-based therap*, internet based therap*, online* therap*, tele-medicine, tele-therap*, tele-psychiatry, tele-psycholog*, computer-assisted therap*, electronic intervention, smartphone intervention online, psychological*, app
  • Mental ill health: stress, mental health, mental illness, mental disorder, depress*, anxiety, affective symptoms, burnout, resilience wellbeing, workplace wellbeing, ptsd, trauma, acute stress disorder, return to work, psychological*, sick* leave*, sick* day*, sick* absen*, absenteeism, emotional stress, interpersonal stress, life stress, mental stress, chronic stress, job stress
  • Acceptance: accept*, willing*, open*, attitude, feasab*, satisfaction compliance, reasons for drop out, drop-out, utilisation, adherence, take-up rates, take up rates, patients drop-out rate
Textbox 1. Search terms organized into 4 concepts.

Study Selection

Duplicates were removed, and titles, abstracts, and full texts were scanned for the inclusion criteria. After the assessment of the full texts’ eligibility by the first author (JS), all the included studies were summarized and synthesized. The study selection process is outlined in the PRISMA flowchart (Figure 1).

Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.
View this figure

Data Collection Process

Data were collected according to the following criteria: reference, characteristics of the intervention, its aims and objectives, study design, population, setting and recruitment, results, acceptability, and—if available—reasons for dropout, as well as qualitative data.

Quality Assessment

The quality of the studies was assessed using the Critical Appraisal Skills Programme (CASP) checklist [20] for both qualitative studies and RCTs. Studies were evaluated based on research design, representativeness, recruitment procedure, presence of a comparison group, dropout rate, validity, reliability, and relevance of the measurement tools. Quality was graded with a traffic light system based on the 10 quality questions of the CASP and answered with yes (green) if the information was present, no information (red), and not available (yellow) if the information was not apparent or clearly outlined within the study (Multimedia Appendix 1 [21-48]). Studies were included in this systematic review irrespective of quality judgment.

Synthesis of Data

Similar to corresponding systematic reviews on the acceptability of web-based interventions [10], the level of acceptability was categorized into the following quartiles: low (− −), moderate (−), high (+), and very high (+ +). This was specifically used for studies that reported a satisfaction rating on a scale, the percentage results of which could then be transferred to the suggested levels of acceptability. In addition, studies reporting dropout rates and compliance percentages were organized according to the 4-quartile rating system for acceptability. If studies reported mixed results, including positive and negative outcomes on different acceptability factors, they were rated with a tilde (~). Qualitative studies were synthesized in an integrative, meta-aggregative style following methodological guidance on the use of meta-aggregations [49] as well as using similar systematic reviews [50]. The key data were extracted and are outlined in Table 1 (see Multimedia Appendix 2 [31-33,35,38,39,45-48] for direct measures and qualitative data sources).

Table 1. Study characteristicsa.
Author and countryIntervention, duration, and aimDesign and recruitmentPopulationResultsAcceptance measureReasons for dropoutLevel (acceptance)



Sample size, NAge (years)GenderEmployment details



Abbott et al [42], AustraliaInternet-delivered CBTb program for employees with tinnitus distress in industrial organizations; 6 weeks; effectiveness of the programClustered RCTc comparing CBT intervention group with IOCd; recruited in industrial organizations (BP Australia and BHP Billiton)56CBT: mean 50.5 (SD 9.5); IOC: mean 48.7 (SD 8.6)CBT: 96% men; IOC: 82% meneCBT program was similarly effective to the information program for treating tinnitus distress, depression, anxiety, stress, and quality of lifeAttrition rate: 50% (CBT: 70%); satisfaction: 73.4% (mean 5.14/7)Unknown~f
Allexandre et al [21], United StatesWeb-based interactive educational stress management program (website), Stress Free Now, using mindfulness meditation; 8 weeks; effectiveness of the interventionRCT comparing 4 groups, including no support, group support, group and expert clinical support, and waitlist control; recruitment via email in a corporate call center161Mean 40.0 (SD 12.6)83.2% women49.1% full-time work shift (days); debt collectors and customer service or fraud representativesParticipants favored guided practices and showed low use of program. All groups decreased in perceived stress and improved in psychological and emotional well-beingWeb-based use: 10% to 15% (intervention)Lack of time− −g
Beiwinkel et al [36], GermanyWeb-based program, HelpID, for depression based on CBT and awareness training; 12 weeks; effectiveness of HelpID in reducing sickness absence and depressionRCT comparing intervention with control; recruitment via health insurance180Mean 4868% women51% full-time workHelpID effectively reduced depressive symptomsDropout: 45.5% after the assessment, 67.7% follow-up; satisfaction: 68.2% (mean 2.04, intervention)Relationship between age (older) and education (higher) and dropout rates (lower)h
Birney et al [37], United StatesMobile app intervention MoodHacker, CBT-based depression self-management; 6 to 10 weeks; effectiveness of a program to reduce stress and prevent depression, anxiety, and substance abuse among employeesRCT: MoodHacker group compared with alternative care with links to websites on depression; recruited via EAPsi and other outreaches300MoodHacker: mean 40.6 (SD 11.5); alternative care: mean 40.7 (SD 11.2)MoodHacker: 74.6% women; alternative care: 78.7% women56% full-time, 35.3% part-time, and 8.7% self-employedMoodHacker caused significant effects on depression symptoms compared with alternative careAttrition: 6.7% follow-up; satisfaction: 76% (mean 4.6/6); system usability: B+Unknown+ +j
Billings et al [22], United StatesWeb-based stress and mood management multimedia program for employees based on CBT; 12-week access; effectiveness of the program to reduce depression and increase behavioral activation, knowledge of depression, and performance at workRCT: experimental and control; recruited from a technology company via email and health fair309Most (51%) between 30 and 4070.6% womenDecrease in stress, increase in knowledge of anxiety and depression as well as positive perception of treatment and improvement in the consumption of alcohol; most used it only onceRatings (0-5): 71% useful (mean 3.55); interesting: M 3.47; appealing: M 3.34; motivating: M 3.21Participants who knew how to handle mood and stress at baseline had an increased likelihood of finishing the study+k
Bolier et al [23], the NetherlandsWeb-based health promotion programs (Colour Your Life, Don\'t Panic Online, Drinking Less, Psyfit, and Strong at work) designed for the work setting aiming to decrease stress and prevent substance abuse, depression, and anxiety in health professionals; 6 to 12 weeks; measure effectiveness of the modulesClustered RCT: web-based condition and waitlist control; recruited nurses and health professionals in a medical center via mail1140Mean 4079.8% women71.9% nursesThe intervention significantly enhanced positive mental healthUptake rate and compliance: 16% logged in, 5% started; dropout: 60.7% (intervention), 44% overallAge predicted dropout (the younger the participants, the more likely they were to drop out)− −
Ebert et al [40], GermanyWeb-based unguided recovery training, GET.ON, for teachers with insomnia and psychological detachment from the workplace; 6 weeks; psychological efficacy of GET.ONRCT: intervention and waitlist control group; recruited via email at schools by the Ministry of Education (Germany, NRWl)64Mean 48.5 (SD 9.9)74.2% womenSignificant reduction in insomnia severityCompletion rate: 48.4% all sessions38.5% technical problems, lack of time or motivation, disputed usefulness, or did not see any more benefit in using the program further before the final module; others did not report reasons
Ebert et al [24], GermanyUnguided web-based stress management program, GET.ON Stress, for employees using problem solving and emotional regulation; 7 weeks; efficacy of the programRCT: intervention or waitlist control; recruited from general employees via the occupational health program of a health insurance company as well as via contacted HRm departments in Germany264Mean 4272% women75% full-time; diverse sectors including economy, health, service, and socialEffectively reduced symptoms of mental and work-related stress among employees with stressAttrition rate: 42% (7 sessions); dropout: 90% provided follow-up data; satisfaction (high): 95% overall+ +
Hamamura et al [25], JapanComputer-delivered intervention (app), Self-Record, that facilitates cognitive restructuring for distress and alcohol consumption through self-monitoring of thought and activities; 4 weeks; effectiveness of the intervention on mental distress and consumption of alcoholPilot non-RCT, quasi-experiment with intervention and control groups; recruited via research marketing company557Mean 38.82 (SD 9.58)41.2% women71.6% employed by a company, 7.5% employed by the government or a nonprofit organization, 6.3% self-employed, and 3.1% professionalsIntervention heightened participants’ perception of their pathological thoughts and alcohol consumption, whereas they only decreased face to faceDropout rate: 15.3% follow up; adherence: 64.8% (intervention) stopped after the first day
Heber et al [26], GermanyStress website, GET.ON, including psychoeducation and interactive exercises tailored through personalized feedback; 4 weeks; efficacy of the interventionRCT: intervention and waitlist control group; recruited by the Ministry of Education from the general working population showing symptoms of stress and through newspaper articles264Mean 43.3 (SD 10.2)73.1% women77.3% full-timeWeb-based interventions effectively decreased stress in employeesCompletion rate: 70.05% all sessions; satisfaction (high): 92.2%Time constraints (4/9), motivation constraints (3/9), technical difficulties (1/9), and dissatisfaction with the intervention (1/9)+/++
Ketelaar et al [27], the NetherlandseMHn interventions for health professionals—Psyfit, Strain at work, Colour your life, Don’t panic online, and Drinking less: self-help on the internet (CBT and other); evaluate eMH approach targeting work functioning and psychological well-beingRCT, randomization at ward level with intervention and control groups; recruited nurses and health professionals employed at an academic hospital1140Mean 39.580% meneMH approach was not more effective than a control to increase work functioning and psychological well-beingCompliance rate: 6% started the intervention; dropout: 45% to follow-upYounger participants were more likely to drop out; technical problems− −
Ly et al [28], SwedenMobile phone stress management intervention for managers including short audio lectures, information, and exercise focusing on acceptance and commitment therapy; 6 weeks; efficacy of the smartphone treatmentRCT: stress intervention and waitlist control group; recruitment took place after a presentation about the project at multiple organizations (Swedish or American) and via advertisements on the internet73Mean 41.5 (SD 7.2)57.5% menIntervention reduced stress and increased general health among managersAdherence: 44.4%
Nevedal et al [43], United StatesDigital health coaching program for chronic pain management using psychoeducation on self-management, coping, and stress; 4 weeks; effectiveness of the program on work interference, activity, stress, pain, quality of life, and healthCase report; 1-group design; recruited via mailings, emails, and posted communications within 37 American organizations or a member of 1 of 18 health care plans645Mean 56.16 (SD 12.83)69.3% womenSupported effectiveness of interventions on pain (1 and 6 months after treatment) as well as quality of life (after 6 months)Satisfaction (good or better): 82.6%+ +
Feicht et al [44], GermanyHappiness exercises to develop a positive psychological state; 7 weeks; examined the impact of the intervention on psychological and physiological parametersLongitudinal design (2 groups—intervention and control); recruited via local insurance company in Germany (2 participating departments were chosen by the company)142Mean 37 (SD 7.7)68.8% womenHappiness, satisfaction, mindfulness, and quality of life improved; stress decreased; and recovery experience increased significantlyDropout rate: 31.3% totalDifference in age in intervention and control groups (10 years)+
Thiart et al [29], GermanyInternet-based CBT-Io intervention, GET.ON Recovery, for stress, work-related strain, and insomnia in teachers; 6 weeks; evaluate the efficacy of the interventionRCT: intervention group and waitlist control; recruited via email sent to schools in Germany128Mean 48.0 (SD 9.9)74.2% women100% school teachersThe intervention reduced sleep difficulties and fostered psychological detachment from workCompletion rate: 95.3%; satisfaction: 91% would recommend it+ +
Umanodan et al [30], JapanComputer-based stress management training using self-paced behavioral, communication, and cognitive techniques; 7 weeks; effectiveness of the program in improving mental health and performance at workClustered RCT; recruited via informational posters and the supervisor during meetings in a manufacturing company263Mean 38.8592.6% men23% managersKnowledge about stress management and coping skills increased (if participants had enough time)Completion rate (intervention group): 89%High baseline levels of distress increased the chance of dropout+ +
Wood et al [41], United StatesResilience mobile app to decrease burnout (assessment tools); 4 weeks; assess usability, acceptability, and effectivenessPilot study; recruited mental health care professionals from a health care system30Mean 42.5 (SD 12)43% psychologists, 30% social workers, 13% psychiatric nurses, 7% psychiatrists, and 7% otherApp reduced burnout and compassion fatigue in participantsSystem usability (overall): 79.4%+ +
Bush et al [31], United StatesT2 Mood Tracker mobile app to track symptoms associated with deployment-related behavioral health issues (well-being, anxiety, stress, PTSDp, injury, and depression); 1.4 weeks; assessment of the utility of the appMixed methods design: qualitative and quantitative (Likert-style and open-ended questions); recruited via posters and flyers distributed by WTUq862% menClinical social work staffThe app was perceived as easy to use, helpful, and beneficialUseful rating: 88%; qualitative: utility rating positive but could incorporate additional factors to make it more manifold2 participants did not want to download the app (privacy concerns were assumed)+ +
Carolan et al [32], United KingdomWeb-based stress management intervention, WorkGuru (CBT, mindfulness, and problem solving); 8 weeks; employees’ attitude toward digital mental health interventions at workQualitative study: 18 semistructured interviews (taken from previous RCT with and without access to a web-facilitated discussion group); recruited from 6 UK-based organizations and invited via mail (universities, local authorities, third sector, and telecommunications)18 (based on the sample N=82)Mean 4578% women78% office work and 22% mixture of office and client workOutlined advantages of digital mental health interventions, but high barriers appeared with the application in the workplaceEngagement: 39%; qualitative: preference for short, interactive, easy to use, personalized, and anonymous interventions and access via computer or mobile phone
Deady et al [48], AustraliaEmergency service workers’ attitude toward mobile mental health appsCross-sectional study; recruited from 4 metropolitan Fire and Rescue stations106Mean 37.8 (SD 9.51)88% menFirefightersParticipants showed positive perception and interest in using mental health apps but had preferences regarding language, features, and therapeutic techniquesDivided interests in using a mental health app. Apps should avoid stigmatized terminology and focus on well-being, mental fitness, resilience, stress, lifestyle, and sleep by implementing attractive multimedia features+
Deady et al [38], AustraliaAcceptance and effectiveness study on HeadGear, an app-based program aiming to decrease depressive symptoms and increase well-being; 5 weeks2-stage pilot study; recruited via email and Facebook from industrial organizations (agriculture, freight or postage, and mining)Stage 1: 21; stage 2: 84Stage 1: mean 37.86 (SD 10.98); stage 2: mean 38 (SD 9.23)Stage 1: 50% women; stage 2: 100% menStage 1: most worked in freight and postage (n=11); stage 2: male-dominated industryHeadGear was effective and reduced symptoms significantly. However, attrition rate was highUtility: 40% to 50% would use it; qualitative: most appreciated the utility, helpfulness, overall ease, and accessibility but complained about engagement and navigation issues
Eklund et al [33], SwedenUniversity staff’s experiences of a customized, interactive, web-based program that aims to change behavior in stress management as well as explore intervention adjustmentsExplorative qualitative study: semistructured interviews; recruitment via 3 departments at the university9Mean 45.9University staffStaff accepted a web-based program for stress-related problemsAcceptance was positive as long as it was short in time and applied in a transparent and tailored way+
Hennemann et al [34], GermanyEmployees’ acceptance of organizational eMH interventions focusing on work-related distressLongitudinal cohort study: self-administered questionnaire; recruited employees showing health problems and previous sickness absence1829Mean 49.93 (SD 4.06)Attitudes toward organizational eMH interventions were disadvantageousAcceptance (low): 89.1%; suggestions for improvement of acceptance: previous education (awareness and attitudes regarding efficacy and usability)Higher scores in men and high-education group, those with previous experience with eHealth, and mentally demanding work types; lower scores in those diagnosed with a mental health disorder and non–internet users− −
Peters et al [45], AustraliaExplorative workshop of perceptions, thoughts, and preferences of employees in male-dominated workplaces to build and adapt a mental health mobile appExploratory qualitative study; recruited via emails distributed to 2 organizations (state fire and rescue service and a freight transport organization)60Between 26 and 6592% men27% rural, 23% suburban, and 50% urbanRelevance of considering language use and preferred features and balancing preferences with the need for evidence-based interventionsMen preferred unstigmatized language use, a simple mood management app, and guidance involvement~
Schneider et al [39], United KingdomViews and acceptance of 2 self-help applications for depression: MoodGYM (cCBTr) and informational websites applied at work; 5 weeksMixed methods; recruited from 3 organizations: 2 private enterprises (telecommunications and transport) and 1 health organization637Mean 42 (SD 9.6)50.2% menEvidence-based computerized approaches supported acceptability, which could be increased by taking care of barriers and users’ expectationsDropout: 63%; positive rating: 24%; various intrinsic and extrinsic barriers that lead to a high unacceptance; acceptance increases with interactive supportIntrinsic: intrapersonal problems; extrinsic: technical problems; generic: perception of cCBT− −
Wang and Ho [46], CanadaExplorative study on barriers and preferences for specific features among male workers in a mental health toolCross-sectional study; recruited by random digit-calling method to households collecting data from 511 men with risk of depression841Mean 44.3 (SD 13.7)100% menOverall positive results, but men’s preferences and perceived barriers should be taken into account to increase acceptabilityAcceptance in men was good, but apps should be mobile and tailored to preferences, including various topics and designsHaving high risk of depression at baseline increased the chance to see the utility of the intervention compared with low-risk individuals (83.4% vs 75%)+
Williams et al [35], United StatesFeasibility of a web-enhanced behavioral self-management program, Stress Gym, in a military setting built on the model of cognitive appraisal by Lazarus and FolkmanCross-sectional study; recruited and invited all active-duty members at Naval Medical Center, Portsmouth, Virginia142Mean 41.1 (SD 9.2)55% women24% officers and 76% enlisted sailorsSupported the feasibility of Stress Gym as being a web-based CBT-based self-help intervention accepted by the users and demonstrated reduction in stressStressGym was rated as very useful and informative+ +
Wilson et al [47], United StatesSoldiers’ attitude toward technology-based approaches to mental health careCross-sectional study; recruited from pre- and postdeployment clinic (in the waiting room for screening visits)352Mean 25.9 (SD 5.8)92% menFeasibility of technology-based approaches was supportedWillingness to use: 84% were willing to use one of the 11 interventions; comfort: 75% felt neutral/very comfortable using a computer/program+ +

aSorted from indirect to more direct measures.

bCBT: cognitive behavioral therapy.

cRCT: randomized controlled trial.

dIOC: information-only control.

eData missing or not relevant.

fMixed results.

gLow.

hModerate.

iEAP: employee assistance program.

jVery high.

kHigh.

lNRW: North Rhine-Westphalia.

mHR: human resources.

neMH: e-mental health.

oCBT-I: cognitive behavioral therapy for insomnia.

pPTSD: posttraumatic stress disorder.

qWTU: Warrior Transition Unit.

rcCBT: computerized cognitive behavioral therapy.


Review Process

The characteristics of the studies are outlined in Table 1 as well as in Multimedia Appendix 2 including more details. Within the review process, 1303 papers were identified, of which 363 (28%) were duplicates, not published in English, or published before 2005. Of the 1303 papers, titles and abstracts were then scanned, a process that identified 940 (72%) and 379 (29%) papers, respectively. Papers were excluded if the interventions were independent of work environments, did not include most employees, and did not focus on mental health issues. Most studies were excluded owing to the involvement of face-to-face or telephone guidance by a coach or therapist. Ultimately, 28 studies were identified for further analysis, which either reported indirect measures of the acceptability of web-based interventions (n=17, 61%) or provided qualitative data on acceptability (n=11, 39%; Figure 1).

Study Characteristics

The 28 included studies had an overall sample of 9739 participants, with sample sizes ranging from 8 to 1140. The mean age of the participants was 40.7 years, and most participants were White and employed full-time.

The studies had various differing methodological designs and study characteristics, which are outlined in Table 1 as well as in Multimedia Appendix 2 including more details on direct measures. Interventions were heterogeneous in type, application outcome focus, length, setting (within a specific organization or random employees), and characteristics of the participants (type of profession and demographics). In addition, they differed in the level of support potentially provided in web-based format (email or message) versus unguided. Most interventions (18/28, 64%) were focused on reducing stress [21-35] and depressive symptoms [23,31,36-39] in employees. Other interventions focused on insomnia (2/28, 7%) [29,40], anxiety (2/28, 7%) [23,31], panic (2/28, 7%) [23,27], psychological detachment from work (1/28, 4%) [40], resilience (burnout; 1/28, 4%) [41], mood (1/28, 4%) [22], tinnitus distress (1/28, 4%) [42], chronic pain (1/28, 4%) [43], substance misuse (2/28, 7%) [23,25], and well-being or happiness (3/28, 11%) [31,38,44]. Studies and interventions often included more than one focus of mental illness, used various treatment techniques, and assessed the acceptability of general web-based mental health interventions [34,45,46]. Cognitive behavioral therapy (CBT) was the most used form of intervention (9/28, 32%). Other interventions applied mindfulness (3/28, 11%), psychological education (3/28, 11%), cognitive appraisal or restructuring (2/28, 7%), emotional regulation (1/28, 4%), acceptance and commitment therapy (1/28, 4%), problem solving (2/28, 7%), exercise (1/28, 4%), or communicational strategies (1/28, 4%). Approximately 7% (2/28) of the studies used tracking or assessment tools for burnout and mood (eg, depression, stress, or well-being). The studies were predominantly RCTs (13/28, 46%). CBT was mostly used in interventions that tackled depression or stress at work. For example, the CBT interventions focused on depression were HelpID, Mood Hacker, Colour Your Life, and MoodGym. The interventions that used CBT and aimed to reduce stress and mental strain were Psyfit, Strong at work, GetON Recovery, and Work Guru.

Of the 28 studies, 13 (46%) were RCTs, 4 (14%) were cross-sectional studies, 3 (11%) were qualitative studies, 3 (11%) were pilot studies, 2 (7%) were longitudinal studies, 2 (7%) were mixed methods studies, and 1 (4%) was a case report. The studies mostly used waitlist control groups, internet-based information website groups, or variations of intervention type groups as comparators. The studies originated in the United States (8/28, 29%), Germany (7/28, 25%), Australia (4/28, 14%), the Netherlands (2/28, 7%), Japan (2/28, 7%), Sweden (2/28, 7%), the United Kingdom (2/28, 7%), and Canada (1/28, 4%).

Methodological Quality

The methodological quality of the studies is summarized in Multimedia Appendix 1. The studies were assessed for quality using the CASP [20] qualitative and quantitative templates and reported in the form of a traffic light schema. Various quality flaws were outlined in the studies, and no study met all 10 criteria marked by the CASP [20]. Independent of quality, all studies (28/28, 100%) were included in the final synthesis. Allocation bias appeared to be low in the quantitative studies as participants were mostly randomly distributed to their condition (15/17, 88%). However, qualitative studies often indicated performance and detection bias as studies often missed reporting on blinding status or researchers’ awareness of the participants’ condition. Selection bias was generally high as participants repeatedly originated from specific population samples (eg, male-dominated industry workers or female educational staff). Attrition bias was predominantly high as various studies reported a high dropout rate, which weakened their generalizability. Several studies missed reporting on the specific demographics of their samples and, thus, might risk the presence of confounders, whereas other studies (4/28, 14%) clearly outlined their risk of confounding [25,26,30,42]. The analysis of quantitative studies was generally good as all studies used data from all participant groups in their final analysis. Qualitative studies showed generally good quality in the guidance of clear questions, taking care of ethical considerations, and the provision of clear information on methodology. However, various studies missed accounting for the potential bias caused by the relationship between the researchers and the participants.

Setting and Types of Employees

The recruitment setting and included characteristics of the employees were very diverse (Table 1). Some studies (10/28, 36%) used the whole working population, recruiting samples via local insurance companies, occupational programs or employee assistance programs, random digit calling, or advertisements [24,26,28,34,36-38,43,44,46]. Alternatively, some studies used specific population samples originating from 1 type of profession. In particular, samples were recruited from the military (2/28, 7%) [35,47], telecommunications (2/28, 7%) [21,39], transport (2/28, 7%) [39,45], the public sector (1/28, 4%) [39], state fire and rescue services (2/28, 7%) [33,45], office and client employees (1/28, 4%) [32], university staff and teachers (3/28, 11%) [29,33,40], clinical staff and health professionals (4/28, 14%) [23,27,31,41], manufacturing and industrial workers (2/28, 7%) [30,42], marketing (1/28, 4%) [25], and technology (1/28, 4%) [22]. Most studies were conducted in the United States (8/28, 29%) and Germany (7/28, 25%). Germany primarily recruited from the general working population [26,34,36,44], whereas the United States mainly recruited from multiple specific locations, including larger organizations (eg, corporate call centers and technology companies), health centers, and military-related workplaces. Their acceptability results were mixed in outcome, ranging from very high in the study by Beiwinkel et al [36] to very low in the study by Hennemann et al [34]. The synthesis did not outline any pattern of setting and participant characteristics that was associated with the acceptability level of web-based interventions.

Intervention Characteristics and Country of Conduct

As outlined in the Study Characteristics section, most studies used CBT in their administered interventions (9/28, 32%). CBT was relatively equally distributed across Western countries, including the United States (2/9, 22%), Germany (2/9, 22%), the Netherlands (2/9, 22%), the United Kingdom (2/9, 22%), and Australia (1/9, 11%). Summarizing the CBT studies, the acceptability level indicated that 44% (4/9) of the studies had a low to moderate level of acceptability, whereas 33% (3/9) of the studies showed a high to very high acceptability level. Approximately 11% (1/9) of the studies had a mix of moderate and high acceptability levels. Other analyzed intervention types (mindfulness, psychological education, cognitive appraisal, emotional regulation, acceptance and commitment therapy, problem solving, cognitive strategies, exercise, and tracking tools) did not indicate any pattern of acceptability level. Broadly speaking, the intervention type, country of conduct, and outcome of the study did not indicate any notable patterns. However, most of the studies (26/28, 93%) were conducted in Western countries.

Measure of Acceptability

Relevant studies measured acceptability in different ways. They used direct measures of acceptability, which included qualitative data through questionnaires and interviews, or indirect quantitative measures by means of take-up, dropout, compliance, adherence, attrition, or completion rate. Some studies used both direct and indirect measures. All measures of acceptability are outlined in either Table 1 or Multimedia Appendix 2 (qualitative synthesized data) in the context of the reference, intervention, sample, study design, recruitment, outcome, indirect and direct acceptability measures, available reasons for dropout, example quotations from interviews, and an individually rated acceptability level.

Direct Measure of Acceptability

Table 1 and Multimedia Appendix 2 present the direct outcome of employees’ acceptability of web-based therapy in the workplace. When categorizing the qualitative outcome into key themes, the following topics commonly emerged: (1) general interest in or willingness to use web-based interventions, (2) employees’ satisfaction rating of the utility of the interventions, and (3) preferred features of the design and application style of the interventions. Most participants reported a generally positive interest in and acceptability of web-based interventions [31,33,35,38,46]. However, there were mixed results and negative opinions in other studies [32,33,39,48]. Common preferred features of web-based mental health interventions were the use of nonstigmatized language [45,48], the preference for interventions with interactive support [39,45], and broad application spectrum as well as short mobile and interactive multimedia interventions [31,35,38,48]. The synthesized outputs of the studies were written in descriptions of each theme as well as provided within the context of the setting and intervention type. To deliver a deeper insight into common themes, Multimedia Appendix 2 provides quotations of interviewees in primary studies. As this systematic review synthesized key themes in an integrative, meta-aggregative way, quotations aid in the understanding of the summarized key themes. Most of the studies reported details on the satisfaction rating on a scale associated with web-based interventions. Employees were mostly satisfied with the interventions and rated their utility positively.

In addition, multiple studies assessed acceptability using satisfaction, usability, or interest ratings of the intervention (Table 1). Satisfaction ratings were frequently used in the studies [22,24,26,29,31,36-38,41,43]. The average satisfaction score was 82.6%, which is similar to a very high individual-defined acceptability level (++) of web-based interventions. Moreover, 14% (4/28) of the studies reported a score of 0.85% for practical use [22,31,38,41], equivalent to a high (+) acceptability. In particular, Wilson et al [47] reported a rate of 75% in “comfortability” of using a mental health program on the computer and an 84% rate in “willingness to use.” In contrast, Hennemann et al [34] reported that 89.1% of participants rated low on the “acceptability” of general occupational web-based mental health interventions. Both studies were very heterogeneous in intervention specificity and sample population. Hennemann et al [34] explained the negative outcome by the direct predictor variables of acceptability with “social influence,” “effort” and “performance expectancy,” and “time spent in the web” as well as the “frequency of searching online for health information.”

Indirect Measure of Acceptability

This systematic review included hypothetical measures of acceptability characterized by dropout, attrition, compliance, adherence, uptake, and completion rate. The indirect or hypothetical measures of the acceptability of web-based interventions in the workplace are summarized in Table 1. The mean percentage of dropout rates from the included studies was 50.9% with a range of 15.3% [25] to 67.7% [36], which is equivalent to a moderate individual-defined level of acceptability [23-25,27,36,39,44]. A few studies reported the reasons for dropout or termination of the interventions. Repeated reasons were lack of time [21,26,40], technical difficulties [26,27,39,40], younger age [23,27,36], lower education [36], lack of motivation [26,40], no need for help [40], ability to manage stress personally [22], dissatisfaction with the intervention [26,39], higher initial level of psychological distress [30], and privacy concerns [31]. Other measures of acceptability included an average attrition rate of 32% [24,37,42], an average adherence rate of 54% [25,28], an uptake and intervention start rate of 11% [23,27], and a completion rate of 68% [30,40]. As visible in the outcome, there was no clear consensus in acceptability level, and the comparison of studies was difficult as they were heterogeneous in study design, sample, and methodology. However, the most frequently reported indirect measure of acceptability was the dropout rate, supported by a moderate (−) level of acceptability.


Principal Findings

This systematic review assessed the levels of employees’ acceptability of web-based interventions aimed at improving mental health. The findings showed a generally positive level of acceptability and highlighted various factors to be considered in making interventions acceptable, engaging, and useful for employees. Themes to be addressed with caution when introducing interventions are the use of stigmatized terminology, including words of ill health and mental illness. In terms of implementation, applications are recommended to be short and use interactive multimedia tools.

Results were obtained from 28 separate studies. Satisfaction ratings and feedback appeared positive, particularly when the interventions included multimedia and nonstigmatizing language. In particular, 79% (22/28) of the studies showed acceptability measures from high to very high, and 54% (15/28) of the studies reported acceptability levels from low to moderate (overlap when studies reported both quantitative and qualitative results). The average satisfaction rating was >80%, and the employees rated the interventions’ utility as good overall. However, quantitative measures contradicted the universal positive perspective of web-based interventions by means of the common measured dropout rate of approximately 50%. Hence, the attrition rate was very high in multiple studies, which questions the efficacy of unguided self-applied interventions.

Collectively, these results are in line with other acceptability studies that supported the general acceptability of web-based interventions in clinical settings [51]. Various studies have outlined barriers to assessing acceptability; for example, negative results from indirect measures. In addition, complications in synthesis owing to the heterogeneity of the interventions have been repeatedly reported.

Stigma and attitudes toward mental health at work were an emerging theme. Acceptability levels may relate to the web-based interventions themselves or to the fact that the intervention relates to mental health. This is supported by other studies showing that there is fear of stigmatization when seeking support [52]. It may also be difficult to successfully implement web-based interventions within an organization as employees prefer to separate health matters and their workplace [32]. Hence, the issue around mental health and stigma, especially at work, may be strongly influenced by the organizational culture that influences the use of mental health interventions [45].

The relationship between dropout and acceptability requires further assessment to interpret the current evidence. Although dropout for web-based workplace interventions was high (the mean percentage score of the included studies was 50.9%), explorations of the reasons for this were limited. Indeed, studies have outlined that dropout rates might not be the result of disinterest in occupational web-based interventions for mental health issues but appear to be generally high in computerized interventions [53], suggesting that these interventions are not as engaging as guided or face-to-face sessions and people might not feel committed enough to complete the treatment or program. Consequently, web-based interventions should be tailored and made as interactive and attractive as possible by using animation tools, pictures, and videos, as well as made as short and simple as possible to increase engagement and decrease the likelihood of technical issues [12]. Furthermore, the findings of this study suggest that, before applying interventions in organizations, people’s needs, the environment, and the culture should be assessed; the interventions should be tailored accordingly; and awareness of the benefits and understanding of the use should be addressed.

Strengths and Limitations

The generalizability of the findings across workplaces may be limited because of the diversity of individual workplaces; for example, their organizational culture and stigma or attitudes toward mental health. In addition, assessing for confounding variables, including recruitment, setting, intervention characteristics, and country of conduct, did not reveal significant information. However, most of the included studies were conducted in Western countries and used CBT-based interventions, which may further limit generalizability.

Assessing acceptability using indirect measures may be flawed as there could be multiple reasons for employees to stop the intervention. Specifically, dropping out of interventions could be the result of feeling rehabilitated and seeing no further benefit of using the intervention. Nevertheless, dropouts provide great insight into the acceptability of interventions, but more in-depth analyses of the reasons for dropping out should be conducted.

Analysis of the specific assessment of acceptability of occupational web-based interventions was limited because of the heterogeneity of the study designs, intervention types, sample characteristics, and conditions under which the interventions were provided to employees. The studies used data assessment techniques, including cross-sectional self-report methods, whereas the qualitative studies used small samples. Data collection and analysis biases may be observed based on the role of the researchers [54]. As qualitative acceptability results were generally higher compared with indirect measures, this further raises the question of the role of researcher bias. In addition, limitations regarding the consistent and objective measurement of acceptability in the wider literature prevent robust conclusions from being drawn. However, the inclusion and critical appraisal of qualitative studies may have added depth to the factors within the acceptability capture in this study [55].

Despite these limitations, this study offers a comprehensive insight into multiple forms of acceptability measures [56]. Using both qualitative and quantitative as well as direct and indirect measures of acceptability provided a deeper insight into the options for assessing the acceptability of interventions in general. Although this study focused on the workplace, it examined the acceptability of web-based interventions that could be applied more generally to support people’s mental health. For example, the findings could support the implementation of interventions outside of the workplace (eg, as part of clinical mental health treatments). These results might help clinicians, developers, researchers, and the health technology industry create effective and engaging tools in the future.

Implications

In relation to workplace practice, before applying interventions, it would be beneficial to increase people’s knowledge of web-based interventions as well as assess their needs in general to improve their attitude toward interventions [13,34]. This is supported by Murray et al [57], whose study found that participants who rejected computerized treatments had significantly lower expectations of the usefulness of self-help and had general concerns, anxiety, and misunderstandings about computerized treatments. Hence, acceptability may be increased by identifying and correcting misperceptions before participation. Similarly, tailoring interventions to the environment and employees’ needs could increase their general interest and willingness to use them [13]. In other words, web-based interventions for employees should be adapted to the specific environment applied as well as to the users’ needs to increase engagement and acceptability levels. Generally, the acceptability of interventions might increase if employees and organizations are made aware of evidence-based web-based interventions that have multiple practical benefits and the potential to increase individuals’ mental health and well-being in the long run. Finally, the ability of web-based interventions to engage and retain users is critical for ensuring reduced dropout and increased acceptability.

Regarding future research, the results of acceptability studies could be influenced by the general stigma on mental health topics and interventions. Therefore, future research should incorporate acceptability measures of mental health issues into their analysis to assess for confounding variables. Second, regarding quantitative data on acceptability, it would be beneficial if future research included a more in-depth analysis of the reasons for dropout or attrition rates. Third, future research should also address the conceptual and methodological limitations of the research in the field. If there were more organizations using mental health interventions from various settings, the research analysis could be more homogeneous. Organizations might lack the knowledge on how to apply personal health support but could provide their employees with interventions that range from broader aspects of stress management to specified apps that tackle specific mental health issues (eg, depression). Finally, this research was conducted before the COVID-19 pandemic, which changed work styles and environments and affected how people sought and received mental health support. Further research should analyze changes in acceptability as a result of the pandemic to examine shifts in use and acceptability of mental health interventions both within and outside of the workplace.

Conclusions

This study assessed the area of acceptability of web-based workplace interventions for mental health. In general, workers are open to web-based mental health interventions. However, qualitative and quantitative studies suggested varying levels of acceptability, raising the possibility of bias. The importance of stigma, organizational culture, and the implementation of the intervention were highlighted, the latter relating to the engaging design and quality of the intervention as well as the approach to delivery in the workplace itself. Several factors were identified that need to be considered to ensure the effective implementation of web-based interventions in the workplace, some aspects of which may also apply to the general use in supporting people’s mental health. Interventions should be tailored to the respective individual needs and cultural context, use nonstigmatized language, and be made interactive and easy to use. It is also recommended to foster an understanding of the potential value of an intervention to increase its acceptability. Methodological limitations were highlighted to guide the cautious interpretation and generalization of early evidence in this area along with the need to improve methodological rigor in emerging research.

Acknowledgments

The author would like to thank their supervisor, Chris Attoe, for his inspiring feedback and guidance. The author would also like to express gratitude to Dr Joshua Harwood for his invaluable advice.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Quality rating.

PNG File , 245 KB

Multimedia Appendix 2

Direct measures.

DOCX File , 36 KB

  1. Health and Well-being at Work. Chartered Institute of Personnel and Development. 2018 May.   URL: https://www.cipd.co.uk/Images/health-and-well-being-at-work_tcm18-40863.pdf [accessed 2022-04-28]
  2. Work-related stress, anxiety or depression statistics in Great Britain, 2021. Health and Safety Executive. 2021 Mar.   URL: http://www.hse.gov.uk/statistics/causdis/stress.pdf [accessed 2022-04-28]
  3. Employing disabled people and people with health conditions. United Kingdom Government. 2020 Mar 25.   URL: https://tinyurl.com/veveznmz [accessed 2022-04-28]
  4. Mental health conditions, work and the workplace. Health and Safety Executive.   URL: https://www.hse.gov.uk/stress/mental-health.htm [accessed 2022-04-28]
  5. Aardoom JJ, Dingemans AE, Spinhoven P, Van Furth EF. Treating eating disorders over the internet: a systematic review and future research directions. Int J Eat Disord 2013 Sep;46(6):539-552. [CrossRef] [Medline]
  6. Abbott JM, Klein B, Ciechomski L. Best practices in online therapy. J Technol Hum Serv 2008 Jul 03;26(2-4):360-375. [CrossRef]
  7. Donker T, Blankers M, Hedman E, Ljótsson B, Petrie K, Christensen H. Economic evaluations of Internet interventions for mental health: a systematic review. Psychol Med 2015 Dec;45(16):3357-3376. [CrossRef] [Medline]
  8. Geraedts AS, Kleiboer AM, Wiezer NM, Cuijpers P, van Mechelen W, Anema JR. Feasibility of a worker-directed web-based intervention for employees with depressive symptoms. Internet Interv 2014 Jul;1(3):132-140. [CrossRef]
  9. Kuster AT, Dalsbø TK, Luong Thanh BY, Agarwal A, Durand-Moreau QV, Kirkehei I. Computer-based versus in-person interventions for preventing and reducing stress in workers. Cochrane Database Syst Rev 2017 Aug 30;8(8):CD011899 [FREE Full text] [CrossRef] [Medline]
  10. Rost T, Stein J, Löbner M, Kersting A, Luck-Sikorski C, Riedel-Heller SG. User acceptance of computerized cognitive behavioral therapy for depression: systematic review. J Med Internet Res 2017 Sep 13;19(9):e309 [FREE Full text] [CrossRef] [Medline]
  11. Lal S, Adair CE. E-mental health: a rapid review of the literature. Psychiatr Serv 2014 Jan 01;65(1):24-32. [CrossRef] [Medline]
  12. Carolan S, Harris PR, Cavanagh K. Improving employee well-being and effectiveness: systematic review and meta-analysis of Web-based psychological interventions delivered in the workplace. J Med Internet Res 2017 Jul 26;19(7):e271 [FREE Full text] [CrossRef] [Medline]
  13. Stratton E, Lampit A, Choi I, Calvo RA, Harvey SB, Glozier N. Effectiveness of eHealth interventions for reducing mental health conditions in employees: a systematic review and meta-analysis. PLoS One 2017 Dec 21;12(12):e0189904 [FREE Full text] [CrossRef] [Medline]
  14. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017 Jan 26;17(1):88 [FREE Full text] [CrossRef] [Medline]
  15. Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med 2008 Nov;38(11):1521-1530. [CrossRef] [Medline]
  16. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
  17. Boland A, Cherry G, Dickson R. Doing a Systematic Review: A Student's Guide. 2nd edition. Thousand Oaks, CA, USA: Sage Publications; 2017.
  18. Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol 2012 Nov 27;12(1):181 [FREE Full text] [CrossRef] [Medline]
  19. Joyce S, Modini M, Christensen H, Mykletun A, Bryant R, Mitchell PB, et al. Workplace interventions for common mental disorders: a systematic meta-review. Psychol Med 2016 Mar;46(4):683-697. [CrossRef] [Medline]
  20. Critical appraisal skills program: making sense of evidence. Critical Appraisal Skills Programme. 2002.   URL: https://casp-uk.net/casp-tools-checklists/ [accessed 2022-04-28]
  21. Allexandre D, Bernstein AM, Walker E, Hunter J, Roizen MF, Morledge TJ. A Web-based mindfulness stress management program in a corporate call center: a randomized clinical trial to evaluate the added benefit of onsite group support. J Occup Environ Med 2016 Mar;58(3):254-264 [FREE Full text] [CrossRef] [Medline]
  22. Billings DW, Cook RF, Hendrickson A, Dove DC. A Web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med 2008 Aug;50(8):960-968 [FREE Full text] [CrossRef] [Medline]
  23. Bolier L, Ketelaar SM, Nieuwenhuijsen K, Smeets O, Gärtner FR, Sluiter JK. Workplace mental health promotion online to enhance well-being of nurses and allied health professionals: a cluster-randomized controlled trial. Internet Interv 2014 Oct;1(4):196-204. [CrossRef]
  24. Ebert DD, Heber E, Berking M, Riper H, Cuijpers P, Funk B, et al. Self-guided internet-based and mobile-based stress management for employees: results of a randomised controlled trial. Occup Environ Med 2016 May;73(5):315-323. [CrossRef] [Medline]
  25. Hamamura T, Suganuma S, Ueda M, Mearns J, Shimoyama H. Standalone effects of a cognitive behavioral intervention using a mobile phone app on psychological distress and alcohol consumption among Japanese workers: pilot nonrandomized controlled trial. JMIR Ment Health 2018 Mar 22;5(1):e24 [FREE Full text] [CrossRef] [Medline]
  26. Heber E, Lehr D, Ebert DD, Berking M, Riper H. Web-based and mobile stress management intervention for employees: a randomized controlled trial. J Med Internet Res 2016 Jan 27;18(1):e21 [FREE Full text] [CrossRef] [Medline]
  27. Ketelaar SM, Nieuwenhuijsen K, Gärtner FR, Bolier L, Smeets O, Sluiter JK. Effect of an E-mental health approach to workers' health surveillance versus control group on work functioning of hospital employees: a cluster-RCT. PLoS One 2013 Sep 12;8(9):e72546 [FREE Full text] [CrossRef] [Medline]
  28. Ly KH, Asplund K, Andersson G. Stress management for middle managers via an acceptance and commitment-based smartphone application: a randomized controlled trial. Internet Interv 2014 Jul;1(3):95-101. [CrossRef]
  29. Thiart H, Lehr D, Ebert DD, Berking M, Riper H. Log in and breathe out: Internet-based recovery training for sleepless employees with work-related strain - results of a randomized controlled trial. Scand J Work Environ Health 2015 Mar;41(2):164-174 [FREE Full text] [CrossRef] [Medline]
  30. Umanodan R, Shimazu A, Minami M, Kawakami N. Effects of computer-based stress management training on psychological well-being and work performance in Japanese employees: a cluster randomized controlled trial. Ind Health 2014;52(6):480-491 [FREE Full text] [CrossRef] [Medline]
  31. Bush NE, Ouellette G, Kinn J. Utility of the T2 Mood Tracker mobile application among army warrior transition unit service members. Mil Med 2014 Dec;179(12):1453-1457. [CrossRef] [Medline]
  32. Carolan S, de Visser RO. Employees' perspectives on the facilitators and barriers to engaging with digital mental health interventions in the workplace: qualitative study. JMIR Ment Health 2018 Jan 19;5(1):e8 [FREE Full text] [CrossRef] [Medline]
  33. Eklund C, Elfström ML, Eriksson Y, Söderlund A. User experiences from a Web-based, self-management programme: struggling with what I need when stress management is about me. Eur J Physiother 2018 May 03;21(1):39-48. [CrossRef]
  34. Hennemann S, Witthöft M, Bethge M, Spanier K, Beutel ME, Zwerenz R. Acceptance and barriers to access of occupational e-mental health: cross-sectional findings from a health-risk population of employees. Int Arch Occup Environ Health 2018 Apr;91(3):305-316. [CrossRef] [Medline]
  35. Williams A, Hagerty BM, Brasington SJ, Clem JB, Williams DA. Stress Gym: feasibility of deploying a web-enhanced behavioral self-management program for stress in a military setting. Mil Med 2010 Jul;175(7):487-493. [CrossRef] [Medline]
  36. Beiwinkel T, Eißing T, Telle NT, Siegmund-Schultze E, Rössler W. Effectiveness of a Web-based intervention in reducing depression and sickness absence: randomized controlled trial. J Med Internet Res 2017 Jun 15;19(6):e213 [FREE Full text] [CrossRef] [Medline]
  37. Birney AJ, Gunn R, Russell JK, Ary DV. MoodHacker mobile Web app with email for adults to self-manage mild-to-moderate depression: randomized controlled trial. JMIR Mhealth Uhealth 2016 Jan 26;4(1):e8 [FREE Full text] [CrossRef] [Medline]
  38. Deady M, Johnston D, Milne D, Glozier N, Peters D, Calvo R, et al. Preliminary effectiveness of a smartphone app to reduce depressive symptoms in the workplace: feasibility and acceptability study. JMIR Mhealth Uhealth 2018 Dec 04;6(12):e11661 [FREE Full text] [CrossRef] [Medline]
  39. Schneider J, Sarrami Foroushani P, Grime P, Thornicroft G. Acceptability of online self-help to people with depression: users' views of MoodGYM versus informational websites. J Med Internet Res 2014 Mar 28;16(3):e90 [FREE Full text] [CrossRef] [Medline]
  40. Ebert DD, Berking M, Thiart H, Riper H, Laferton JA, Cuijpers P, et al. Restoring depleted resources: efficacy and mechanisms of change of an Internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychol 2015 Dec;34S:1240-1251. [CrossRef] [Medline]
  41. Wood AE, Prins A, Bush NE, Hsia JF, Bourn LE, Earley MD, et al. Reduction of burnout in mental health care providers using the provider resilience mobile application. Community Ment Health J 2017 May;53(4):452-459. [CrossRef] [Medline]
  42. Abbott JA, Kaldo V, Klein B, Austin D, Hamilton C, Piterman L, et al. A cluster randomised trial of an Internet-based intervention program for tinnitus distress in an industrial setting. Cogn Behav Ther 2009;38(3):162-173. [CrossRef] [Medline]
  43. Nevedal DC, Wang C, Oberleitner L, Schwartz S, Williams AM. Effects of an individually tailored Web-based chronic pain management program on pain severity, psychological health, and functioning. J Med Internet Res 2013 Sep 25;15(9):e201 [FREE Full text] [CrossRef] [Medline]
  44. Feicht T, Wittmann M, Jose G, Mock A, von Hirschhausen E, Esch T. Evaluation of a seven-week Web-based happiness training to improve psychological well-being, reduce stress, and enhance mindfulness and flourishing: a randomized controlled occupational health study. Evid Based Complement Alternat Med 2013;2013:676953 [FREE Full text] [CrossRef] [Medline]
  45. Peters D, Deady M, Glozier N, Harvey S, Calvo RA. Worker preferences for a mental health app within male-dominated industries: participatory study. JMIR Ment Health 2018 Apr 25;5(2):e30 [FREE Full text] [CrossRef] [Medline]
  46. Wang JL, Ho K. Preferred features of e-mental health programs for prevention of major depression in male workers: results from a Canadian National Survey. Stud Health Technol Inform 2017;245:249-252. [CrossRef] [Medline]
  47. Wilson JA, Onorati K, Mishkind M, Reger MA, Gahm GA. Soldier attitudes about technology-based approaches to mental health care. Cyberpsychol Behav 2008 Dec;11(6):767-769. [CrossRef] [Medline]
  48. Deady M, Peters D, Lang H, Calvo R, Glozier N, Christensen H, et al. Designing smartphone mental health applications for emergency service workers. Occup Med (Lond) 2017 Aug 01;67(6):425-428. [CrossRef] [Medline]
  49. Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc 2015 Sep;13(3):179-187. [CrossRef] [Medline]
  50. Berry N, Lobban F, Emsley R, Bucci S. Acceptability of interventions delivered online and through mobile phones for people who experience severe mental health problems: a systematic review. J Med Internet Res 2016 May 31;18(5):e121 [FREE Full text] [CrossRef] [Medline]
  51. Álvarez-Jiménez M, Gleeson JF, Bendall S, Lederman R, Wadley G, Killackey E, et al. Internet-based interventions for psychosis: a sneak-peek into the future. Psychiatr Clin North Am 2012 Sep;35(3):735-747. [CrossRef] [Medline]
  52. Shocking new poll reveals mental health stigma in the workplace. Mental Health Foundation. 2017.   URL: https://www.mentalhealth.org.uk/news/shocking-new-poll-reveals-mental-health-stigma-workplace [accessed 2022-04-28]
  53. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to Web-based interventions. J Med Internet Res 2012 Nov 14;14(6):e152 [FREE Full text] [CrossRef] [Medline]
  54. Anderson C. Presenting and evaluating qualitative research. Am J Pharm Educ 2010 Oct 11;74(8):141 [FREE Full text] [CrossRef] [Medline]
  55. Bearman M, Dawson P. Qualitative synthesis and systematic review in health professions education. Med Educ 2013 Mar;47(3):252-260. [CrossRef] [Medline]
  56. Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health 2014;35:29-45. [CrossRef] [Medline]
  57. Murray K, Pombo-Carril MG, Bara-Carril N, Grover M, Reid Y, Langham C, et al. Factors determining uptake of a CD-ROM-based CBT self-help treatment for bulimia: patient characteristics and subjective appraisals of self-help treatment. Eur Eat Disord Rev 2003;11(3):243-260. [CrossRef]


CASP: Critical Appraisal Skills Programme
CBT: cognitive behavioral therapy
ENTREQ: Enhancing the Transparency in Reporting the Synthesis of Qualitative Research
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
RCT: randomized controlled trial


Edited by J Torous; submitted 04.11.21; peer-reviewed by A Richert-Kaźmierska, O Chelidoni; comments to author 08.12.21; revised version received 31.01.22; accepted 08.02.22; published 11.05.22

Copyright

©Johanna Scheutzow, Chris Attoe, Joshua Harwood. Originally published in JMIR Mental Health (https://mental.jmir.org), 11.05.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.