There is an increasing level of awareness regarding the importance of health and well-being in the workplace [1]. Anxiety, stress, and depression are the dominant mental health issues for workers in the United Kingdom, with a prevalence of 1320 cases per 100,000 workers, causing close to 18 million lost working days per year [2]. Employers have a responsibility to take care of their employees and provide support for both their physical and mental health [3,4].

Web-based mental health interventions are increasingly being used in the work environment as they have the advantage of being cost-effective, efficient, anonymous, location-independent, flexible, and empowering. They are regularly used for both prevention and intervention [5-10].

Web-based interventions also have multiple flaws, including technical difficulties, ethical concerns, increased attrition rates, and low engagement in the absence of guidance by professional support [6,11]. Therefore, it is important to understand the barriers that reduce engagement and acceptability of web-based interventions. Multiple systematic studies provide evidence of the effectiveness of web-based mental health interventions at work [12,13]. Importantly, they outline the need to tailor interventions to populations’ needs, which requires greater insight into barriers and the acceptability of web-based mental health interventions in the workplace.

The acceptability of an intervention includes users’ emotional and cognitive responses to the intervention [14], including affective perceptions, burden and barriers, perceived benefits, understanding of the intervention, opportunity costs, and usability. In practice, this takes into account the individuals’ preferences for features and tools, their willingness to use web-based interventions, their engagement (eg, dropout and attrition rate), and users’ perceived utility or satisfaction with the intervention.

Studying users’ acceptability of new treatments has ethical, methodological (validity), and practical applications [15]. Specifically, ethical obligations include the exploration of reasons for acceptable or unacceptable treatments as perceived by the users. It is important to understand potential barriers to intervention engagement before introducing the intervention to employees. Awareness of intervention efficacy alone does not mean that employees accept web-based interventions as a useful tool for self-help.

Sekhon et al [14] outlined studies assessing interventions’ acceptability by using operational definitions in line with measurable acceptability data (dropout rate and satisfaction rating) and qualitative studies focusing on in-depth user experiences. Current research has been limited to studies on clinical populations. Clinical populations differ significantly in symptom severity, level of risks, functionality, and response to treatment; thus, the results might not be generalizable to occupational populations [13]. Therefore, it is relevant to explicitly assess employees’ acceptability of web-based interventions.

This systematic review aimed to assess employees’ acceptability of web-based interventions to improve their mental health. The study aimed to inform intervention design and utility by evaluating user experience and barriers and facilitators to using web-based mental health interventions in the workplace.

Eligible studies met the Population, Intervention, Comparison, Outcome criteria and included qualitative interviews, quantitative studies including scale measures of satisfaction and forms of attrition rates, and mixed methods studies. Acceptability was assessed by means of both direct (acceptability, satisfaction, and experience) and indirect (compliance, completion, adherence, attrition, and dropout rate) measures. Studies were included if they were available in English and published after 2005.

Studies were excluded if they did not meet the Population, Intervention, Comparison, Outcome criteria; that is, if they included guidance through coaches, therapists, or face-to-face interactions and were applied to participants who were retired or unemployed (>40%). In addition, studies were excluded if they did not measure acceptability or willingness of use as an outcome variable or used interventions that were not focused on the users’ mental health.

The search was conducted in July 2019 and included the following electronic databases: PsycINFO (Ovid), Embase (Ovid), MEDLINE (Ovid), Global Health (Ovid), and the Cochrane Library Trials (CENTRAL). Backward searching was used to ensure that no key papers were missed.

Databases were searched for studies published between 2005 and 2019. Duplicates were removed (Ovid search option). The Boolean system was applied using AND and OR (Textbox 1) to combine different terminologies of 4 key concepts included in a free-text and keyword search. Specific occupations were added to the general search of employees to increase the likelihood of finding studies on high stress–exposed work settings; for example, in the military or firefighter professions. The search terms used were categorized as occupational settings (employee), web-based interventions, mental ill health, and acceptability of interventions. All the key terms considered American and British spelling.

Duplicates were removed, and titles, abstracts, and full texts were scanned for the inclusion criteria. After the assessment of the full texts’ eligibility by the first author (JS), all the included studies were summarized and synthesized. The study selection process is outlined in the PRISMA flowchart (Figure 1).

Data were collected according to the following criteria: reference, characteristics of the intervention, its aims and objectives, study design, population, setting and recruitment, results, acceptability, and—if available—reasons for dropout, as well as qualitative data.

The quality of the studies was assessed using the Critical Appraisal Skills Programme (CASP) checklist [20] for both qualitative studies and RCTs. Studies were evaluated based on research design, representativeness, recruitment procedure, presence of a comparison group, dropout rate, validity, reliability, and relevance of the measurement tools. Quality was graded with a traffic light system based on the 10 quality questions of the CASP and answered with yes (green) if the information was present, no information (red), and not available (yellow) if the information was not apparent or clearly outlined within the study (Multimedia Appendix 1 [21-48]). Studies were included in this systematic review irrespective of quality judgment.

Similar to corresponding systematic reviews on the acceptability of web-based interventions [10], the level of acceptability was categorized into the following quartiles: low (− −), moderate (−), high (+), and very high (+ +). This was specifically used for studies that reported a satisfaction rating on a scale, the percentage results of which could then be transferred to the suggested levels of acceptability. In addition, studies reporting dropout rates and compliance percentages were organized according to the 4-quartile rating system for acceptability. If studies reported mixed results, including positive and negative outcomes on different acceptability factors, they were rated with a tilde (~). Qualitative studies were synthesized in an integrative, meta-aggregative style following methodological guidance on the use of meta-aggregations [49] as well as using similar systematic reviews [50]. The key data were extracted and are outlined in Table 1 (see Multimedia Appendix 2 [31-33,35,38,39,45-48] for direct measures and qualitative data sources).

The characteristics of the studies are outlined in Table 1 as well as in Multimedia Appendix 2 including more details. Within the review process, 1303 papers were identified, of which 363 (28%) were duplicates, not published in English, or published before 2005. Of the 1303 papers, titles and abstracts were then scanned, a process that identified 940 (72%) and 379 (29%) papers, respectively. Papers were excluded if the interventions were independent of work environments, did not include most employees, and did not focus on mental health issues. Most studies were excluded owing to the involvement of face-to-face or telephone guidance by a coach or therapist. Ultimately, 28 studies were identified for further analysis, which either reported indirect measures of the acceptability of web-based interventions (n=17, 61%) or provided qualitative data on acceptability (n=11, 39%; Figure 1).

The 28 included studies had an overall sample of 9739 participants, with sample sizes ranging from 8 to 1140. The mean age of the participants was 40.7 years, and most participants were White and employed full-time.

The studies had various differing methodological designs and study characteristics, which are outlined in Table 1 as well as in Multimedia Appendix 2 including more details on direct measures. Interventions were heterogeneous in type, application outcome focus, length, setting (within a specific organization or random employees), and characteristics of the participants (type of profession and demographics). In addition, they differed in the level of support potentially provided in web-based format (email or message) versus unguided. Most interventions (18/28, 64%) were focused on reducing stress [21-35] and depressive symptoms [23,31,36-39] in employees. Other interventions focused on insomnia (2/28, 7%) [29,40], anxiety (2/28, 7%) [23,31], panic (2/28, 7%) [23,27], psychological detachment from work (1/28, 4%) [40], resilience (burnout; 1/28, 4%) [41], mood (1/28, 4%) [22], tinnitus distress (1/28, 4%) [42], chronic pain (1/28, 4%) [43], substance misuse (2/28, 7%) [23,25], and well-being or happiness (3/28, 11%) [31,38,44]. Studies and interventions often included more than one focus of mental illness, used various treatment techniques, and assessed the acceptability of general web-based mental health interventions [34,45,46]. Cognitive behavioral therapy (CBT) was the most used form of intervention (9/28, 32%). Other interventions applied mindfulness (3/28, 11%), psychological education (3/28, 11%), cognitive appraisal or restructuring (2/28, 7%), emotional regulation (1/28, 4%), acceptance and commitment therapy (1/28, 4%), problem solving (2/28, 7%), exercise (1/28, 4%), or communicational strategies (1/28, 4%). Approximately 7% (2/28) of the studies used tracking or assessment tools for burnout and mood (eg, depression, stress, or well-being). The studies were predominantly RCTs (13/28, 46%). CBT was mostly used in interventions that tackled depression or stress at work. For example, the CBT interventions focused on depression were HelpID, Mood Hacker, Colour Your Life, and MoodGym. The interventions that used CBT and aimed to reduce stress and mental strain were Psyfit, Strong at work, GetON Recovery, and Work Guru.

Of the 28 studies, 13 (46%) were RCTs, 4 (14%) were cross-sectional studies, 3 (11%) were qualitative studies, 3 (11%) were pilot studies, 2 (7%) were longitudinal studies, 2 (7%) were mixed methods studies, and 1 (4%) was a case report. The studies mostly used waitlist control groups, internet-based information website groups, or variations of intervention type groups as comparators. The studies originated in the United States (8/28, 29%), Germany (7/28, 25%), Australia (4/28, 14%), the Netherlands (2/28, 7%), Japan (2/28, 7%), Sweden (2/28, 7%), the United Kingdom (2/28, 7%), and Canada (1/28, 4%).

The methodological quality of the studies is summarized in Multimedia Appendix 1. The studies were assessed for quality using the CASP [20] qualitative and quantitative templates and reported in the form of a traffic light schema. Various quality flaws were outlined in the studies, and no study met all 10 criteria marked by the CASP [20]. Independent of quality, all studies (28/28, 100%) were included in the final synthesis. Allocation bias appeared to be low in the quantitative studies as participants were mostly randomly distributed to their condition (15/17, 88%). However, qualitative studies often indicated performance and detection bias as studies often missed reporting on blinding status or researchers’ awareness of the participants’ condition. Selection bias was generally high as participants repeatedly originated from specific population samples (eg, male-dominated industry workers or female educational staff). Attrition bias was predominantly high as various studies reported a high dropout rate, which weakened their generalizability. Several studies missed reporting on the specific demographics of their samples and, thus, might risk the presence of confounders, whereas other studies (4/28, 14%) clearly outlined their risk of confounding [25,26,30,42]. The analysis of quantitative studies was generally good as all studies used data from all participant groups in their final analysis. Qualitative studies showed generally good quality in the guidance of clear questions, taking care of ethical considerations, and the provision of clear information on methodology. However, various studies missed accounting for the potential bias caused by the relationship between the researchers and the participants.

The recruitment setting and included characteristics of the employees were very diverse (Table 1). Some studies (10/28, 36%) used the whole working population, recruiting samples via local insurance companies, occupational programs or employee assistance programs, random digit calling, or advertisements [24,26,28,34,36-38,43,44,46]. Alternatively, some studies used specific population samples originating from 1 type of profession. In particular, samples were recruited from the military (2/28, 7%) [35,47], telecommunications (2/28, 7%) [21,39], transport (2/28, 7%) [39,45], the public sector (1/28, 4%) [39], state fire and rescue services (2/28, 7%) [33,45], office and client employees (1/28, 4%) [32], university staff and teachers (3/28, 11%) [29,33,40], clinical staff and health professionals (4/28, 14%) [23,27,31,41], manufacturing and industrial workers (2/28, 7%) [30,42], marketing (1/28, 4%) [25], and technology (1/28, 4%) [22]. Most studies were conducted in the United States (8/28, 29%) and Germany (7/28, 25%). Germany primarily recruited from the general working population [26,34,36,44], whereas the United States mainly recruited from multiple specific locations, including larger organizations (eg, corporate call centers and technology companies), health centers, and military-related workplaces. Their acceptability results were mixed in outcome, ranging from very high in the study by Beiwinkel et al [36] to very low in the study by Hennemann et al [34]. The synthesis did not outline any pattern of setting and participant characteristics that was associated with the acceptability level of web-based interventions.

As outlined in the Study Characteristics section, most studies used CBT in their administered interventions (9/28, 32%). CBT was relatively equally distributed across Western countries, including the United States (2/9, 22%), Germany (2/9, 22%), the Netherlands (2/9, 22%), the United Kingdom (2/9, 22%), and Australia (1/9, 11%). Summarizing the CBT studies, the acceptability level indicated that 44% (4/9) of the studies had a low to moderate level of acceptability, whereas 33% (3/9) of the studies showed a high to very high acceptability level. Approximately 11% (1/9) of the studies had a mix of moderate and high acceptability levels. Other analyzed intervention types (mindfulness, psychological education, cognitive appraisal, emotional regulation, acceptance and commitment therapy, problem solving, cognitive strategies, exercise, and tracking tools) did not indicate any pattern of acceptability level. Broadly speaking, the intervention type, country of conduct, and outcome of the study did not indicate any notable patterns. However, most of the studies (26/28, 93%) were conducted in Western countries.

Relevant studies measured acceptability in different ways. They used direct measures of acceptability, which included qualitative data through questionnaires and interviews, or indirect quantitative measures by means of take-up, dropout, compliance, adherence, attrition, or completion rate. Some studies used both direct and indirect measures. All measures of acceptability are outlined in either Table 1 or Multimedia Appendix 2 (qualitative synthesized data) in the context of the reference, intervention, sample, study design, recruitment, outcome, indirect and direct acceptability measures, available reasons for dropout, example quotations from interviews, and an individually rated acceptability level.

Table 1 and Multimedia Appendix 2 present the direct outcome of employees’ acceptability of web-based therapy in the workplace. When categorizing the qualitative outcome into key themes, the following topics commonly emerged: (1) general interest in or willingness to use web-based interventions, (2) employees’ satisfaction rating of the utility of the interventions, and (3) preferred features of the design and application style of the interventions. Most participants reported a generally positive interest in and acceptability of web-based interventions [31,33,35,38,46]. However, there were mixed results and negative opinions in other studies [32,33,39,48]. Common preferred features of web-based mental health interventions were the use of nonstigmatized language [45,48], the preference for interventions with interactive support [39,45], and broad application spectrum as well as short mobile and interactive multimedia interventions [31,35,38,48]. The synthesized outputs of the studies were written in descriptions of each theme as well as provided within the context of the setting and intervention type. To deliver a deeper insight into common themes, Multimedia Appendix 2 provides quotations of interviewees in primary studies. As this systematic review synthesized key themes in an integrative, meta-aggregative way, quotations aid in the understanding of the summarized key themes. Most of the studies reported details on the satisfaction rating on a scale associated with web-based interventions. Employees were mostly satisfied with the interventions and rated their utility positively.

In addition, multiple studies assessed acceptability using satisfaction, usability, or interest ratings of the intervention (Table 1). Satisfaction ratings were frequently used in the studies [22,24,26,29,31,36-38,41,43]. The average satisfaction score was 82.6%, which is similar to a very high individual-defined acceptability level (++) of web-based interventions. Moreover, 14% (4/28) of the studies reported a score of 0.85% for practical use [22,31,38,41], equivalent to a high (+) acceptability. In particular, Wilson et al [47] reported a rate of 75% in “comfortability” of using a mental health program on the computer and an 84% rate in “willingness to use.” In contrast, Hennemann et al [34] reported that 89.1% of participants rated low on the “acceptability” of general occupational web-based mental health interventions. Both studies were very heterogeneous in intervention specificity and sample population. Hennemann et al [34] explained the negative outcome by the direct predictor variables of acceptability with “social influence,” “effort” and “performance expectancy,” and “time spent in the web” as well as the “frequency of searching online for health information.”

This systematic review included hypothetical measures of acceptability characterized by dropout, attrition, compliance, adherence, uptake, and completion rate. The indirect or hypothetical measures of the acceptability of web-based interventions in the workplace are summarized in Table 1. The mean percentage of dropout rates from the included studies was 50.9% with a range of 15.3% [25] to 67.7% [36], which is equivalent to a moderate individual-defined level of acceptability [23-25,27,36,39,44]. A few studies reported the reasons for dropout or termination of the interventions. Repeated reasons were lack of time [21,26,40], technical difficulties [26,27,39,40], younger age [23,27,36], lower education [36], lack of motivation [26,40], no need for help [40], ability to manage stress personally [22], dissatisfaction with the intervention [26,39], higher initial level of psychological distress [30], and privacy concerns [31]. Other measures of acceptability included an average attrition rate of 32% [24,37,42], an average adherence rate of 54% [25,28], an uptake and intervention start rate of 11% [23,27], and a completion rate of 68% [30,40]. As visible in the outcome, there was no clear consensus in acceptability level, and the comparison of studies was difficult as they were heterogeneous in study design, sample, and methodology. However, the most frequently reported indirect measure of acceptability was the dropout rate, supported by a moderate (−) level of acceptability.

This systematic review assessed the levels of employees’ acceptability of web-based interventions aimed at improving mental health. The findings showed a generally positive level of acceptability and highlighted various factors to be considered in making interventions acceptable, engaging, and useful for employees. Themes to be addressed with caution when introducing interventions are the use of stigmatized terminology, including words of ill health and mental illness. In terms of implementation, applications are recommended to be short and use interactive multimedia tools.

Results were obtained from 28 separate studies. Satisfaction ratings and feedback appeared positive, particularly when the interventions included multimedia and nonstigmatizing language. In particular, 79% (22/28) of the studies showed acceptability measures from high to very high, and 54% (15/28) of the studies reported acceptability levels from low to moderate (overlap when studies reported both quantitative and qualitative results). The average satisfaction rating was >80%, and the employees rated the interventions’ utility as good overall. However, quantitative measures contradicted the universal positive perspective of web-based interventions by means of the common measured dropout rate of approximately 50%. Hence, the attrition rate was very high in multiple studies, which questions the efficacy of unguided self-applied interventions.

Collectively, these results are in line with other acceptability studies that supported the general acceptability of web-based interventions in clinical settings [51]. Various studies have outlined barriers to assessing acceptability; for example, negative results from indirect measures. In addition, complications in synthesis owing to the heterogeneity of the interventions have been repeatedly reported.

Stigma and attitudes toward mental health at work were an emerging theme. Acceptability levels may relate to the web-based interventions themselves or to the fact that the intervention relates to mental health. This is supported by other studies showing that there is fear of stigmatization when seeking support [52]. It may also be difficult to successfully implement web-based interventions within an organization as employees prefer to separate health matters and their workplace [32]. Hence, the issue around mental health and stigma, especially at work, may be strongly influenced by the organizational culture that influences the use of mental health interventions [45].

The relationship between dropout and acceptability requires further assessment to interpret the current evidence. Although dropout for web-based workplace interventions was high (the mean percentage score of the included studies was 50.9%), explorations of the reasons for this were limited. Indeed, studies have outlined that dropout rates might not be the result of disinterest in occupational web-based interventions for mental health issues but appear to be generally high in computerized interventions [53], suggesting that these interventions are not as engaging as guided or face-to-face sessions and people might not feel committed enough to complete the treatment or program. Consequently, web-based interventions should be tailored and made as interactive and attractive as possible by using animation tools, pictures, and videos, as well as made as short and simple as possible to increase engagement and decrease the likelihood of technical issues [12]. Furthermore, the findings of this study suggest that, before applying interventions in organizations, people’s needs, the environment, and the culture should be assessed; the interventions should be tailored accordingly; and awareness of the benefits and understanding of the use should be addressed.

The generalizability of the findings across workplaces may be limited because of the diversity of individual workplaces; for example, their organizational culture and stigma or attitudes toward mental health. In addition, assessing for confounding variables, including recruitment, setting, intervention characteristics, and country of conduct, did not reveal significant information. However, most of the included studies were conducted in Western countries and used CBT-based interventions, which may further limit generalizability.

Assessing acceptability using indirect measures may be flawed as there could be multiple reasons for employees to stop the intervention. Specifically, dropping out of interventions could be the result of feeling rehabilitated and seeing no further benefit of using the intervention. Nevertheless, dropouts provide great insight into the acceptability of interventions, but more in-depth analyses of the reasons for dropping out should be conducted.

Analysis of the specific assessment of acceptability of occupational web-based interventions was limited because of the heterogeneity of the study designs, intervention types, sample characteristics, and conditions under which the interventions were provided to employees. The studies used data assessment techniques, including cross-sectional self-report methods, whereas the qualitative studies used small samples. Data collection and analysis biases may be observed based on the role of the researchers [54]. As qualitative acceptability results were generally higher compared with indirect measures, this further raises the question of the role of researcher bias. In addition, limitations regarding the consistent and objective measurement of acceptability in the wider literature prevent robust conclusions from being drawn. However, the inclusion and critical appraisal of qualitative studies may have added depth to the factors within the acceptability capture in this study [55].

Despite these limitations, this study offers a comprehensive insight into multiple forms of acceptability measures [56]. Using both qualitative and quantitative as well as direct and indirect measures of acceptability provided a deeper insight into the options for assessing the acceptability of interventions in general. Although this study focused on the workplace, it examined the acceptability of web-based interventions that could be applied more generally to support people’s mental health. For example, the findings could support the implementation of interventions outside of the workplace (eg, as part of clinical mental health treatments). These results might help clinicians, developers, researchers, and the health technology industry create effective and engaging tools in the future.

In relation to workplace practice, before applying interventions, it would be beneficial to increase people’s knowledge of web-based interventions as well as assess their needs in general to improve their attitude toward interventions [13,34]. This is supported by Murray et al [57], whose study found that participants who rejected computerized treatments had significantly lower expectations of the usefulness of self-help and had general concerns, anxiety, and misunderstandings about computerized treatments. Hence, acceptability may be increased by identifying and correcting misperceptions before participation. Similarly, tailoring interventions to the environment and employees’ needs could increase their general interest and willingness to use them [13]. In other words, web-based interventions for employees should be adapted to the specific environment applied as well as to the users’ needs to increase engagement and acceptability levels. Generally, the acceptability of interventions might increase if employees and organizations are made aware of evidence-based web-based interventions that have multiple practical benefits and the potential to increase individuals’ mental health and well-being in the long run. Finally, the ability of web-based interventions to engage and retain users is critical for ensuring reduced dropout and increased acceptability.

Regarding future research, the results of acceptability studies could be influenced by the general stigma on mental health topics and interventions. Therefore, future research should incorporate acceptability measures of mental health issues into their analysis to assess for confounding variables. Second, regarding quantitative data on acceptability, it would be beneficial if future research included a more in-depth analysis of the reasons for dropout or attrition rates. Third, future research should also address the conceptual and methodological limitations of the research in the field. If there were more organizations using mental health interventions from various settings, the research analysis could be more homogeneous. Organizations might lack the knowledge on how to apply personal health support but could provide their employees with interventions that range from broader aspects of stress management to specified apps that tackle specific mental health issues (eg, depression). Finally, this research was conducted before the COVID-19 pandemic, which changed work styles and environments and affected how people sought and received mental health support. Further research should analyze changes in acceptability as a result of the pandemic to examine shifts in use and acceptability of mental health interventions both within and outside of the workplace.

This study assessed the area of acceptability of web-based workplace interventions for mental health. In general, workers are open to web-based mental health interventions. However, qualitative and quantitative studies suggested varying levels of acceptability, raising the possibility of bias. The importance of stigma, organizational culture, and the implementation of the intervention were highlighted, the latter relating to the engaging design and quality of the intervention as well as the approach to delivery in the workplace itself. Several factors were identified that need to be considered to ensure the effective implementation of web-based interventions in the workplace, some aspects of which may also apply to the general use in supporting people’s mental health. Interventions should be tailored to the respective individual needs and cultural context, use nonstigmatized language, and be made interactive and easy to use. It is also recommended to foster an understanding of the potential value of an intervention to increase its acceptability. Methodological limitations were highlighted to guide the cautious interpretation and generalization of early evidence in this area along with the need to improve methodological rigor in emerging research.