Up to 27% of the workforce in Europe suffers from elevated stress levels [1]. According to the effort-reward imbalance model [2] and the job demand-control model [3] situations characterized by an imbalance between high effort (eg, workload) and low reward (eg, job insecurity) or high demand and low job decision latitude lead to high levels of strain. This strain is known to be a risk factor for psychological and physiological health consequences such as sleeping problems [4], mental health problems [5], cardiovascular disease [6], and chronic pain [7]. Consequently, the resulting economic burden due to productivity losses (eg, sick leave) [8] and higher health care consumption and out-of-pocket payments is substantial [9-11]. The estimated costs of work-related stress range from US $221.13 million to US $187 billion and therefore impose a tremendous burden on society [12]. Psychological interventions can be effective in reducing stress [13], but the availability of face-to-face treatments is limited [14]. Web-based and mobile-based interventions have been proposed to overcome the limitations of traditional face-to-face interventions. Such interventions are low-threshold interventions, are available 24/7, and are associated with low costs [15].

In a recent meta-analysis, it has been shown that internet-based stress management interventions (iSMIs) are effective with an effect size of d=0.43 (95% CI 0.31-0.51) on perceived stress [16] and a small effect on depression and anxiety, but lack evidence regarding cost-effectiveness. Internet-based interventions are often argued to be cost-effective, yet there exists little evidence. Donker et al [17] found that internet-based interventions for common mental health disorders have a considerable probability of being more cost-effective when compared with control groups. Most health economic outcome studies evaluated internet-based interventions for alcohol consumption [18], smoking cessation [19], anxiety [20], and depression [17,21]. Hedman et al compared an iSMI with an internet-based cognitive behavior therapy (iCBT) for treatment of health anxiety, where the iSMI resulted in lower costs [22].

To our knowledge, there exist no cost-effectiveness and cost-utility analyses of iSMIs from the societal perspective. Thus our aim was to establish the cost-effectiveness and cost-utility of this iSMI for employees.

This study is a health-economic evaluation with a 6-month time horizon from a societal perspective alongside a 2-arm randomized controlled trial (RCT) in Germany to establish the cost-effectiveness and cost-utility of an iSMI for employees with elevated work-related stress in combination with usual care compared with a waitlist control condition (WLC) with access to treatment as usual [23]. The present health-economic evaluation followed guidelines from the International Society for Pharmacoeconomics and Outcomes Research RCT-cost-effectiveness analysis Task Force report and the recommendations of the Consolidated Health Economic Evaluation Reporting Standard [24,25]. The trial included 264 participants who were randomly allocated in a 1:1 ratio with a block size of 2 to either iSMI or WLC. An independent researcher not otherwise involved in the study performed the randomization using randomization software (Randlist, Datinf GmbH) [26]. Participants were included in the study if they were 18 years or older, currently employed, and scored 22 or above on the Perceived Stress Scale (PSS-10). One SD (SD 6.2) above the mean (PSS-10=15.3) in a large working population [27] was chosen as a cut-off value to select participants with an elevated level of stress. The exclusion criteria were to be at risk of suicide or dissociative symptoms or having been diagnosed with a psychosis. The Ethics Committee of the Philipps-University of Marburg, Germany, approved the study. The trial was registered (DRKS00004749) in the German Clinical Trials Register.

The most popular models to explain work-related stress are the effort-reward imbalance and the job demand-control model. According to the effort-reward imbalance model [2], work-related stress is generated by high effort (eg, pace of work and workload) and low reward received in return (eg, inadequate salary, promotion prospects, and job security). The job demand-control model [3] identifies high demand (eg, high workload) and low job decision latitude (eg, autonomy and control over the job) as factors that lead to high levels of job strain. This strain is known to be a risk factor for adverse health consequences, such as mental health problems [5], chronic pain [7], and cardiovascular disease [6]. Ideally, job strain should be reduced by changing adverse working conditions such as small rooms and bad equipment. As changing these may be difficult, stressors on an individual level such as inadequate coping strategies can also be addressed. Interventions based on Lazarus’s transactional model aim to empower the individual to reduce or modify problems at work (ie, high effort, low rewards, or low decision latitudes). This model identifies 2 strategies of coping with stressors: problem-oriented coping, to actively change or adapt stressors, and emotion-oriented coping, to cope with negative emotions due to stressors at the workplace. Thus, the iSMI is based on 2 main components: problem solving and emotion regulation. Problem solving is an evidence-based method for dealing with such problems and has been proven to be successful in improving mental health [28]. However, employees are frequently faced with unsolvable problems, which are associated with strong negative affective reactions and require effective regulation strategies. Improvement of emotion regulation skills has been shown to be both promising for reducing psychopathological symptoms [29] and a mechanism of change in previous studies using this iSMI [30]. Deficits in emotion regulation may also be an important factor for the development and persistence of mental health symptoms [31]. Yet, emotion-focused coping is regarded as the forgotten component, whereas problem-focused coping by means of problem-solving techniques is a well-established component of most cognitive-behavioral stress management trainings.

The iSMI is based on Lazarus’s transactional model of stress and includes problem solving and emotion regulation. The intervention consists of 8 sessions composed of modules for psycho-education (session 1), problem solving (sessions 2 and 3), emotion regulation (sessions 4-6), planning for the future (session 7), and a booster session (session 8). In addition, participants could choose optional modules covering different topics, for example, time management, rumination and worrying, psychological detachment from work, and sleep hygiene. Each module takes approximately 45 to 60 min to complete. Participants were advised to complete 1 to 2 modules per week. Transfer tasks such as homework assignments were integrated into the intervention to help participants integrate learned skills into daily life. Participants received nontherapeutic feedback by an e-Coach after each completed module. E-Coaches had a degree in psychology, and feedbacks were based on a standardized manual on feedback writing. Participants could also opt in for an additional text message coach along the iSMI (eg, short relaxation exercises). A detailed description of the iSMI can be found elsewhere [32]. The clinical effectiveness of the iSMI has been positively evaluated in a series of RCTs [23,30,31,33,34].

Self-reported measures of stress and social functioning (PSS-10 and Short-Form Six-Dimension; SF-6D) were collected at baseline (T1), post treatment (T2; 7 weeks after randomization), and 6-month follow-up (T3) using a secured Web-based assessment system (AES, 256-bit encrypted).

We assessed direct and indirect costs which occurred over the previous 3 months at baseline, and at 6-month follow-up. All costs were calculated in Euros for the reference year 2013 (index factor 1.04 based on the year 2010), referring to the German consumer price index [38]. Costs were converted to US dollar using the purchasing power parities reported by the Organization for Economic Cooperation and Development. For the reference year 2013, €1 was equated to US $1.29.

The Trimbos Institute and Institute of Medical Technology Questionnaire for Costs Associated with Psychiatric Illness (TiC-P) adapted to the German health care system was used [39]. This is a widely used and reliable instrument for collecting self-reported data on health care utilization and productivity losses in patients with mild to moderate mental health conditions [40-46]. The German version has been used in a number of health economic evaluations alongside randomized trials [21,41,42,44]. The standard unit cost prices were multiplied by the units of resource use for each participant. Multimedia Appendix 1 presents direct medical and direct nonmedical costs by health service type. Cumulated costs of the trial were estimated using the area under curve method to linearly interpolate 3 months costs as measured at each measurement point to cover the full follow-up period of 6 months [47].

This study was powered to detect a mean difference of d=0.35 in the primary outcome (PSS) between the groups at post measurement. Cost data are usually heavily skewed to the right, with large variance requiring very large sample sizes to test the statistical significance of cost differences. Instead, we adopted a probabilistic decision-making approach for our economic analyses [54]. This procedure takes the stochastic uncertainty of the trial data into account [55] and informs the decision makers on probabilities rather than statistical significance. Due to the 6-month follow-up period, no discounting was applied.

All analyses were conducted in accordance with the intention-to-treat (ITT) principle. Missing clinical outcome data were imputed using a Markov Chain Monte Carlo multivariate imputation algorithm with 10 estimations per missing value.

Missing cost data were imputed using the regression imputation procedure implemented in Stata to obtain the required predicted values. Predictors of outcome and dropout were identified via (logistic) regression. Differences in PSS score and symptom-free status between groups were assessed at follow-up using the Chi-square test. At baseline, mean SF-6D utility values were similar in both groups (WLC: mean 0.65, SD 0.08 and iSMI: mean 0.65, SD 0.11). Therefore, no baseline adjustments were made when calculating QALYs. Differences in QALYs between iSMI and WLC were assessed using independent samples t tests.

For the cost-effectiveness analyses, the incremental cost-effectiveness ratio (ICER) was calculated as incremental costs per unit of effect (QALY and symptom-free status). Symptom-free status is meaningful for decision makers and was used as the preferred effect measure as there was no difference between beta coefficients from an OLS regression on the binary outcome compared with beta coefficients from a linear probability model in GLM (GLM: beta=.36, P<.001 and OLS: beta=.36, P<.001).

The ICER was calculated as ICER=(costs_iSMI−costs_WLC) /(effects_iSMI−effects_WLC), where costs are the cumulated costs over the 6-month period and effect are QALY gains or symptom-free status.

Stochastic uncertainty in the ICER was handled using nonparametric bootstrapping, which is a resampling technique applied to the trial data, which generates 5000 simulations of the ICER. The incremental costs and incremental effects were obtained under a bootstrapped seemingly unrelated regression equations model and allowed for correlated residuals of the cost and effect equations [56]. The 5000 bootstrap replications of costs and effects were also used to obtain 95% CIs based on the percentile method.

In a next step, the simulated ICERs were plotted in a cost-effectiveness plane. On the plane, incremental effects are depicted on the horizontal x-axis and the incremental costs on the vertical y-axis. Each dot in the cost-effectiveness plane represents 1 bootstrapped ICER.

The willingness-to-pay (WTP) threshold reflects the maximum amount the society would be willing to pay for a health benefit (eg, a symptom-free person or a QALY gained). As the WTP ceiling for gaining 1 unit of health (eg, gaining 1 QALY or obtaining symptomatic remission in 1 person) is an unknown quantity, a cost-effectiveness acceptability curve was presented, which displays the probability of the intervention being cost-effective for 1 additional unit of health gained at varying WTP ceilings. All analyses were performed using Stata version 13 [57].

Multimedia Appendix 2 presents the baseline characteristics. Interested participants were recruited from the general working population via mass media (eg, newspaper articles and television) and with the aid of a health insurance company (BARMER) within their occupational health management program. An open-access website [59] was used to sign-up for study participation. The sample predominately consisted of full-time employed middle-aged women living with a partner. A comprehensive description of the study sample and the participant flow can be found elsewhere [23]. We did not observe any clinically relevant baseline differences between study conditions.

The study attrition was low: 10.6% (28/264) of participants did not complete the 6-month follow-up assessment. The dropout rates between the groups, with 12.8% (17/132) in the iSMI condition and 8.33% (11/132) in the WLC condition, did not differ significantly (χ²₁=1.4 P=.23).

The iSMI improved by 9.75 (SD 6) PSS-10 stress units between pre and 6-month follow-up, whereas the WLC improved by 3.0 units (SD 6) PSS. Differences regarding symptom-free status based on the PSS-10 between groups were assessed at follow-up (iSMI: 79/132, 59.8%; WLC: 31/132, 23.5%; χ²₁=35.9; P<.001; NNT (Number needed to treat) =2.75, 95% CI 2.11-3.95) [23]. However, the intervention and the WLC did not differ significantly in terms of SF-6D QALY gains (iSMI=0.35, SD 0.04 vs WLC=0.35, SD 0.35; t₂₆₂=−1.625; P=.10).

At baseline, mean total costs were €3239 (US $4178) in the iSMI and €3183 (US $4178) in the WLC, which is only a small difference of €56 (US $72), indicating that randomization had resulted in a well-balanced trial. Table 1 presents the average 6-month accumulated per-participant costs by study condition. The costs are clustered into health care costs, patient and family costs, and costs stemming from productivity losses. After 6 months, total incremental costs were €380 (US $490); thus, the iSMI group had less costs than WLC (iSMI: €5258 and WLC: €5642). Health care costs were, on average, higher in the iSMI group compared with WLC. Hospital admissions were a major cost driver. Regarding the patient and family costs, the iSMI had less costs than WLC. Informal care was decreased by €241 for the iSMI. Finally, productivity losses produced the highest cost differences of €487, exceeding the intervention costs, meaning that the iSMI produced less cost than WLC.

Table 2 shows the incremental costs, effects, and cost-effectiveness ratios based on 5000 bootstrapped simulations. The bootstrapped ICER for symptom-free status on the PSS-10 was dominant. The cost-effectiveness plane is shown in Figure 1. The majority (70%) of the bootstrapped ICERs fell in the south-east quadrant, indicating a 70% probability that the intervention produces greater health at lower costs than WLC. Hence, the iSMI intervention dominates the WLC condition from a societal perspective. The remaining 30% of ICERs fell in the north-east quadrant, indicating a 30% probability that the intervention produces greater health at greater costs than WLC. Figure 2 presents the cost-effectiveness acceptability curve. If the decision maker is willing to pay €1000 and €3000 for gaining a symptom-free person, the intervention’s probability of being more cost-effective than WLC rises to 85% and 97%, respectively.

The ICER based on QALY gains showed a small health benefit (approximately 0.001 QALYs gained) for lower mean costs (€386; US $498). Of the simulated ICERs, 69% (as seen in Figure 3) fell in the south-east quadrant, reflecting the intervention’s probability of dominating WLC, whereas 26% fell in the north-east quadrant , indicating higher costs and health gains, and 2% fell in the south-west quadrant and 3% in north-west quadrant. Assuming a WTP of €10,000 and €20,000 for gaining 1 QALY, the probability rose to 73% and 76%, respectively (Figure 4).

Using the EQ-5D-3L resulted in a smaller incremental QALY gain in favor of the intervention group (0.28 QALY, SD 0.05) compared with WLC (0.28 QALY, SD 0.05), which was not statistically significant (t₂₆₂=−0.296; P=.77). This is in line with available evidence that the EQ-5D-3L suffers from ceiling effects in milder conditions [37]. Nevertheless, at a WTP of €20,000 for gaining a QALY, the probability of being cost-effective was 71%.

As inpatient costs were reported from only a few participants but were associated with high costs, these costs might have distorted the results. Excluding these costs led to higher ICERs for both outcomes (eg, symptom-free status and QALYs). The probability of being cost-effective rose to 86% and 96% at a WTP of €0 and €1000 with regard to symptom-free status, and 86% and 90% for gaining a QALY, respectively.

Increasing and subsequently reducing the intervention costs by €100 led to a 66% and 76% probability that the intervention produces a greater health gain at lower costs than WLC with regard to symptom-free status and 1-point improvement.

This study evaluated the cost-effectiveness and cost-utility of a Web-based guided self-help intervention for employees with elevated stress levels aimed at reducing perceived stress compared with WLC from the societal perspective. The intervention had a significant and favorable effect on perceived stress after 6 months and a high probability of being cost-effective compared with the control condition. The overall conclusion of this study does not change when using any of the assumptions, as explored in the sensitivity analyses.

First, we had missing data, which were handled using imputation techniques to perform an ITT analysis of both effects and costs [60]. As dropout rate was very low (12.8% for the iSMI and 8.33% for the WLC at 6 months), it is unlikely that this has biased the results substantially. Second, the costs and effects were only evaluated over a 6 months period. Hence, we cannot draw any conclusions about long-term effects. Third, self-reported costs and effects might have led to social desirability and/or recall bias. Nonetheless it seems unlikely that this bias differed systematically between groups due to absent baseline differences. Fourth, approaches used for cost estimation of lost productivity are based on the participants’ wages which do not reflect the average wages in the general population. Fifth, a waitlist control group design with unrestricted access to treatment as usual was chosen, which causes participants to be less motivated to initiate health-related behavior changes and thus over-accentuates effects [61]. Sixth, the majority of the sample was female, which is a common feature of mental health internet-based interventions [62]. The gender imbalance might limit the generalizability of study findings. Finally, the use of behavioral interventions does not result in improved working conditions that could cause less job strain. However, the potential of workplace-related interventions is often not fully utilized, and hence, such interventions are not systematically implemented. Thus, we recommend a combined implementation to design healthy working conditions.

The results of this study with an effect size of d=0.83 [23] on perceived stress are in line with the meta-analytic evidence (pooled effect size of d=0.43, 95% CI: 0.31-0.54) [16].

In addition, some evidence exists for the economic benefits of stress management and internet-based interventions to reduce depressive symptoms in employees. However, to the best of our knowledge, this study is the first study to evaluate the cost-effectiveness of a Web-based guided self-help intervention for employees with elevated stress levels.

Jacobsen et al evaluated the costs of a self- and professional-administered stress-management intervention not delivered over the internet in patients undergoing chemotherapy compared with usual care [63]. Lower costs and statistically higher quality of life outcomes were found in the intervention group. Hedman et al compared behavioral stress management with iCBT for treatment of severe health anxiety. The iSMI resulted in lower costs but was not considered cost-effective [22].

In a Web-based intervention by Geraedts [64], the probabilities of cost-effectiveness were 0.62 (societal perspective) and 0.55 (employer’s perspective) compared with WLC in employees with depressive symptoms. The intervention was not judged cost-effective. Besides that, the reduction of depressive symptoms was rather small (d=0.16) [65] compared with our study (d=0.64) [23] at post measurement. However, Buntrock et al reported an effect size of d=0.69 for a Web-based intervention for the prevention of depression. This intervention has an acceptable likelihood of being more cost-effective than enhanced usual care [21]. Focusing on perceived stress rather than on depressive symptoms in employees seems to be a cost-effective strategy to reduce the mental burden.

The results of this study support the idea that iSMIs could be a promising cost-effective strategy in reducing adverse effects of persistent stress in the workplace. Intervention costs were mainly driven by psychologists who acted as e-Coaches. Yet, studies showed that iSMIs are also effective when delivered in a less costly adherence-focused guidance and pure self-help format [33]. However, meta-analytic evidence shows that guidance yields higher effect sizes [66]. Therefore, the cost-effectiveness of guided versus unguided iSMI needs to be evaluated.

Long-term costs caused by persistent stress, such as staff turnover or mental health disorder onsets, were not taken into account. Future studies should investigate the long-term economic effects of iSMIs. The sample consisted predominately of middle-aged women. Future research should focus on the general German working population regarding recruitment, implementation, and dissemination.

This study demonstrated that this iSMI has a high probability of being cost-effective in reducing stress levels when compared with WLC. Given the increasing stress in the workplace and the small number of people who are reached via available health care services [67], it would be worthwhile to integrate such iSMIs into routine occupational health care, which conventionally only consists of face-to-face therapy by occupational health physicians.