Published on 26.04.19 in Vol 6, No 4 (2019): April
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/11401, first published Jul 10, 2018.
Evaluating the Feasibility of an Innovative Self-Confidence Webinar Intervention for Depression in the Workplace: A Proof-of-Concept Study
Background: Depression in the workplace is a very common problem that exacerbates employees’ functioning and consequently influences the productivity of organizations. Despite the commonness of the problem and the currently available interventions, a high proportion of employees do not seek help. A new intervention, a webinar (Web-based seminar), was developed, which integrated the use of technology and the traditional guided therapist support to provide accessible help for the problem of depression in the workplace.
Objective: The aim of this study was to explore the feasibility, preliminary outcome, and acceptability of the webinar intervention conducted in organizations.
Methods: In total, 2 organizations were invited to participate, and 33 employees participated in this proof-of-concept study. The webinar intervention consisted of 6 1-hour sessions conducted via the Adobe Connect platform, developed by Adobe Inc. The intervention was developed based on a systematic review, focus group studies, and face-to-face self-confidence workshops that utilized cognitive behavior therapy (CBT). The final webinar intervention used CBT and the coping flexibility approach. The structure of the intervention included PowerPoint presentations, animation videos, utilization of chat panels, and whiteboard features. The intervention was conducted live and guided by a consultant psychologist assisted by a moderator. Study outcomes were self-assessed using self-reported Web surveys. The acceptability of the intervention was assessed using self-reported user experience Web surveys and open-ended questions.
Results: The findings showed: (1) evidence of feasibility of the intervention: the webinar intervention was successfully conducted in 3 groups, with 6 1-hour sessions for each group, with 82% (23/28) participants completing all 6 sessions; (2) positive improvements in depression: the linear mixed effects modeling analysis recorded a significant overall effect of time primarily for depression (F2, 48.813=31.524; P<.001) with a Hedge g effect size of 0.522 at 1-month follow-up. Individually, 8 subjects showed significant reliable and clinically significant changes, with 3 subjects showing clinically significant change only; and (3) encouraging evidence regarding the acceptability of the webinar intervention among the employees: the user experience score was above average for 4 out of 6 domains measured (perspicuity mean 1.198 [95% CI 0.832-1.564], efficiency mean 1.000 [95% CI 0.571-1.429], dependability mean 1.208 [95% CI 0.899-1.517], and stimulation mean 1.323 [95% CI 0.987-1.659]). The open-ended questions also yielded 52% (47/91) of the responses that reported facilitators, whereas only 12% (11/91) of the responses reported barriers.
Conclusions: The self-confidence webinar intervention appears to be a potentially feasible, effective, and acceptable intervention for depression in the workplace that merits further investigation.
JMIR Ment Health 2019;6(4):e11401
Depression has been shown to cost employers in Europe £77 billion each year . However, despite the high prevalence of depression, employees are reluctant to seek help. One possible reason for this is that employees view seeking help as a sign of weakness rather than a medical condition that can be treated [ ]. Obstacles to seeking help can also be seen in the general public. The general public often conceptualizes depressive symptoms as problems of living rather than symptoms of a mental illness [ ]. Recent studies have found that interventions that used nondiagnostic labels such as self-confidence rather than depression are important in engaging groups who may prefer not to medicalize mental health [ , ]. This also provides an alternative and accessible route to psychological help, which is more congruent with the health beliefs of the public [ ]. This self-confidence program has also been shown to maintain effects after 2 years [ ].
Self-confidence workshops using a self-referral system, which were designed to be accessible to adults with depression in the community, found the workshops to be clinically effective and reached a large number of people who were reluctant to engage . However, participants were largely not employed [ , ]. This suggested that a different approach would be needed for the workplace. It has been suggested that the workplace is an ideal setting to provide information about depression to employees [ ].
A systematic review by Wan Mohd Yunus et al  reported that most evidence-based workplace interventions for depression were delivered face-to-face either individually or in groups. However, more recently, technological advances have been used in the mental health services and showed promising results, using methods such as interventions on the internet, software packages, and mobile apps. The integration of therapist support with technology-mediated interventions could also have a strong influence on new intervention processes and outcomes. It is likely that these technological advances could improve access and have other advantages such as reduced stigma, facilitating some people to share about sensitive issues, shortening waiting lists and cost-effectiveness [ ]. Despite these available interventions, mental health services are still underutilized by employees [ , ].
Given the technological advances, the importance of depression in the workplace and the problems in reaching people with depression who are employed, a more interactive form of computerized cognitive behavior therapy (CBT) using a webinar (a seminar delivered over the Web) was developed. Using the self-confidence label to make the intervention more accessible to people with depression, the webinar aimed to combine aspects of traditional face-to-face interventions and also take advantage of the technology that is currently available to develop an innovative webinar package for reducing depression in the workplace.
The intervention development process was in line with the Medical Research Council guideline for developing and evaluating complex interventions . It involves a systematic review [ ] and results from focus groups [ ], as well as previous research by Brown et al [ , , , ] and Cheng et al [ , ]. The new intervention incorporates a self-confidence webinar delivered during working hours in the workplace and is, to our knowledge, the first webinar intervention developed to target depression in the workplace. This webinar also used the more user-friendly and nondiagnostic label of self-confidence to reach out to more people, especially to those who were previously reluctant to seek help [ , , ]. This research reports on the preliminary findings of this new webinar intervention regarding its feasibility, preliminary outcomes, and acceptability, which was tested pragmatically with interested organizations.
The aim of this study was to assess feasibility, preliminary outcomes, and acceptability of running the self-confidence webinar in the workplace. This is in line with Eldridge et al’s recommendation that feasibility studies should include studies assessing whether a future study, project or, development can be done .
The first objective of this study was to assess whether the self-confidence webinar was feasible and whether the intervention could be implemented as planned. It was decided that the intervention would be feasible if these objectives were met:
- At least 20 employees were recruited;
- A total of 6 sessions were able to be conducted;
- Dropout rates were below 32%;
- Completion rates were at least 80% (participants attending all 6 sessions);
- There were no major technical issues that significantly affected the running of each session.
The second objective was to investigate the preliminary outcomes of a self-confidence webinar to reduce depression and to investigate whether there was an improvement in other measures including anxiety, self-esteem, coping flexibility, absenteeism, and presenteeism. The third objective was to investigate the acceptability of the intervention and whether participants were satisfied with the intervention.
This proof-of-concept study employs a single-group pre-post design. Assessments were taken at baseline, postintervention, and 1 month postintervention. The study was granted ethical approval by the King’s College London Research Ethics Committee (LRS-14/15-0857).
Procedure of Recruiting Organizations and Participants
The recruitment processes involved several stages: (1) Identifying interested organizations; (2) initial communication with interested organizations; (3) meeting with line managers and well-being teams to obtain permission from the organization; (4) if the organization was interested, on-site testing of technical aspects of readiness for the webinar was conducted; (5) organizations were provided with information about the study, which was then disseminated to the organizations’ employees. Employees thus received flyers and an information sheet about the webinar and research from the gatekeeper (the lead person from each organization). The information sheet included detailed information about the study, whereas the flyer included an invitation to the introductory meeting; and (6) If any employees were interested, they were asked to contact the first author and were invited to the introductory meeting held in an office in the organization. Alternatively, if they were unable to attend the introductory meeting, interested employees were provided with further information about the study. To maintain confidentiality, employees’ managers were not informed about which employees had participated.
Before the webinar, consent from line managers or team leaders was obtained as the sessions were to be conducted during working hours, as this was a strong view from the focus group that was conducted before the webinar was offered .
The Template for Intervention Description and Replication checklist and guide was employed to describe the intervention . The self-confidence workshop program was offered as Web-based seminars using a Web-based webinar provider (Adobe Connect) with the participating employees. The webinar intervention for improving self-confidence lasted 6 weeks, with 1 session per week. Every session involved an hour-long live webinar, which included time for questions as well as homework. Each session involved a PowerPoint presentation with videos and comic strips, an Interactive Zones component where participants interacted with each other, a virtual whiteboard, use of the chat feature, and a webcam feature for the therapist. To attend the sessions, participants could use a computer, laptop, or mobile device including smartphones (iPhones or Android phones) and tablets (iPads and tablets). Thus, attendance at these sessions was possible wherever the internet was available. The content of the sessions was based on a cognitive-behavioral approach by Brown et al [ , , , ] and coping flexibility by Cheng et al [ ] and is outlined in . includes screenshots of the webinar sessions.
The Beck Depression Inventory II (BDI-II) is a widely recognized 21-item self-report inventory for the assessment of depressive symptoms. Each item consists of 4 statements scored on a scale ranging from 0 to 3 with higher scores indicating higher depressive symptoms. The scores can be calculated and categorized into 4 levels of severity: minimal (0 to 13); mild (14 to 19); moderate (20 to 28); and severe (29 to 63) .
The Beck Anxiety Inventory is a 21-item self-report inventory to assess overall anxiety. Respondents are asked to rate the severity of each symptom using a 4-point scale with higher scores reflecting higher anxiety. The sum of all items corresponds to 4 levels of severity: minimal anxiety (0 to 9); mild anxiety (10 to 16); moderate anxiety (17 to 29); and severe anxiety (30 to 63) .
The Rosenberg Self-Esteem Scale is a 10-item self-report inventory to measure individual self-esteem. Each item comprises a 4-point scale which contains 5 positively and 5 negatively worded items. The scores of all items are totaled with a higher score suggesting a higher level of self-esteem .
Coping flexibility was measured using the 10-item Coping Flexibility Scale. This self-report inventory measures an individual’s ability to evaluate coping strategies and adopt alternative coping strategies depending on the situation. Respondents are required to respond to the items on a 4-point scale, and the sum of all items represents the overall score of coping flexibility with a higher score indicating a higher level of coping flexibility .
Presenteeism and Absenteeism
The short form World Health Organization Health and Work Performance Questionnaire is a self-report inventory that measures absenteeism and presenteeism. For absenteeism, respondents respond to 8 items that relate to the number of hours lost per month, with a higher score indicating a higher amount of absenteeism. Absolute absenteeism is measured in raw hours where a negative lower bound suggests that employees work more than expected and a higher score indicates a higher amount of absenteeism. Relative absenteeism is expressed as the percentage of expected hours whereby a negative number suggests that individuals work more than expected, and a score of 1.0 suggests that the individual is always absent . There are 3 items for presenteeism that correspond to reduced work performance with a higher score indicating a lower amount of lost performance. An absolute presenteeism score with a lower bound of 0 suggests a total lack of performance during the time on the job, whereas an upper bound of 100 indicates no lack of performance during the time on the job. For relative presenteeism, a minimum score of 0.25 indicates the worst relative performance compared with other employees, whereas a maximum score of 2.0 indicates the best performance compared with other employees [ ].
User Experience Questionnaire for Acceptability Measure
The User Experience Questionnaire (UEQ) is a self-report inventory to assess the ability of a product to engage the user. It consists of 26 items grouped by 6 domains: Attractiveness, Efficiency, Perspicuity, Dependability, Stimulation, and Novelty. Each item is randomly ordered along a 7-point scale representing 2 graded contrasting attributes . Values between –0.8 and 0.8 represent a neutral evaluation of the corresponding scale. On the contrary, values >0.8 represent a positive evaluation, whereas values <–0.8 represent a negative evaluation. Although the range of the scale is between –3 (horribly bad) and +3 (extremely good), it is extremely unlikely to observe values above +2 or below –2 [ ].
Open-Ended Satisfaction Questions
Additionally, 4 open-ended questions were also included in the postintervention Web-based questionnaire. The questions were as follows:
- “If the webinar was to run again, can you please recommend things we should start doing that we are not currently doing?”
- “What are the things we should stop doing that we did during this webinar that did not work for you?”
- “What are the things that worked well that we should continue doing?”
- “Please type in the space below if you have any other comments/feedback or suggestions about the webinar program.”
Data Analysis for Outcome Measures
Descriptive statistics were used to report the scores at baseline (T0), postintervention (T1), and 1-month postintervention (T2). The data were then analyzed using a mixed model analysis or a nonparametric Friedman test to analyze the difference in scores at T0, T1, and T2. For linear mixed effects modeling, the required assumptions were initially met for self-esteem and coping flexibility. Attempts to transform the data were performed on other variables but were only successful for depression and anxiety by using square root transformation. For absolute absenteeism, relative absenteeism, absolute presenteeism, and relative presenteeism variables, the nonparametric Friedman test was performed.
Effect sizes and reliable and clinically significant changes were computed using data from completers at 1-month follow-up as well as those who only provided postintervention data but did not respond to the follow-up. Postintervention data were used instead if the follow-up data were not available.
The standardized effect size was calculated using both Cohen dav and Hedge g correction. Lakens  recommended the use of Cohen dav for within-subject research design but because this is positively biased as it is based on a sample estimate, Hedges gav correction was also applied. The effect size (Hedge gav) of the change in scores was calculated manually using a scientific calculator based on the following formula:
Hedges gav=Mdiff/[(SD1 + SD2)/2] x [1–3/[4(n1 + n2)–9]
The reliable and clinically significant changes for depression were calculated using the following formulas [- ]:
Reliable Change Index (RCI)=√(2 [SDpre x √(1–α)]2) x 1.96
Clinically significant change=[(meanclin x SDnorm) + (meannorm x SDclin)]/(SDnorm + SDclin)
The number of participants and their depression level classification at each time point are also described.
In total, 2 participating organizations were recruited. One organization was an administrative body funded directly by the government. The other organization was a local authority of one of the 32 London borough councils in the United Kingdom.
In total, 33 employees volunteered to participate.displays the demographic characteristics of the study participants.
Objective 1: Feasibility of the Self-Confidence Webinar
All feasibility criteria were met, indicating that the self-confidence webinar was feasible for a larger study in the future. This study showed that:
- 33 employees were recruited;
- The webinar was successfully conducted for 3 groups, with 6 sessions for each group;
- The dropout rate was 24% (8/33);
- All 6 sessions were completed by 82% (23/28) participants;
- There were no major technical issues that significantly affected the running of each session.
The initial list consisted of 6 organizations, where interest from these organizations was initially established from previous research or contact involving the third author. However, following initial contact, 2 organizations did not follow up their interest and another 2 chose to opt out because of practicality (confidentiality and open plan office) and information technology resource issues. In total, 2 organizations decided to proceed with the next stage.
A total of 37 employees attended the on-site introductory meetings. There were also some employees who contacted the first author directly for further questions and/or registered their interest but did not come to the meeting. Overall, there were 38 employees who attended or contacted the first author directly who ended up registering their interest with the webinar. Of the 38 employees, 33 provided consent and submitted the Web-based baseline questionnaire, following ongoing email and telephone communications with the first author. Reasons for nontakeup included 2 who did not respond to the invitation email, 1 mistakenly thinking it was a face-to-face program, 1 being unable to take part during the specified day and time, and 1 being too busy with work demands.displays the summary of the recruitment and reach process.
|Age (years), mean (range)||39.6 (21-62)|
|Gender, mean (SD)|
|Marital status, mean (SD)|
|Married or civil partnership||10 (30.3)|
|Living together||9 (27.3)|
|Highest level of education, mean (SD)|
|University degree||17 (51.5)|
|Postgraduate degree||6 (18.1)|
|A level or National Vocational Qualification||5 (15.2)|
|Diploma or Business and Technology Education Council||5 (15.2)|
|Ethnicity, mean (SD)|
|White—English or Welsh or Scottish or Northern Irish or British or Irish or any other white background||18 (54.5)|
|Black or African or Caribbean or black British—Caribbean or African||6 (18.2)|
|Asian or Asian British—Indian or Bangladeshi or any other Asian group||6 (18.2)|
|Mixed or multiple ethnic groups—white and black Caribbean or white and black African||3 (9.1)|
|Work experience (years), median (range)||18 (2-45)|
In total, 2 organizations participated and each held one introductory session. Owing to the timing preferences of the employees, 3 webinar groups were held for 33 participants. The webinar presenter and first author, who acted as the moderator, conducted all 6 sessions of the webinar, giving a total of 18 sessions conducted for the 3 groups.
Of the 33 participants who initially registered, 28 took part in the first session and 25 took part in the sixth session, meaning 8 participants dropped out (24%). However, if only those who attended session 1 are included, the dropout rate was only 11% (3/28). Notably, 82% (23/28) participants completed all 6 sessions and 86% (24/28) participants completed at least 4 sessions. The reasons for not being able to attend the live session included attending other meetings and being on leave. The follow-ups at postintervention and at 1-month postintervention were conducted over the Web and blind-carbon copies, with reminder emails, were also sent intermittently. A total of 73% (24/33) postintervention questionnaires were obtained at postintervention and 70% (23/33) questionnaires obtained at 1-month follow-up.
Objective 2: Preliminary Outcomes of the Self-Confidence Webinar
shows the means and SDs of the outcome measure scores. Depression, anxiety, absolute absenteeism, and relative absenteeism decreased at T1 and T2, whereas self-esteem, coping flexibility, absolute presenteeism, and relative presenteeism increased at T1 and T2.
|Measures||T0 (n=33), mean (SD)||T1 (n=24a), mean (SD)||T2 (n=23a), mean (SD)|
|Depression||13.91 (9.77)||8.63 (8.71)||9.09 (8.47)|
|Anxiety||11.42 (9.17)||10.42 (10.98)||9.09 (9.96)|
|Self-esteem||18.15 (4.87)||19.63 (5.75)||20.09 (5.67)|
|Coping flexibility||14.27 (6.17)||18.42 (4.93)||18.96 (5.14)|
|Absolute absenteeism||8.52 (27.51)||1.54 (17.30)||5.20 (25.58)|
|Relative absenteeism||0.06 (0.20)||0.02 (0.15)||0.03 (0.18)|
|Absolute presenteeism||57.58 (14.15)||69.58 (15.46)||71.30 (13.92)|
|Relative presenteeism||0.78 (0.18)||1.01 (0.23)||1.19 (0.65)|
aOnly available data are included.
Linear Mixed Effects Modeling
To get a preliminary estimate of the improvement in the outcome measures in a group of participants for the webinar intervention, linear mixed effect modeling was conducted. Time of assessment (T0, T1, and T2) was included as a main effect in the fixed factors of the model to account for the 3 different time points of the assessment. A random intercept by participants was included in the model to account for the different scores at baseline. An autoregressive -AR(1) covariance matrix was used to account for constant variance at each time point assuming that the correlation gets less as time points get further apart (T0, T1, and T2). Maximum Likelihood estimation was therefore chosen to fit the models.
The overall effect of time was significant for all measures: depression (F2, 48.813=31.524; P<.001), anxiety (F2, 49.428=3.945; P=.03), self-esteem (F2, 49.519=31.524; P=.02), and coping flexibility (F2, 48.623=14.184; P<.001).shows the summary of the result of the mixed models analyses for depression, anxiety, self-esteem, and coping flexibility.
|Time point||Depressiona||Anxietya||Self-esteem||Coping flexibility|
|Mean (95% CI)||P value||Mean (95% CI)||P value||Mean (95% CI)||P value||Mean (95% CI)||P value|
|T0||3.492 (3.030-3.954)||—b||3.064 (2.529-3.599)||—||18.152 (16.375-19.928)||—||14.273 (12.371-16.175)||—|
|T1||2.400 (1.920-2.880)||—||2.717 (2.159-3.276)||—||19.510 (17.651-21.369)||—||18.550 (16.487-20.613)||—|
|T2||2.514 (2.001-2.993)||—||2.415 (1.836-2.994)||—||20.324 (18.394-22.254)||—||19.380 (17.217-21.542)||—|
|T0 versus T1||1.092 (0.751 to 1.433)c,d||<.001||0.346 (–0.078 to 0.770)c,d||.14||–1.359 (–2.813 to 0.095)c,d||.07||–4.277 (–6.434 to –2.120)c,d||<.001|
|T0 versus T2||0.995 (0.530 to 1.460)c,d||<.001||0.649 (0.075 to 1.223)c,d||.02||–2.173 (–4.133 to –0.212)c,d||.02||–5.107 (–7.861 to –2.353)c,d||<.001|
aDue to non-normality of the residuals, square root transformation was performed.
b—: not applicable.
cSignificance level P<.05.
dMdiff (95% CI).
Both absolute and relative absenteeism were maintained throughout the 3 time points at T0, T1, and T2 (median=0), and the differences were not statistically significant (absolute absenteeism, χ22=2.358; P=.31; Relative absenteeism, χ22=1.701; P=.43).
For presenteeism outcomes, pairwise comparisons were performed with a Bonferroni correction for multiple comparisons. Absolute presenteeism was statistically significantly different at the different time points during the study period; χ22=22.116; P<.001. A posthoc analysis revealed statistically significant differences in absolute presenteeism from T0 (median=60.00) to T1 (median=70.00; P=.006) and T0 to T2 (median=80.00; P<.001). Relative presenteeism was statistically significantly different at the different time points during the study period; χ22=26.325; P<.001. A posthoc analysis revealed statistically significant differences in absolute presenteeism from T0 (median=0.750) to T1 (median=1.000; P=.002) and T0 to T2 (median=1.125; P<.001).
The results are summarized in. Overall, improvements were recorded for all measures at T1 and T2 when compared with T0 with the highest effect size recorded for presenteeism measures, followed by coping flexibility and depression. The positive Hedges gav indicated a reduction in scores, whereas the negative Hedges gav indicated an increase in scores.
The reliable and clinically significant change analysis only includes participants who completed the assessment at all 3 time points or those who completed the assessment at T0 and T1 or T2. The SD of the BDI-II at baseline (SDpre) was 10.18, whereas the calculated Cronbach alpha was .92. The calculated RCI was 7.98. Thus, the changes in BDI-II scores can be categorized into 3 groups: a reliable increase in depression symptoms (an increase of 8 points more on the BDI-II), no reliable change (less than 8 points increase or decrease on the BDI-II), and reliable improvement in depression symptoms (a reduction of 8 points or more on the BDI-II). On the basis of the data in, a total of 35% (9/26) participants showed reliable change and a reduction in depression symptoms. In contrast, 65% (17/26) participants showed a change smaller than this.
|T0 versus T1||T0 versus T2|
|Clinically significant change (score criterion of >12)||Total at baseline||Reliable change postintervention (change of score of >8)|
|Started lower than the criterion for clinically significant change (baseline score <12)||10||0||10|
|Started higher than the criterion but failed to achieve clinically significant change (baseline score >12 but postintervention score still >12)||5||1||4|
|Clinically significant change (baseline score >12; postintervention score <12)||11||8||3|
Clinically Significant Change
To calculate this change, normative data from a community sample of 7500 respondents in another European study using BDI-II were used, which showed a mean score of 10.6 and an SD of 10.9 . Hence, using the formula in the Methods section, the clinically significant change criterion score was calculated to be 12.34. In summary, 38% (10/26) participants started with baseline BDI-II scores below the criterion and therefore did not show clinically significant change and none showed reliable improvement. In total, 19% (5/26) participants started with baseline scores above the criterion but failed to improve to a lower score below the criterion, with 4% (1/26) participant showing reliable improvement. In total, 12% (3/26) participants did not record reliable change despite recording clinically significant change. This suggests that it is not sufficient to imply that the change is not influenced by simple measurement unreliability [ ]. Finally, 42% (11/26) participants showed clinically significant change, of whom 31% (8/26) also showed reliable improvement. summarizes the data for reliable and clinically significant change.
Depression Level at Different Time Points
The distribution of scores for depression at each time point is presented in. The figure only includes available data at the 3 time points. At baseline, nearly a quarter of the participants had a moderate-to-severe level of depression symptoms. Generally, improvement was observed in terms of the depression severity when compared with baseline. The percentage of employees in moderate and severe depression severity decreased when compared with baseline.
Objective 3: Acceptability of the Self-Confidence Webinar
On the basis of the result of the UEQ, participants reported satisfaction in all domains: Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty.
Although figures are not available for comparing the webinar intervention with any other type of psychological intervention, the UEQ is able to compare the measured user experience with the results from 163 different types of products such as business software, Web pages, Web shops, and social networks with a total of 4818 participants . displays the user experience results and the interpretation of the results of the webinar intervention compared with the user experience of other products.
|Scale||Mean (95% CI)||Relative comparison to other products||Interpretation|
|Attractiveness||1.618 (1.242-1.994)||Good||10% of results better; 75% of results worse|
|Perspicuity||1.198 (0.832-1.564)||Above average||25% of results better; 50% of results worse|
|Efficiency||1.00 (0.571-1.429)||Above average||25% of results better; 50% of results worse|
|Dependability||1.208 (0.899-1.517)||Above average||25% of results better; 50% of results worse|
|Stimulation||1.323 (0.987-1.659)||Above average||10% of results better; 75% of results worse|
|Novelty||1.083 (0.718-1.449)||Good||10% of results better; 75% of results worse|
Facilitators, barriers, suggestions, and general comments were analyzed separately using qualitative content analysis on each question using Microsoft Excel. As this was likely to be the first study exploring the application of a webinar intervention within the workplace setting, an inductive approach was chosen . The responses were divided into 4 main categories: facilitators 52% (47 out of 91), barriers 12% (11 out of 91), suggestions 14% (13 out of 91), and general comments 22% (20 out of 91). Example quotes from the feedback corresponding to each category are presented in the .
The findings of this study indicate that the innovative self-confidence webinar intervention is feasible, and positive improvements were observed particularly on depression, presenteeism, and other outcome measures. Moreover, employees also reported that the webinar intervention was very acceptable.
The webinar recorded low dropout rates and high completion rates among employees who registered. Although the sample size was small, the self-confidence webinars also had low dropout rates with high completion rates among those who attended the first session. Of the 33 registered participants, 25 (76%) completed the final session, giving a dropout rate of 24%. Of the 85% (28/33) participants who attended the first session, 82% (23/28) completed all 6 sessions, giving a dropout rate of 18%. These figures are favorable than the completion rates (66%) in a much larger multicenter randomized controlled trial (RCT) of face-to-face psychoeducational CBT workshops  and multiservice practice research using the Stress Control (SC) program that recorded a completion rate of 70% [ ]. The SC program study highlights the importance of participants’ subsequent attendance as one of the predictors for a positive outcome.
There is much evidence of the high dropout rate for computerized CBT. A systematic review on barriers and uptake of computerized CBT reported that those receiving computerized CBT were twice as likely to drop out as those receiving face-to-face CBT . Another meta-analysis on computer-based psychological treatment for depression however found a variation and reported a dropout rate of 28% for interventions with therapist support, 38.4% for administrative support, and 74% for no support [ ]. The dropout rate of 24% found in this study is therefore comparable with that of supported psychological therapies.
The fact that the presenter and moderator played significant roles in the webinar throughout all sessions indicates that it was a highly guided intervention. Hence, the dropout rates and completion rates among the webinar participants were comparable with other guided internet interventions. A meta-analysis review of the dropout rates for guided internet interventions compared with face-to-face interventions showed no significant difference . Moreover, the percentage of completed sessions by the subjects was also found to be comparable between guided internet-based interventions and face-to-face interventions for depression. A study reported that the mean completion rate for face-to-face CBT was 83.9%, whereas the mean completion rate for guided internet CBT was 80.8% [ ]. The review highlights the importance of the guided component within a technology-mediated intervention.
It is also important to note that the self-confidence webinar was accessed by employees in different age ranges (range=21-62; mean 39.6), as age has been found to be a highly significant factor in accessibility and utilization of the internet and new technology . It has been argued that the use of the internet falls off sharply with age, owing to factors such as attitudinal beliefs, age-related changes, cognitive barriers, and privacy concerns [ ]. It was not found to be the case with the webinar.
However, there are several challenges that need to be highlighted. First, the difficulties in the recruitment process of organizations need to be mentioned. One organization had undergone restructuring during the study period, which involved the movement of employees, which in turn affected the communication between the first author and the organization. Olsen et al  highlighted these challenges and suggested that changes in the organizations should be considered when planning intervention studies; this, in turn, can cause delays and communication problems with the research.
Another aspect was the competing time commitments of employees. Owing to the live nature of the webinar, employees are required to access the webinar intervention during a predetermined date and time. Although recorded versions of the webinar were available, it was felt that the learning experience and contribution of participants would be more restricted than attending live sessions. When this problem did occur, employees were provided with the full recording of that particular webinar session. With this, they were able to follow the session asynchronously and obtain some input about the session.
Technical barriers to accessing the webinar program may be important. For instance, in one of the organizations, there was an initial difficulty in accessing the webinar program because of the tight internet security settings used by the organization. This was used to restrict the use of internet connection to only Web pages and links that were directly related to work. Although this issue was successfully overcome, it is important for future research to take note of this problem and make sure that on-site testing is conducted before the intervention is implemented. Furthermore, it is important to make sure that the computer, laptop, or mobile devices fulfill the minimum technical specification required to run the webinar program.
The follow-up results after 1 month suggested positive changes for all outcome measures. For depression, the analysis indicated that symptoms of depression significantly improved at postintervention and 1 month postintervention compared with baseline with moderate effect sizes of 0.563 and 0.522, respectively. This result was lower than the 0.844 effect size obtained in the RCT study of group face-to-face self-confidence workshops to reduce depression for the public . Nevertheless, the outcome of this webinar study suggested that adaptations to the intervention content, through a webinar platform, produce encouraging results for the intervention in the workplace, especially considering that this is a universal intervention. This small pre-post study provides preliminary support for the position that a guided webinar form of delivery of an intervention can provide positive effects for individuals.
Notably, although the intervention was presented as self-confidence and targeted self-esteem, it was interesting that the difference in effect sizes for self-esteem was less significant than that of depression with recorded effect sizes of 0.563 postintervention and 0.522 at follow-up for depression and –0.275 postintervention and –0.363 at follow-up for self-esteem. More importantly, this may suggest that the use of a nondiagnostic label of self-confidence, compared with the more medicalized label of depression, was successful in attracting people with some degree of depression. Consistent with other studies, the use of a diagnostic label, such as depression or insomnia, may affect the uptake and engagement of intervention with people who would prefer not to medicalize mental health [, , ].
An important finding from this study was that the largest effect size found at 1 month postintervention was for the outcome measurement of presenteeism (absolute presenteeism=–0.963 and relative presenteeism=–0.962). This is a significant finding as the intervention contributes a positive impact to work-related outcomes such as presenteeism.
Although the positive outcomes may be contributed to by intervention content, it is also important to highlight the group nature of the webinar. During each session, participants interacted with other participants, as well as the webinar presenter; this was especially the case during the interactive zones. Previous studies have provided the benefits of group intervention, such as providing natural social networks, being cost-effective and more accessible, providing opportunities to learn and support others, having less stigma, and obtaining more neutral views from other participants rather than the therapist, as well as being useful for those who are uncomfortable or struggle with individual intervention [- ].
Owing to the universal nature of the intervention, it was expected that participants would consist of those with mixed levels of self-reported depression. It follows that there were participants with few or no depressive symptoms reported at baseline as well as participants with varying depressive severity levels, supporting the findings from a systematic review that universal interventions also attract those with higher levels of depression . The mean score for depression among the participants was 13.91 on the BDI-II, indicating mild depression. More importantly, in line with previous self-confidence workshops that used a self-referral system and where 75% met diagnostic levels of problems [ ], the webinar intervention also attracted those with higher levels of depression and not just those who are worried well.
Acceptability of the Self-Confidence Webinar
Furthermore, positive facilitator factors were identified; the interactive zones and content recorded the highest number of positive feedback comments. The webinar consists of 2 main components: visual (eg, share visual files, webcam, and virtual whiteboard) and audio (eg, telephone or voice over internet protocol) components that provide different ways to facilitate interaction . It is reassuring that participants found the content and intervention to be useful for them, indicating that the material had been well-adapted for this purpose. Most of the positive feedback was related to the implementation of the interactive zones, which was the main adaptation made to allow engagement between participants and the webinar’s presenter. This is encouraging, as this finding suggests that the application of a Web-based interaction platform through the webinar was well-received by the intervention participants. Previous evidence has also reported that webinars with its interactive features may facilitate participant interaction while being capable of providing a communication environment that is nearly the same as a face-to-face environment [ ]. Notably, an important element of the webinar is the presence of a therapist and moderator. Moreover, with consent from the participants, each session was also recorded, and the participants could access the session anywhere and anytime. These benefits are in line with the findings from a Delphi study that highlighted the importance of effective moderation within an online discussion forum and 24-hour mobile access for the design of a Web-based intervention [ ].
An important feature of the webinar that emerged during the study was that it can be accessible without restricting geographical context. In this study, a participant reported attending a session from abroad during a 3-week holiday. Another participant also reported attending a session at home while on leave rather than at work. Even with limited evidence, this study showed that some of the barriers to accessing treatment, which may be present in normal face-to-face intervention, were overcome in the webinar intervention.
Limitations and Future Directions
There were several limitations of this study. First, the timeframe and nature of the recruitment process made it possible to only have an intervention group with no control group. The absence of a control group means it is not possible to determine whether the improvement in outcome scores is because of the intervention or other factors not covered in the study, such as time or a placebo effect. Second, the small sample size limits the ability to generalize from these results about other employees in other organizations. Hence, the results and conclusions derived from this study need to be interpreted with caution. Third, the unbalanced number of male and female participants may limit the extent to which the results can be generalized to males. Although this is a common pattern among mental health intervention studies, it needs to be noted. Fourth, all outcome measures were based on self-reports, which limits the quality of the data; this was the case for the absenteeism measure. Access to data on absenteeism via the participating organizations was not planned nor deemed feasible within the research timescale. Finally, the study only utilizes a short follow-up (1 month postintervention). This short follow-up time limits the study findings to the short-term only.
Given the promising results, there needs to be further evaluation of the effectiveness of the intervention. This can be done gradually by starting with a simpler research design (eg, nonrandomized control trial) to a more complex research design (eg, RCT), depending on the feasibility, timeframe, and available expertise. Additionally, it would be more informative to assess the effects of the webinar intervention with longer follow-up (eg, 3 months). Given that this is a universal intervention, the intervention can be compared with other forms of universal interventions in the workplace as well as with a wait-list control group. As this was a complex intervention, further investigations are also needed in terms of understanding the active ingredients and change processes through a mediation and mediator analysis, as well as assessing the cost-effectiveness. These figures can then be compared with those for other face-to-face or internet interventions.
As this intervention is designed for employees within the workplace, it is also important to consider the study design possibilities bearing in mind the hierarchical nature of the organization, in which employees can be nested in teams and departments within the organization’s organizational structure. In relation to this, future trials may consider cluster randomization as opposed to individual randomization considering that each employee is usually attached to the work structure of their respective teams or departments or organizations. Additionally, future trials need to also include procedures and time for recruiting at the organization and individual levels, which involved identifying and recruiting interested organizations, obtaining organizational permission and agreement, on-site technical readiness assessment, and introductory meetings as well as recruiting employees.
This proof-of-concept study provides preliminary evidence that self-confidence webinars can be a potentially feasible, effective, and acceptable intervention for depression in the workplace. The webinar intervention appears to be feasible as indicated by the successful recruitment of employees, the webinar sessions successfully running, relatively low dropout, and high completion rates as well as no major technical issues. The outcome analysis reported a lower level of depression, anxiety, absenteeism, and presenteeism as well as improved self-esteem and coping flexibility among employees postintervention and 1 month after completion of the intervention. It has also been shown to be acceptable.
In addition to its potential effectiveness in an RCT, the self-confidence webinar intervention is likely to be acceptable among employees in the workplace. Apart from its accessibility advantage, the webinar offers an interactive environment that may not be possible in other technology-mediated or face-to-face interventions. The webinar was able to maintain some elements of face-to-face intervention while utilizing economical delivery and technological features that were available. This may serve as an alternative and fruitful way of reaching depressed people in the workplace.
WMAWMY was a PhD candidate funded by a scholarship from the Malaysian Ministry of Education under Universiti Teknologi Malaysia without which this research could not be possible. The funder had no involvement in the conduct of the research.
Conflicts of Interest
Multimedia Appendix 1
Screenshots during the webinar sessions.PDF File (Adobe PDF File), 5MB
Multimedia Appendix 2
Example quotes from the feedback corresponding to each category.PDF File (Adobe PDF File), 54KB
- Evans-Lacko S, Knapp M. Importance of social and cultural factors for attitudes, disclosure and time off work for depression: findings from a seven country European study on depression in the workplace. PLoS One 2014;9(3):e91053 [FREE Full text] [CrossRef] [Medline]
- Burton WN, Conti DJ. Depression in the workplace: the role of the corporate medical director. J Occup Environ Med 2008 Apr;50(4):476-481. [CrossRef] [Medline]
- Cornford CS, Hill A, Reilly J. How patients with depressive symptoms view their condition: a qualitative study. Fam Pract 2007 Sep;24(4):358-364. [CrossRef] [Medline]
- Brown JS, Boardman J, Whittinger N, Ashworth M. Can a self-referral system help improve access to psychological treatments? Br J Gen Pract 2010 May;60(574):365-371 [FREE Full text] [CrossRef] [Medline]
- Horrell L, Goldsmith KA, Tylee AT, Schmidt UH, Murphy CL, Bonin E, CLASSIC trial group. One-day cognitive-behavioural therapy self-confidence workshops for people with depression: randomised controlled trial. Br J Psychiatry 2014 Mar;204(3):222-233 [FREE Full text] [CrossRef] [Medline]
- Brown JS, Elliott SA, Boardman J, Andiappan M, Landau S, Howay E. Can the effects of a 1-day CBT psychoeducational workshop on self-confidence be maintained after 2 years? A naturalistic study. Depress Anxiety 2008 Jul;25(7):632-640. [CrossRef] [Medline]
- Brown JS, Boardman J, Elliott SA, Howay E, Morrison J. Are self-referrers just the worried well?--A cross-sectional study of self-referrers to community psycho-educational stress and self-confidence workshops. Soc Psychiatry Psychiatr Epidemiol 2005 May;40(5):396-401. [CrossRef] [Medline]
- Wan Mohd Yunus WMA, Musiat P, Brown J. Systematic review of universal and targeted workplace interventions for depression. Occup Environ Med 2018 Dec;75(1):66-75. [CrossRef] [Medline]
- Emmelkamp PM. Technological innovations in clinical assessment and psychotherapy. Psychother Psychosom 2005;74(6):336-343. [CrossRef] [Medline]
- Dewa CS, Thompson AH, Jacobs P. The association of treatment of depressive episodes and work productivity. Can J Psychiatry 2011 Dec;56(12):743-750. [CrossRef] [Medline]
- Birnbaum HG, Kessler RC, Kelley D, Ben-Hamadi R, Joish VN, Greenberg PE. Employer burden of mild, moderate, and severe major depressive disorder: mental health services utilization and costs, and work performance. Depress Anxiety 2010;27(1):78-89. [CrossRef] [Medline]
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J 2008 Sep 29;337:a1655 [FREE Full text] [CrossRef] [Medline]
- Wan Mohd Yunus WM, Musiat P, Brown J. The formative development of a new self-confidence webinar intervention for depression in the workplace. Cogn Behav Pract 2020 (forthcoming).
- Brown JS, Cochrane R, Hancox T. Large-scale health promotion stress workshops for the general public: a controlled evaluation. Behav Cogn Psychother 2000;28(2):139-151.
- Brown JS, Elliott SA, Boardman J, Ferns J, Morrison J. Meeting the unmet need for depression services with psycho-educational self-confidence workshops: preliminary report. Br J Psychiatry 2004 Dec;185:511-515 [FREE Full text] [CrossRef] [Medline]
- Cheng C. Assessing coping flexibility in real-life and laboratory settings: a multimethod approach. J Pers Soc Psychol 2001;80(5):814-833. [CrossRef]
- Cheng C, Kogan A, Chio JH. The effectiveness of a new, coping flexibility intervention as compared with a cognitive-behavioural intervention in managing work stress. Work Stress 2012 Jul;26(3):272-288. [CrossRef]
- Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One 2016;11(3):e0150205 [FREE Full text] [CrossRef] [Medline]
- Hoffmann T, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J 2014 Mar 07;348:g1687. [CrossRef] [Medline]
- Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, Texas: Psychological Corporation; 1996.
- Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 1988 Dec;56(6):893-897. [CrossRef] [Medline]
- Rosenberg M. Society and the adolescent self-image. Princeton, New Jersey: Princeton University Press; 1965.
- Kato T. Development of the Coping Flexibility Scale: evidence for the coping flexibility hypothesis. J Couns Psychol 2012 Apr;59(2):262-273. [CrossRef] [Medline]
- Kessler RC, Barber C, Beck A, Berglund P, Cleary PD, McKenas D, et al. The World Health Organization Health and Work Performance Questionnaire (HPQ). J Occup Environ Med 2003 Feb;45(2):156-174. [Medline]
- Laugwitz B, Held T, Schrepp M. Construction and Evaluation of a User Experience Questionnaire. : Springer, Berlin, Heidelberg; 2008 Presented at: 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2008; November 20-21, 2008; Graz, Austria. [CrossRef]
- Schrepp M. User Experience Questionnaire (UEQ). 2015 Sep 21. User Experience Questionnaire Handbook URL: https://www.ueq-online.org/Material/Handbook.pdf [accessed 2019-03-27] [WebCite Cache]
- Hedges LV, Olkin I. Statistical Methods For Meta-Analysis. Orlando, Florida: Academic Press; 1985.
- Jacobson N, Follette W, Revenstorf D. Psychotherapy outcome research: methods for reporting variability and evaluating clinical significance. Behav Ther 1984 Sep;15(4):336-352. [CrossRef]
- Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [Medline]
- Christensen L, Mendoza JL. A method of assessing change in a single subject: an alteration of the RC index. Behav Ther 1986 Jun;17(3):305-308. [CrossRef]
- Roelofs J, van Breukelen G, de Graaf LE, Beck AT, Arntz A, Huibers MJ. Norms for the Beck Depression Inventory (BDI-II) in a Large Dutch Community Sample. J Psychopathol Behav Assess 2012 Jul 31;35(1):93-98. [CrossRef]
- Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs 2008 Apr;62(1):107-115. [CrossRef] [Medline]
- Delgadillo J, Kellett S, Ali S, McMillan D, Barkham M, Saxon D, et al. A multi-service practice research network study of large group psychoeducational cognitive behavioural therapy. Behav Res Ther 2016 Dec;87:155-161. [CrossRef] [Medline]
- Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [CrossRef] [Medline]
- Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
- Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided internet-based vs face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
- van BW, Cuijpers P, van SA, Karyotaki E, Andersson G, Smit JH, et al. Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One 2014;9(7):e100674 [FREE Full text] [CrossRef] [Medline]
- Selwyn N, Gorard S, Furlong J, Madden L. Older adults' use of information and communications technology in everyday life. Ageing Soc 2003 Sep 2;23(05):561-582. [CrossRef]
- Charness N, Boot WR. Aging and information technology use. Curr Dir Psychol Sci 2009 Oct;18(5):253-258. [CrossRef]
- Olsen O, Albertsen K, Nielsen ML, Poulsen KB, Gron SM, Brunnberg HL. Workplace restructurings in intervention studies-a challenge for design, analysis and interpretation. BMC Med Res Methodol 2008 Jun 13;8:39 [FREE Full text] [CrossRef] [Medline]
- Huntley AL, Araya R, Salisbury C. Group psychological therapies for depression in the community: systematic review and meta-analysis. Br J Psychiatry 2012 Mar;200(3):184-190 [FREE Full text] [CrossRef] [Medline]
- Whitfield G. Group cognitive–behavioural therapy for anxiety and depression. Adv Psychiatr Treat 2018 Jan 02;16(3):219-227. [CrossRef]
- Yalom ID, Leszcz M. The Theory and Practice of Group Psychotherapy. 5th edition. New York, NY, US: Basic Books; 2005.
- Morrison N. Group cognitive therapy: treatment of choice or sub-optimal option? Behav Cognit Psychother 2002 Mar 5;29(3):311-332. [CrossRef]
- Daft RL, Lengel RH. Organizational information requirements , media richness and structural design. Manag Sci 1986 May;32(5):554-571. [CrossRef]
- Zoumenou V, Sigman-grant M, Coleman G, Malekian F, Zee JM, Fountain BJ, et al. Identifying best practices for an interactive webinar. J Fam Consum Sc 2015;107(2):62-69 [FREE Full text]
- Wang S, Hsu H. The effects of webinar-learning implementation from student-trainers’ perspective. Journal of Interactive Online Learning 2008;7(3):175-194.
- Pezaro SC. Securing the evidence and theory-based design of an online intervention designed to support midwives in work-related psychological distress. J Int Soc Telemed eHealth 2018 Jun 14;6(1):1-12. [CrossRef]
|BDI-II: Beck Depression Inventory II|
|CBT: cognitive behavior therapy|
|RCI: Reliable Change Index|
|RCT: randomized controlled trial|
|SC: stress control|
|UEQ: User Experience Questionnaire|
Edited by G Eysenbach; submitted 10.07.18; peer-reviewed by M Deady, S Pezaro; comments to author 07.10.18; revised version received 13.12.18; accepted 10.02.19; published 26.04.19
©Wan Mohd Azam Wan Mohd Yunus, Peter Musiat, June SL Brown. Originally published in JMIR Mental Health (http://mental.jmir.org), 26.04.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.