This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
There has been a growing trend in the delivery of mental health treatment via technology (ie, electronic health, eHealth). However, engagement with eHealth interventions is a concern, and theoretically based research in this area is sparse. Factors that influence engagement are poorly understood, especially in trauma survivors with symptoms of posttraumatic stress.
The aim of this study was to examine engagement with a trauma recovery eHealth intervention using the Health Action Process Approach theoretical model. Outcome expectancy, perceived need, pretreatment self-efficacy, and trauma symptoms influence the formation of intentions (motivational phase), followed by planning, which mediates the translation of intentions into engagement (volitional phase). We hypothesized the mediational effect of planning would be moderated by level of treatment self-efficacy.
Trauma survivors from around the United States used the eHealth intervention for 2 weeks. We collected baseline demographic, social cognitive predictors, and distress symptoms and measured engagement subjectively and objectively throughout the intervention.
The motivational phase model explained 48% of the variance, and outcome expectations (beta=.36), perceived need (beta=.32), pretreatment self-efficacy (beta=.13), and trauma symptoms (beta=.21) were significant predictors of intention (N=440). In the volitional phase, results of the moderated mediation model indicated for low levels of treatment self-efficacy, planning mediated the effects of intention on levels of engagement (B=0.89, 95% CI 0.143-2.605; N=115).
Though many factors can affect engagement, these results offer a theoretical framework for understanding engagement with an eHealth intervention. This study highlighted the importance of perceived need, outcome expectations, self-efficacy, and baseline distress symptoms in the formation of intentions to use the intervention. For those low in treatment self-efficacy, planning may play an important role in the translation of intentions into engagement. Results of this study may help bring some clarification to the question of what makes eHealth interventions work.
There has been a growing trend in the delivery of mental health treatment over the internet [
Exposure to potentially traumatic events in adult US populations is widespread [
Ample research has shown eHealth interventions are effective in decreasing distress symptoms in trauma survivors [
Our study aimed to examine the utility of using a single theoretical model, the Health Action Process Approach (HAPA) [
The term engagement has been used in a variety of ways, making it challenging to synthesize consistent models and measures. Generally, engagement is described as efforts by a user to start and continue with an intervention and encapsulates objective and subjective experiences [
Predictors of engagement with eHealth interventions more generally and trauma programs more specifically have not been studied in a systematic, theoretically based way [
Other researchers have focused on effects of the technical design aspects on engagement [
Researchers from areas beyond trauma (eg, health and illness issues) have applied theoretical frameworks to explain eHealth engagement. The Technology and Acceptance Model has been used to explore intentions to engage with information and communication technologies among health care providers [
The HAPA [
The HAPA motivational phase is typically characterized by awareness of risk, outcome expectancies, and perceived task self-efficacy (ie, pretreatment self-efficacy). For our eHealth intervention, positive outcome expectancies may refer to the ability to cope with posttraumatic distress. Pretreatment self-efficacy reflects beliefs about the ability to initiate eHealth engagement [
Besides self-efficacy and outcome expectations, the role of other motivational variables such as perceived need and posttraumatic symptoms may be considered in the motivational phase of the HAPA. Perceived need [
After intention has been formed in the motivational stage of the HAPA, an individual enters the volitional stage where self-regulation skills such as planning and treatment self-efficacy prompt behavior enactment [
The primary outcome of this study was engagement with a trauma recovery eHealth intervention. A major challenge in the study of engagement is the lack of a shared definition and conceptualization of user engagement [
Longitudinal revised Health Action Process Approach (HAPA) research model. In the motivational phase, pretreatment self-efficacy, outcome expectations, perceived need, and trauma symptoms are predicted to have a significant positive effect on the formation of intentions. In the volitional phase, intentions are translated into engagement, mediated by planning and moderated by levels of treatment self-efficacy. Engagement is a latent construct consisting of both subjective (estimated frequency and duration) and objective measures. Objective engagement is continuously measured by the electronic health (eHealth) intervention.
High quality, objectively measurable information on engagement can be acquired from page logs, time on site, and other indicators of treatment exposure. These objectively measurable metrics were included in our study. Our study also included self-report measures of engagement to capture subjective perceptions of usage.
Using the HAPA as a model guiding the relationships between the study variables (
To increase external validity, participants were recruited from the Trauma, Health, and Hazards Center trauma registry (15/440, 3.4%), national domestic violence advocate and rape crisis center registries (64/440, 14.5%), the national development and research institute list servers (56/440, 12.7%), social media (19/440, 4.3%), and the University of Colorado at Colorado Springs (UCCS) student population (286/440, 65.0%) from May 2015 to October 2016. All participants included in the study had directly experienced one or more traumatic events as measured by the Life Events Checklist [
All participants who met criteria at T1 (N=440) reported that they were directly exposed to one or more traumatic events either through experiencing or witnessing the event, including physical assault (248/440, 56.4%), transportation accidents (300/440, 68.3%), unwanted sexual contact (219/440, 49.9%), sexual assault (158/440, 35.9%), life-threatening illness or injury (225/440, 51.1%), fire or explosion (167/440, 38%), natural disasters (175/440, 39.7%), sudden violent death of someone close (109/440, 24.9%), serious accidents (150/440, 34.2%), severe human suffering (136/440, 30.9%), toxic exposure (68/440, 15.5%), military combat (37/440, 8.5%), and other traumatic events (258/440, 58.6%). Participants experienced the same traumatic event with varying frequency, ranging from 56.4% (248/440) who experienced the event once, to 9.1% (40/440) who experienced the event at least 14 times.
The UCCS Institutional Review Board approved the study. UCCS psychology students signed up for this study via the Sona online system, and nonstudents were contacted via email or a flyer. All participants were provided a brief statement explaining the procedure and purpose of the study along with a link to the T1 survey on Qualtrics.
One week after qualifying for the study, participants were sent an email asking them to complete the short T2 survey. One week after finishing the T2 survey, participants were prompted by email to take the T3 survey on Qualtrics. After finishing the final survey, participants were debriefed, and UCCS psychology students received additional extra credit. Nonpsychology students were entered into a raffle for one of four US $25.00 gift cards. Local and national mental health resources were provided to all participants after the study.
The following measures incorporated variables in the motivation and volitional phases of the HAPA model (see
In addition to the HAPA variables, participant website satisfaction was also measured. All measures except trauma symptoms were developed for this study, as there were no measures available to assess these constructs. Their psychometric properties are shown in
Pretreatment self-efficacy was measured by eight questions that began with the sentence stem “I am confident that I can start using an eHealth intervention in the next two weeks...” The sentence stem was followed by items representing technological and coping related barriers such as “even if I am uncomfortable using the internet” or “even if I am having difficulty handling all the things I have to do.” Participants responded on a 5-point scale ranging from not at all confident to very confident.
Descriptive statistics for demographics for time 1 (baseline), time 2 (one week after baseline), and time 3 (two weeks after baseline). Some percentages do not add up to 100% because of missing data.
Measure | Time 1 (N=440) | Time 2 (N=161) | Time 3 (N=115) | |
Mean age in years (SD) | 25.57 (11.02) | 28.11 (13.31) | 28.49 (12.98) | |
Age range in years | 18-80 | 18-80 | 18-78 | |
Female | 337 (76.6) | 128 (79.5) | 94 (81.7) | |
Male | 101 (23.0) | 33 (20.5) | 21 (18.3) | |
Other | 2 (0.5) | 0 (0.0) | 0 (0.0) | |
Singlea | 192 (43.6) | 38 (39.2) | 43 (37.4) | |
Committedb | 213 (48.4) | 50 (51.5) | 60 (52.2) | |
Other | 35 (8.0) | 9 (9.3) | 12 (10.4) | |
High school | 280 (63.3) | 43 (44.3) | 60 (52.2) | |
Associates degree | 95 (21.6) | 24 (24.7) | 30 (26.1) | |
Bachelor’s degree | 45 (10.2) | 20 (20.6) | 18 (15.7) | |
Graduate degree | 18 (4.1) | 10 (10.3) | 07 (6.1) | |
None | 117 (26.6) | 22 (22.7) | 24 (20.9) | |
Part-time | 196 (44.5) | 34 (35.1) | 52 (45.2) | |
Full-time | 116 (26.4) | 33 (34.0) | 34 (29.6) | |
Retired | 10 (2.3) | 8 (8.2) | 05 (4.3) | |
$0-$25,000 | 186 (42.3) | 42 (43.3) | 46 (40.0) | |
$25,001-$70,000 | 143 (32.5) | 34 (35.1) | 42 (36.5) | |
$70,001-$100,000 | 58 (13.2) | 10 (10.3) | 13 (11.3) | |
>$100,000 | 50 (11.4) | 10 (10.3) | 12 (10.4) | |
Treatment (current) | 71 (16.1) | 17 (17.5) | 22 (19.1) | |
Treatment (past year) | 32 (7.3) | 9 (9.3) | 7 (6.1) | |
Treatment (lifetime) | 183 (41.6) | 51 (52.6) | 57 (49.6) | |
1 time | 248 (56.4) | 80 (49.7) | 53 (46.1) | |
2-13 times | 129 (29.3) | 51 (31.7) | 36 (31.3) | |
>14 times | 40 (9.1) | 20 (12.4) | 17 (14.8) |
aIncludes widowed or divorced.
bIncludes married couples and couples in a committed relationship.
Participant flowchart.
Number of items, scoring range, and Cronbach alpha for time 1 (N=440), time 2 (N=161), and time 3 (N=115) measures. PCL-5: Posttraumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition.
Scale | Number of items | Scoring range | Cronbach alpha T1 | Cronbach alpha T2 | Cronbach alpha T3 |
1. PCL-5 | 20 | 0-80 | .95 | —a | .95 |
2. Outcome expectations | 10 | 10-50 | .85 | .83 | .85 |
3. Pretreatment self- efficacy | 8 | 8-40 | .95 | — | — |
4. Perceived need | 6 | 6-30 | .92 | — | — |
5. Intention | 5 | 5-25 | .88 | — | — |
6. Treatment self-efficacy | 8 | 8-40 | — | .96 | .94 |
7. Planning | 4 | 4-20 | — | .80 | .79 |
8. Engagement (subjective) | 10 | — | — | — | .86 |
aNot measured at respective time point.
Both positive (pros) and negative (cons) outcome expectancies were assessed with 10 questions that started with the sentence stem “If I use the eHealth intervention on a regular basis I expect that...” followed by the items measuring possible pros and cons. Example pros and cons include “it will help me to relax more” or “it will not make any difference in how I feel.” Cons were reverse scored, and the total score was computed by adding the answers to all items.
Perceived need was measured with six responses to the following statement: “Please indicate your perception of how much you believe you need an intervention for the following issues.” Issues pertain to dealing with the trauma such as “to feel normal again?” and “to be able to manage distressing dreams or images about the traumatic experience.” Participants responded on a 5-point scale ranging from strongly disagree to strongly agree.
The PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5; PCL-5) was used to measure the distress symptoms associated with trauma. The PCL-5 is a 20-item self-report measure that assesses the 20 DSM-5 symptoms of PTSD [
Behavioral intentions are the perceived likelihood to act in a certain way, and for this study, they comprised a person’s motivation toward using an eHealth intervention. Intention to perform a behavior should be measured the same way as assessing the behavior itself [
Treatment self-efficacy was measured by eight questions that began with the sentence stem “I am confident I could continue to use an eHealth intervention over the next two weeks...” followed by items measuring treatment self-efficacy related technology and trauma coping self-efficacy. Example items include “even if I do not like it initially” and “even if it brings up difficult memories.” Participants responded on a 5-point scale ranging from not at all confident to very confident.
Individuals were asked if they had made a plan or schedule for using the eHealth intervention. For those who planned, details of their plan were measured by four questions that began with the sentence stem “My plan included...” followed by questions regarding when, where, what, and how often they would use the intervention. Example questions include “how often I would use the eHealth intervention” and “what modules of the eHealth intervention I would use.” Responses ranged from strongly disagree to strongly agree.
Engagement was measured both subjectively and objectively. Subjective measures included questions regarding frequency and duration assessed at T3. Frequency was assessed with five questions that began with the sentence stem “How often did you use the following eHealth intervention modules...” followed by a list of five modules (unhelpful ways of coping, relaxation, social support, self-talk, trauma triggers, and memories). Answers were rated on a 6-point scale ranging from 1 (“never”) to 6 (“more than once a day”). Duration was measured by the total estimated usage (in minutes) of the five modules. Objective measures consisted of automatically recorded data that quantified the frequency (number of pages visited) and duration (total number of minutes logged in) of intervention usage [
My Trauma Recovery (MTR) is a self-guided, theoretically based, interactive internet application with no interaction with a therapist. MTR is based mainly on social cognitive theory [
The intervention focuses on increasing an individual’s ability to cope with trauma via six self-directed modules: (1) unhelpful ways of coping, (2) relaxation, (3) social support, (4) self-talk, (5) trauma triggers and memories, and (6) seeking professional help. The first five modules were included in the subjective engagement measure. A self-test provides users the opportunity to gain feedback on their current emotional distress and provides graphs that depict their assessment results. Throughout the six modules, the site utilizes mastery experiences, vicarious success modeling, verbal persuasion, and tools to monitor and regulate internal distress to increase coping self-efficacy through interactive, tailored experiences. Individuals can use any of modules as often as needed and can assess their progress over time. On the basis of their assessments, suggestions are made to maximize trauma coping self-efficacy gains. Completing all the MTR modules requires approximately 2 hours. However, users generally do not finish all the modules in a single sitting. Additionally, the interactive components of the website offer opportunity to revisit the different modules over time (eg, triggers management or relaxation). Therefore, we allowed participants access to the intervention for 2 weeks to provide ample opportunity to explore all components of MTR.
MTR has received initial support for its effectiveness in reducing symptoms in two separate randomized clinical trials [
Due to the small sample size at T3 (N=115) compared with T1 (N=440), separate analyses were run for the motivational (T1) and volitional (T2, T3) phases. The motivational hypothesis was analyzed with the T1 sample (N=440), and the volitional hypothesis was analyzed with completers only (N=115; see
The motivational phase hypothesis (
A moderated mediation analysis was performed using Mplus version 7.4 (Muthén & Muthén, Los Angeles, CA, USA) (
The bootstrapping method was used to test inferences about the significance of mediation effects when treatment self-efficacy was high (1 SD above the mean) and low (1 SD below the mean), with 5000 bootstrap samples. Bootstrap CIs not including zero indicate a significant indirect effect. The bootstrap approach is considered superior to normal theory-based Sobel test for the significance of the mediation [
The descriptive statistics for the demographic variables are shown in
The correlations between the volitional phase predictors (T1, T2, and T3) revealed that intention was significantly positively correlated with planning, treatment self-efficacy, and subjective engagement. Notably, treatment self-efficacy exhibited significant positive medium-sized correlations with all the motivational predictors and most of the volitional phase predictors. Interestingly, only the subjective measures of engagement showed significant positive correlations with intention.
Importantly, paired samples
Correlations, means, and SDs of Health Action Process Approach (HAPA) variables for time 1 (N=440), time 2 (N=161), and time 3 (N=115). Correlations in the upper diagonal region for time 1 show values for all participants who met criteria at time 1 (N=440). Correlations in the lower diagonal region for time 1 show values of participants who created an account (N=115). Time 1 was assessed at baseline, time 2 was assessed one week after baseline, and time 3 was two weeks after baseline. PCL-5: Posttraumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition
HAPA variables | Time 1 | Time 2 | Time 3 | ||||||||
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | |
1. PCL-5 | 1.00 | .07 | .10a | .52c | .41c | .27b | .25b | .34b | .33b | .16 | .12 |
2. Outcome expectations | .11 | 1.00 | .34c | .37c | .54c | .12 | .36c | .15 | .15 | −.10 | .01 |
3. Pretreatment self-efficacy | .06 | .19a | 1.00 | .17c | .32c | .31c | .38c | .01 | .09 | .15 | .12 |
4. Perceived need | .55b | .44b | −.05 | 1.00 | .58c | .33c | .33c | .41c | .21 | .14 | .13 |
5. Intention | .54c | .56c | .02 | .59c | 1.00 | .31c | .39c | .38c | .23a | .02 | .11 |
6. Planning | 1.00 | .40c | .41c | .15 | .35b | .20a | |||||
7. Treatment self-efficacy | 1.00 | .39c | .27b | .16 | .10 | ||||||
8. Subjective engagement frequency | 1.00 | .37c | .33b | .19 | |||||||
9. Subjective engagement minutes | 1.00 | .05 | .15 | ||||||||
10. Objective engagement pages | 1.00 | .55b | |||||||||
11. Objective engagement minutes | 1.00 | ||||||||||
Mean (SD) | 24.83 (19.05) | 32.67 (5.75) | 28.54 (8.56) | 17.67 (6.42) | 17.51 (4.78) | 32.54 (3.50) | 24.53 (8.39) | 7.85 (4.01) | 75.61 (91.60) | 81.77 (70.59) | 54.16 (83.79) |
a
b
e
To test the motivational phase, an SEM was analyzed using T1 participants (N=440). Missing data for all items were 0.05% for T1, 1.24% for T2, and 1.39% for T3. Missing data were imputed with maximum likelihood procedure using AMOS v.24. Additionally, we performed a Little’s missing completely at random (MCAR) test, a stricter criterion than missing at random. A Little’s MCAR test with sex and employment as reference variables showed missing data were MCAR for pretreatment self-efficacy items, χ27=6.6,
The original independence SEM yielded a poor fit with χ26(N=440)=245.1,
A moderated mediation analysis was performed to test the volitional phase with completers (N=115). We handled missing data using the full information maximum likelihood method. The assumption of full information maximum likelihood estimation is that missing data must be at least missing at random to have reliable outcomes. A Little’s MCAR test with sex and employment status as reference points showed that missing data were MCAR, χ213=5.6,
Next, to examine whether the variables that were significantly different between the completers of T3 and dropouts affected the results, these variables were included in the model. The baseline PTSD was included in the motivational phase; thus, we did not include it in this analysis. Age and trauma frequency were entered in the model as covariates for engagement. Results were consistent with the results without the covariates. The conditional indirect effect was significant at low levels of treatment self-efficacy (−1 SD; B=0.90; 95% CI, 0.124-2.330). The conditional indirect effect was not significant at high levels of treatment self-efficacy (+1 SD; B=0.47; 95% CI, −0.043 to 1.751).
The aim of this study was to examine the associations between motivational and volitional predictors of engagement with an eHealth intervention for trauma recovery. Previous engagement research focused primarily on a-theoretical approaches such as user characteristics and interventions features. These approaches are heavily tied to individual attributes or unique aspects of the eHealth intervention, and few offer general theoretical frameworks for understanding the process of engagement. Thus, no clear model exists to explain what factors influence engagement in eHealth interventions. To the best of our knowledge, this is the first study to focus on the psychological process of eHealth intervention engagement using the theoretical frameworks of social cognitive theory and the HAPA.
The motivational component of the HAPA model indicates individual intention to utilize an eHealth intervention for trauma is significantly related to outcome expectations, pretreatment self-efficacy, perceived need, and PTSD symptom severity. As hypothesized, higher baseline levels of pretreatment self-efficacy (beta=.13) and outcome expectancy (beta=.36) predict greater intention to engage. These results support the findings of previous studies that used the HAPA model for predicting engagement with other health behaviors such as physical activity [
In line with previous mental health research [
Higher baseline PTSD symptoms also predicted greater intention (beta=.21). Research suggests that PTSD symptomatology is one of the determinants of the intention to seek help [
Our results extend the HAPA motivational model suggesting that symptom severity (eg, PTSD) may be an important factor in understanding intention to utilize an eHealth program. These findings indicate that models such as HAPA may need to consider symptoms of physical or mental illness in understanding motivational factors related to intention. Though higher symptoms predicted greater intention to use the intervention, previous studies found higher baseline symptoms associated with lower usage [
The volitional section of the HAPA model suggests that the role of intention on engagement was differential relative to the individual perceptions of treatment self-efficacy. High intentions are associated with higher levels of planning, yet this relationship is relative to the level of perceived treatment self-efficacy. Planning promoted greater engagement only for individuals with low treatment self-efficacy (B=0.89). Consistent with Schwarzer [
Improving engagement with therapy, whether in-person or online, can potentially lead to improved therapeutic outcomes. By understanding the impact of phase-specific self-efficacy, perceived need, and outcome expectations, interventions can be designed to enhance these perceptions, which in turn could lead to improved engagement. Specifically, these results suggest that communicating the expected outcomes of an intervention could have a significant impact on initial engagement. For those who have low perceived need and high PTSD symptoms, motivational enhancement to increase perceived need before treatment may lead to improved engagement [
Collectively our findings provide a new way to approach our understanding of engagement with eHealth interventions for trauma and eHealth more generally. Future studies should examine additional mediators and moderators to engagement. For example, recent HAPA related research has found perceived social support and self-regulation (ie, motivation and willpower) to also mediate the intention-behavioral gap for physical activity uptake [
Although the motivational phase has a high amount of explained variance (48%), it might be because of the cross-sectional analysis. Future studies should examine this phase longitudinally. Whereas our 2-week study examined the pretreatment and treatment phases of the behavioral change process, the HAPA model typically is also applied to the maintenance and recovery phases of health behavior change. These phases may not apply to engaging in an eHealth intervention to manage psychological distress. However, it might be interesting to conceptualize the processes that would bring a person back to an eHealth intervention following an upsurge in symptoms. Individual perceptions of optimism or self-efficacy in managing these challenges, including returning to an eHealth program, are important to consider. This is akin to optimistic beliefs about one’s ability to deal with barriers that arise while maintaining the behavioral change and recovery self-efficacy associated with one’s conviction to get back on track after being derailed [
It should be noted that our research design did not allow us to compare the interaction of the HAPA model factors with important aspects of different eHealth trauma interventions (ie, active comparison group). Future studies will need to focus on deconstructing critical intervention components for different eHealth approaches in relation to the HAPA factors (eg, outcome expectations and perceived treatment self-efficacy).
Another limitation of this study was the high dropout attrition rate over the course of the three measurements. Of the 440 who met criteria for the study, only 115 completed the T3 survey. This attrition rate of almost 73.9% was not unusual for eHealth interventions [
Finally, and perhaps most importantly, the conceptualization of engagement needs further examination. Our measures of engagement focused on frequency and duration. Additional measures of engagement (sometimes referred to as measures of adherence) can include number of log-ins, number of exercises completed, and the number of days used [
In conclusion, this study found support for the utilization of the HAPA model for understanding predictors of engagement with an eHealth intervention for trauma recovery. Results of this study help to clarify what makes eHealth interventions work [
comparative fit index
Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition
health action process approach
missing completely at random
My Trauma Recovery
Posttraumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition
posttraumatic stress disorder
root mean square error of approximation
structural equation model
time
Tucker-Lewis index
University of Colorado at Colorado Springs
The authors would like to acknowledge Robert Durham, PhD, and Thomas Pyszczynski, PhD, who provided expertise in the design and implementation of the research. Dr. Aleksandra Luszczynska’s contribution was supported by grant no. BST/2017/A/07 from the Ministry of Science and Higher Education, Poland.
None declared.