This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
The COVID-19 pandemic has escalated the global need for remotely delivered and scalable interventions after psychological trauma. A brief intervention involving a computer game as an imagery-competing task has shown promising results for reducing the number of intrusive memories of trauma—one of the core clinical symptoms of posttraumatic stress disorder. To date, the intervention has only been delivered face-to-face. To be tested and implemented on a wider scale, digital adaptation for remote delivery is crucial. An important first step is to develop digitalized intervention materials in a systematic way based on feedback from clinicians, researchers, and students in preparation for pilot testing with target users.
The first aim of this study is to obtain and analyze qualitative feedback on digital intervention materials, namely two animated videos and two quizzes that explain the target clinical symptoms and provide intervention instructions. The second aim is to refine the digitalized materials based on this feedback.
We conducted semistructured interviews with 12 participants who had delivered or had knowledge of the intervention when delivered face-to-face. We obtained in-depth feedback on the perceived feasibility of using the digitalized materials and suggestions for improvements. Interviews were assessed using qualitative content analysis, and suggested improvements were evaluated for implementation using a systematic method of prioritization.
A total of three overarching themes were identified from the data. First, participants were highly positive about the potential benefits of using these digital materials for remote delivery, reporting that the videos effectively conveyed key concepts of the symptom and its treatment. Second, some modifications to the materials were suggested for improving clarity. On the basis of this feedback, we made nine specific changes. Finally, participants raised some key challenges for remote delivery, mainly in overcoming the lack of real-time communication during the intervention.
Clinicians, researchers, and clinical psychology students were overall confident in the use of digitalized materials to remotely deliver a brief intervention to reduce intrusive memories of trauma. Guided by participant feedback, we identified and implemented changes to refine the intervention materials. This study lays the groundwork for the next step: pilot testing remote delivery of the full intervention to trauma survivors.
Most people will experience a psychologically traumatic event in their lifetime, that is, experience or witness actual or threatened serious injury, sexual violence, or death (eg, traffic accidents, assaults, war, or natural disasters) [
Recent clinical guidelines for PTSD highlight the utility of targeting single symptoms [
Building on laboratory findings using analog trauma (ie, stressful film clips) [
Clinical studies have shown preliminary evidence that the intervention may lead to a reduced number of intrusive memories, reported in a daily diary over the course of 1 week, when compared with participants who did not use the intervention (treatment as usual or control task) [
Remote delivery of digital interventions comes with several challenges, such as generally high attrition [
We followed guiding principles for the development of the intervention materials, in line with PBA [
We purposively recruited [
Semistructured interviews were carried out by the first author (KD), who had recently joined the research team and was not involved in the creation of the videos. We created an interview guide (
Videos can increase the accessibility and scalability of mental health treatments, which otherwise face barriers to access [
The first video (“What Are Intrusive Memories?”) defines and describes intrusive memories of trauma, the target symptom of the intervention (3 minutes 17 seconds; screenshots in
Screenshots from the “What are Intrusive Memories?” video, which explains the target symptom of the intervention in an accessible way.
Screenshots from the “How to Play Tetris” video, which instructs participants on how to complete the imagery-competing task in the specific way required (eg, by emphasizing "mental rotation").
A total of two quizzes, each containing four questions relating to one of the videos, were created using Survey Monkey. The quizzes were designed to check the target user’s understanding of the video content (ie, what intrusive memories are and instructions for Tetris gameplay), which would typically be covered in a face-to-face session by the researcher asking the participant to summarize. The quizzes also aim to boost target user understanding and later recall of the content [
Interviews were transcribed verbatim and analyzed using qualitative content analysis [
All the participants (N=12) were female. The mean age of the sample was 31.3 years (SD 7.5). Participants described their ethnicity as Swedish (n=3), Swedish/European (n=1), Italian (n=1), Mixed/White and Asian (n=1), White/European (n=1), Icelandic/Caucasian (n=1), Irish/Caucasian/White (n=1), White/British (n=1), White/Caucasian Australian (n=1), and Caucasian (n=1). The sample comprised 6 researcher clinicians, 3 clinical psychology students, 1 researcher, 1 clinician, and 1 intern. Across the sample, participants worked with a broad range of groups with a history of trauma (
Participant occupation and relevant experience with the intervention.
ID | Occupation | Experience with the intervention |
P1 | Researcher clinician | Has delivered the intervention to women who have a history of sexual trauma |
P2 | Researcher clinician | Involved in the conceptual and clinical development of the intervention |
P3 | Clinical psychology student | Has delivered the intervention to refugees in Sweden and the United Kingdom |
P4 | Intern | Has observed the intervention being delivered and worked alongside those who have translated the intervention protocol |
P5 | Clinical psychology student | Is preparing to use the intervention in a laboratory setting in the near future |
P6 | Researcher | Has been part of clinical training meetings for the intervention and meetings concerning its development |
P7 | Researcher clinician | Has delivered the intervention to frontline hospital staff in the emergency department and intensive clinical care |
P8 | Researcher clinician | Has delivered the intervention to refugees in the United Kingdom |
P9 | Researcher clinician | Has delivered the intervention to parents of children who have been recently admitted or discharged from the pediatric intensive care unit |
P10 | Researcher clinician | Has helped create and design the intervention and has delivered it to emergency department patients |
P11 | Clinical psychology student | Is preparing to use the intervention in a laboratory setting in the near future |
P12 | Clinician | Has delivered the intervention to patients in the emergency department |
The qualitative content analysis generated three main themes, each with a number of subthemes.
Although participants were not probed directly to describe potential benefits of the digital materials, all expressed positive comments about their utility in promoting remote delivery of the intervention.
Participants reported that intrusive memories were clearly explained and that the illustrations should help target users to understand the symptoms:
I really liked when, [it was showing] it can be like a camera picture, and it had the picture, and the camera sounds, [or] it can be like a movie, and it had a movie reel... I think that really clarified that difference... that was really effective.
Instructions for the imagery-competing task were said to be clearly conveyed, and the animations may help target users understand the instructions even more so than face-to-face delivery:
I think it’s really visually good that you have like, the little brain on the side, and ...you can show the blocks inside the brain to really like, show that you have to think about it, and it becomes very clear how mental rotation works because you have that little brain on the side.
One participant noted that the videos would be useful because target users can go back and rewatch them at any time to review the information. This participant works with parents of children in the pediatric intensive care unit, a target population that is often under acute stress and/or sleep deprivation and may benefit from the opportunity to revisit instructions:
In fact, you could probably be more helpful over video because parents can pause and stop and rewind.
General positive feedback was given on the style of the animation for depicting what can often be a difficult topic:
It’s simple, but respectful at the same time, if that makes sense, and without seeming like a cartoon, but doing that in a thoughtful but very accessible way. So, I think that balance has been struck beautifully.
The videos were also praised for their steady pace and frequent pauses to allow target users to process the content. Feedback suggested that the new materials hold initial promise regarding target user
The depiction of intrusive memories was said to convey empathy for target users, perhaps even more than a face-to-face explanation typically would:
It explains the concept. But I think it also shows participants that you understand what “I'm” going through. And I think that's really, really powerful.
One participant commented on the facial expression of the character, stating that, from her experience with patients in the emergency department, the character showed an appropriate amount of distress to depict the experience of an intrusive memory, without being over the top:
...she looked sad or like she was experiencing stress when she got the intrusive memories. But she didn't look like, that scared.
Participants reported that the materials could be used across studies for different groups of target users. For example, if used for research purposes, a standardized approach to intervention delivery would increase experimental control and allow more direct comparison of results across studies:
I think that videos would be great, not only for remote, but also for a more like, improved generalisation of the intervention in both an experimental and clinical setting.
It was not required for participants to answer the quiz questions, only to review them. All participants who did submit answers (9 of 12) scored 100%. Participants expressed that the quiz questions overall captured the important concepts from the video and should be useful to check target user comprehension:
[They are] good because it checks out the learning they’ve had from like, watching the video, making sure they understand what they’ve watched... They’ve covered the main points of what you’re aiming to get out of it.
In addition to the aforementioned positive feedback, participants suggested some changes to help refine the digital materials before being implemented for pilot testing with target users.
Some participants suggested drawing a clearer distinction between intrusive memories and related concepts such as rumination, by including more specific examples (eg, stating that they are not the same as thinking, “I wish I could have done something differently”).
One participant said that the video did not clearly cover flashbacks, that is, a dissociative experience where the individual feels or acts as if the trauma event is happening again [
I don't really think that the video captured flashbacks... if it's explaining intrusive memories, it doesn't have to be explaining flashbacks. But if I were to show this to a participant, I think I would pause and then want to add it can also be a flashback.
Two participants suggested that a particular aspect of the animation was unclear: the three droplets of blood that depict an example of an intrusive memory. They suggested that it be replaced by a red puddle instead of droplets; therefore, it is more apparent that the character has a memory involving blood:
Why [are] there some drops, red drops?... I [didn’t] realise what it was.
Some small modifications were also suggested to clarify the wording of two quiz questions, such as when multiple answer options could be conceived as technically correct, as opposed to only one, as intended.
Some participants expressed that some aspects of Tetris gameplay were not addressed in the video, such as what to do when
...it would be helpful to inform participants that the Tetris will speed up. And so, I know that kind of was asked by participants a couple of times as they were going through the intervention. They were saying, you know, “it's getting faster!”
It was also suggested that the video should have more emphasis on
During the interviews, participants also raised some key challenges that should be considered in the shift toward full digitalization and remote delivery of the intervention.
Main challenges raised for remote delivery were how to allow target users to ask questions and how to give real-time feedback or encouragement. This was said to be especially important for target users who have severe dissociation symptoms (ie, the intrusive memory leading to the person losing touch with the here and now). One participant described that from her experience of working with refugees with complex trauma history, guidance when completing the intervention can help keep these users stay
The one kind of drawback that I can think of is that, if the participant is wanting to ask a question, or wanting to pause and get that kind of straightaway feedback to answer questions, that might not be possible... The researcher might remind the participant to keep doing the mental rotation, especially when it starts moving quite quickly. I find that people then say, “Oh, I just totally lost track of what I was doing... I was just trying to focus on not filling up the screen.”
Some participants raised the question of how quizzes will be implemented, specifically how to proceed if target users answer any of the questions incorrectly. It was suggested that immediate feedback could be provided within the platform or an answer key provided.
Participants were asked in each interview if they had recommendations for additional videos that could help advance remote delivery. All participants had at least one suggestion for videos to explain other aspects of the procedure that are normally described face-to-face. This could include how to fill in the intrusive memory diary to assess the target symptom or how to instruct participants to create a list of their intrusive memories for the memory reminder part of the intervention. It was suggested that videos could help instruct how to access the game and troubleshoot common technical problems but that such instructions might differ across studies, for example, when playing Tetris across different platforms. Several participants raised the need to translate the materials for future planned studies that will include participants from other countries, such as women in Iceland, health care staff in Sweden, and refugees across Europe.
Some participants raised the importance of fostering a sense of a bidirectional relationship between the intervention provider and target user, as this may help with retention:
I think, when we digitalise something we need, as we are losing, you know, the interpersonal component. ... we need to make sure we kind of fill that gap. Relationship still is the key for every treatment.
A total of 9 suggested changes were considered to be
Suggested changes to the digital intervention materials that were categorized by the research team as
Suggested change during interview | Changes made to the digital intervention materials |
Add that intrusive memories come without warning or are |
Changed the summary sentence at the end of the video from “pop up in your mind without warning” to “pop suddenly into your mind when you don’t want them to” to emphasize that they are involuntary |
Clarify the distinction between rumination and intrusive memories, by adding examples of the former, such as thinking “what could I have done differently?” or “why did this happen to me?” | Added to the script for the video on intrusive memories, “they are also NOT the same as thinking in words, like ‘Something awful happened to me’” |
Clarify the three red drops of blood image, perhaps by drawing a red puddle instead, as it was difficult to distinguish what this was | Animation in the video was altered to represent blood more clearly |
Emphasize mental rotation more as it is thought to be critical to the intervention’s efficacy | Changed the script and rerecorded to say, “for this intervention to work, the most important thing for you to do is focus on the blocks that are coming up next” |
Annunciate the word |
Both videos have both been revoiced by a different person, with emphasis on clear annunciation |
Say in the video that this is playing Tetris with special instructions | Change the name of the video when it is presented to target users to “Tetris with Special Instructions” |
For the “How to Play Tetris” quiz, the first question, two of the possible answers are technically correct | Changed the first alternative for the first question from “Score as many points as possible” to “Getting as many orange blocks as possible” so that latter is more clearly incorrect |
Some phrasing from the video is not included in the “How to Play Tetris” quiz, like saying “planning in your mind’s eye” | Changed the first option of the last question from “Visualise and rotate them in your head” to “Visualise and rotate them in your mind’s eye” |
The wording of one question in the “What Are Intrusive Memories?” quiz, where the one option says, “intrusive memories usually take the form of |
Changed the third option of the second question stating that intrusive memories usually take the form of “a really high fever” to be more clearly incorrect |
In line with our first aim, we obtained and analyzed qualitative feedback on digital intervention materials, which showed that participants were generally positive about the potential benefits of the digital materials for remotely delivering the intervention. The animated videos were thought to effectively explain the target symptom and to instruct participants in the imagery-competing task. Participants noted that the digital materials were engaging and could likely be used across different demographics of target users, highlighting their potential to standardize delivery of the intervention across studies. Relatively minor suggestions were made to clarify some aspects of the materials and, in line with our second aim, we systematically implemented these changes, making 9 modifications to the videos and quizzes. Participants also raised some key challenges in moving forward with successful remote delivery of the intervention, such as overcoming the lack of real-time communication between the researcher and target user. We discuss how these considerations will shape the next steps of intervention development.
Participants expressed several potential benefits of digital materials that align with previous findings. For example, participants reported that the videos effectively conveyed the content, and previous research has shown that animations with visual cues can be an effective medium for communicating scientific concepts, sometimes leading to better retention than static images [
A number of specific suggestions for refining the materials were also raised, some to clarify the explanation of intrusive memories. Psychoeducation is a staple of most posttrauma interventions, and encouraging participants’ understanding of symptoms is a core aspect of psychoeducation [
Finally, participants raised key challenges to address in advancing the remote delivery of the intervention. These challenges generally align with those encountered in previous research on digital interventions. Participants raised the question of how to address target user questions if they arise during the session and the need to give real-time feedback. In digital interventions, for individuals with chronic physical health conditions, for example, remote interventions have been found to both empower and motivate participants; however, when feedback is given too often, patients may feel overly reliant on their health care providers [
These findings will help guide the next steps of intervention development. One of the next steps, suggested by participants, will be to create additional study-specific videos. For instance, videos could help to guide participants through the intervention, from baseline assessment to accessing the Tetris game to keeping track of the target symptom. Such instructions will often differ across studies, depending on the target group and the platform used for intervention delivery. These study-specific videos may therefore be more practical to make as simple film clips of the researcher giving instructions, rather than highly polished animations that would only be used for a single study. Such clips of the researcher may also help users feel that the material is more personalized, as videos with real people could foster a more personal connection with the study. On the basis of participants’ feedback, the next steps should also include implementing real-time feedback to the quiz questions via the platform to provide encouragement or correct misunderstandings.
Critically, the next step is then to obtain feedback from different populations of target users, who do not have previous experience with the intervention, on the feasibility and acceptability of the digital materials. Given that many digital interventions are hampered by low rates of
Regarding quizzes, it will be important to examine (and then optimize) the difficulty level for target users. Target users in different groups should be able to understand the content and answer all quiz questions correctly. Ideally, this brief comprehension check should be in line with face-to-face procedures; quizzes should be a helpful
Many previous works have demonstrated the value of obtaining target user feedback to improve intervention materials, which is core to PBA [
This study has several strengths. First, systematic adjustments were made to the materials by first obtaining feedback from nontarget users of the intervention [
Digital interventions hold exciting promise for the dissemination of evidence-based treatments globally; however, they must be developed in a careful, stepwise manner to be engaging and effective for target users [
Guiding principles for development of the digitalized intervention materials.
Interview guide.
Qualitative content analysis codes.
Suggested changes and MoSCoW categorization. MoSCoW: Must Have, Should Have, Could Have, Would Like.
Must Have, Should Have, Could Have, Would Like
person-based approach
patient and public involvement
posttraumatic stress disorder
The author would like to thank the clinicians, researchers, and students for their input on the digitalized materials. In addition, they thank Katy Metcalf for her animation work. This project was supported by grants to EH from the Oak Foundation (OCAY-18-442), the Lupina Foundation, the Swedish Research Council (2017-00957 and 2020-00873), AFA Insurance (200342) and a Medical Psychology grant to MK from Karolinska Institutet and Karolinska University Hospital.
EH, MK, and BG conceived and designed the study. BG and KD drafted the manuscript, with critical feedback and revisions from MK and EH. KD conducted the interviews and data coding as part of her master’s thesis. All authors approved the final version of the manuscript.
EH reports serving on the board of trustees of the charity MQ: Transforming Mental Health but receives no remuneration for this role. EH receives royalties from books and occasional fees for workshops and invited addresses and receives occasional consultancy fees from the Swedish agency for health technology assessment and assessment of social services. The other authors have no conflicts of interest to declare.