This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Victims of trauma are at high risk for mental health conditions such as posttraumatic stress disorder and depression. Regular assessment of mental health symptoms in the post-trauma period is necessary to identify those at greatest risk and provide treatment. The multiple demands of the acute post-trauma period present numerous barriers to such assessments. Mobile apps are a method by which to overcome these barriers in order to regularly assess symptoms, identify those at risk, and connect patients to needed services.
The current study conducted a usability evaluation of a system to monitor mental health symptoms after a trauma. The system was developed to promote ease of use and facilitate quick transmission of data.
A sample of 21 adults with a history of trauma completed a standardized usability test in a laboratory setting followed by a qualitative interview.
Usability testing indicated that the app was easy to use and that patients were able to answer several questions in less than 1 minute (mean [SD] 29.37 [7.53]; range 15-57). Qualitative analyses suggested that feedback should be included in such an app and recommendations for the type of feedback were offered.
The results of the current study indicate that a mobile app to monitor post-trauma mental health symptoms would be well received by victims. Personalized feedback to the user was identified as critical to promote the usability of the software.
Approximately one in three victims of a traumatic injury will develop a chronic mental health disorder including posttraumatic stress disorder (PTSD) within 1 year after the trauma [
There are several barriers, however, in implementation of such early interventions [
Mobile apps can advance acute post-trauma care and mental health treatment more broadly [
In order for mobile app post-trauma care to have the proposed impact on health care, it is necessary to design systems that address the needs of this patient population. Those recently exposed to a trauma have multiple competing concerns in the acute aftermath of an event that place significant demands on their time [
A total of 21 college-aged adults with a history of a trauma exposure that resulted in a hospital visit participated in the study. Participants were 19 years old (mean [SD] 18.8 [0.87]), and the majority were female (16/21, 76%) and White (15/21, 71%). All participants owned a smartphone, primarily iPhones (15/21, 71%). Participants all texted, took pictures, listened to music, downloaded apps, recorded videos, and accessed the Internet on their phones. A majority used their devices to obtain information about physical health (17/21, 81%) and mental health (14/21, 67%). All participants provided verbal consent as the Institutional Review Board did not require written consent. Consent was recorded using a required documentation form.
A development team with expertise in mobile app development, database creation, acute trauma care, and post-trauma mental health care created a prototype app. Design was guided by the Technology Acceptance Model (TAM) [
A distributed system comprising several major software components was created (
Schematic diagram of the components of the system used to monitor symptoms after a trauma.
Several steps were taken to ensure question sets could be completed quickly. First, two types of question responses were implemented: Sliders and Toggles (
Second, app speed was prioritized. Initial prototypes included multiple icons on the home page that ultimately interfered with speed of use. A home page with a single icon “Begin Questions” in the center was used instead. Transitions between screens were removed. Responses were stored locally on the device and transmitted when the assessment was completed, to eliminate network latency that is common in Web forms [
Screenshot of slider and toggle type questions.
The types of questions allowed were selected to improve the ease of use. Consideration was given to presenting multiple items on a single screen in which users would scroll through all items or presenting multiple screens. The use of a native app, as opposed to a Web survey, reduced load times to overcome the limitation of using multiple screens to complete a survey [
Considerable flexibility was needed to assess a range of symptoms. For example, there are 20 possible symptoms that make up the diagnosis for PTSD. Surveys were allowed to be of unlimited length, have editable content, and have additional items added or removed via the Web interface. Furthermore, participants could be assigned different surveys based on the time of day.
Usability was assessed with the Perceived Useful and Ease of Use Survey (PUEU) [
The mobile app was evaluated on an iOS (iPod Touch 5th Gen) and an Android device (Motorola Moto G). The interface was nearly identical across both platforms. Half of the sample used each device. Use of the mobile app was monitored using a universal serial bus (USB) camera mounted to the device.
Standardized tasks took place within a laboratory. To standardize the use of the app, participants were read a script describing a motor vehicle accident that required immediate and sustained medical attention. They were told that this app was being given to them to monitor their recovery after they left the hospital and they were asked to complete a set of self-report assessments in the coming weeks. Participants used the app a total of 5 times, each time progressing further in their recovery. Trained research assistants observed the participants during their interaction with the app and interactions were recorded with a usability mounted camera [
A clinical psychologist analyzed the qualitative data. A constructivist grounded theory approach was used in which comments and interviews were reviewed multiple times, coded, and primary themes were extracted. Themes that were present for more than three cases were retained. Themes that were present in three or fewer cases were reviewed, merged with other themes, or discarded. Coding and thematic analyses were conducted after each wave to determine the point at which saturation had been obtained and when no new bugs were identified. A hierarchical structure in which themes were evaluated as representing perceived usefulness or perceived ease of use was then evaluated to determine the extent that the qualitative data corresponded to the quantitative data. Several passes of the data determined that this structure represented the data well. Matrix analyses combined the quantitative data from the PUEU and the qualitative data from the interview. Triangulation of the mixed-method yielded a high degree of overlap across the quantitative and qualitative data, which adds validity to the conclusions drawn from the qualitative analysis.
Participants used the app a total of five times. Participants completed a standard question set that contained 7 items (6 slider-type and 1 toggle-type). Questions assessed symptoms of PTSD (re-experiencing, avoidance, hyperarousal, numbing), pain, and social support, and the presence of trauma-related cues. The time to complete the question sets was mean (SD) 29.37 (7.53) seconds (range 15-57). A review of the video recordings of participant interaction revealed minimal user interaction errors. Participants were able to navigate each use of the app without error. The app stalled for approximately 30 seconds for 2 participants after all responses were logged. No other usability issues were observed.
Results of the perceived usefulness and perceived ease of use survey.
|
1 (Unlikely) | 2 | 3 | 4 | 5 | 6 | 7 (Likely) | |
|
||||||||
|
The app would enable me to communicate with my doctor more quickly. | 0.0 | 4.8 | 0.0 | 0.0 | 28.6 | 38.1 | 28.6 |
|
The app would improve my recovery from a traumatic event. | 0.0 | 14.3 | 9.5 | 23.8 | 28.6 | 19.0 | 4.8 |
|
The app would improve the quality of medical care I received after a traumatic event. | 0.0 | 4.8 | 28.6 | 4.8 | 28.6 | 14.3 | 19.0 |
|
The app would make it easier for me to remember to follow the doctor’s instructions after a traumatic event. | 0.0 | 0.0 | 4.8 | 23.8 | 28.6 | 23.8 | 19.0 |
|
The app would make it easier for me to seek additional medical care after a traumatic event. | 0.0 | 4.8 | 4.8 | 9.5 | 33.3 | 23.8 | 23.8 |
|
I would find this app useful after a traumatic event. | 0.0 | 4.8 | 9.5 | 14.3 | 28.6 | 19.0 | 23.8 |
|
||||||||
|
Learning to use the app would be easy for me. | 0.0 | 0.0 | 0.0 | 9.5 | 4.8 | 23.8 | 61.9 |
|
I would find it easy to get the app to do what I want it to do. | 4.8 | 0.0 | 0.0 | 19.0 | 19.0 | 14.3 | 42.9 |
|
My interaction with the app would be clear and understandable. | 0.0 | 0.0 | 0.0 | 28.6 | 14.3 | 4.8 | 52.4 |
|
I would find the app to be flexible to interact with. | 0.0 | 4.8 | 0.0 | 14.3 | 23.8 | 33.3 | 23.8 |
|
It would be easy for me to become skillful at using the system. | 0.0 | 0.0 | 0.0 | 4.8 | 23.8 | 19.0 | 52.4 |
|
I would find the system easy to use. | 4.8 | 0.0 | 0.0 | 14.3 | 9.5 | 9.5 | 61.9 |
Ratings of overall perceived usefulness according to the PUEU suggested that participants thought a mobile monitoring system would be useful in improving post-trauma recovery (mean [SD] 5.14 [1.10]). Participants reported the app would facilitate communication with their provider (mean [SD] 5.81 [1.17]). A substantial portion (11/21, 52%) reported that this app would improve communication above and beyond traditional follow-up methods in the qualitative interview. Participants requested two-way communication with their provider through the app (12/21, 57%). That is, they wanted a provider to give feedback, but the type varied. A portion wanted personalized feedback (6/21, 29%), whereas others preferred a notification that the doctor received or viewed their responses (12/21, 57%). Several (3/21, 14%) recommended the app list contact information for a provider. Last, several participants (5/21, 24%) reported that reminders for intervention (eg, take medication, complete physical therapy) would be helpful.
Participants rated the app as moderately likely to improve their recovery from the traumatic event (mean [SD] 4.42 [1.43]) and thought it would be useful after a trauma (mean [SD] 5.19 [1.47]). A majority thought a monitoring system would be helpful (13/21, 62%), with a portion stating it would indicate their provider cared about their recovery (4/21, 19%). However, several participants voiced concerns that this app would replace face-to-face provider contact (2/21, 10%).
Overall ratings suggested the app was easy to use (mean [SD] 5.92 [1.05]; mean calculated out of 7), easy to learn to use (mean [SD] 6.38 [0.97]), and it would be easy to become skillful with the app (mean [SD] 6.19 [0.98]). Qualitative responses were supportive of these data. Nearly all (19/21, 90%) reported the app was easy to use and they enjoyed the simplicity of the design. Several found the design calming and engaging (5/21, 24%). A substantial majority reported assessments took minimal time to complete and would impose minimal burden (17/21, 81%). Indeed, participants reported they would be willing to answer mean (SD) 2.86 (1.85) question sets per day, mean (SD) 4.90 (2.41) days per week.
Participants had several recommendations to enhance the design and features of the app. Half (11/21, 52%) suggested that personalizing the app would be helpful. Specific recommendations included changing colors, setting backgrounds, and personalizing the question content. Personalized content involved using specific details about the individual (eg, name) and questions about their trauma (eg, “how is the pain in your left leg?”). Second, it was recommended that each question have a free text response option to clarify ratings (7/21, 33%).
Participants reported they wanted the app to provide feedback, including a graph of their responses (10/21, 48%). Participants wanted to receive positive feedback that informed them of areas where they were improving and did not wish to be notified if symptoms were worsening (8/21, 38%). Rather, they preferred that worsening outcomes be reported to their provider and the provider contact them.
The current study obtained important information about user preferences for a monitoring system for mental health symptoms following a trauma. Participants preferred an app that was easy to use, would not impose a significant burden, and was customizable. The findings are consistent with the TAM [
Participants were moderately positive that a mobile monitoring app would help their recovery. This is consistent with evidence suggesting that monitoring is helpful in reducing symptoms of PTSD [
A key theme was the importance of providing feedback. Participants were unanimous in their request to interact with their provider through the app rather than as a one-way communication tool. Most users wanted immediate feedback from their provider after completing an assessment. An immediate response, however, would be challenging given the burden this would impose on a provider [
A related theme was the personalization of the app to the needs of the patient. Personalized feedback is highly relevant to outcomes and sustained use [
Participants provided two areas of caution. First, participants wanted automated feedback to be positive and preferred that negative outcomes trigger a provider touchpoint. Within the context of TAM, it is possible that negative feedback may diminish the perceived usefulness of an app. Those who are not recovering likely do not want feedback reinforcing their lack of progress but rather want intervention. Alternatively, providing positive feedback about their progress may be perceived as encouraging and supportive. Second, participants cautioned that such monitoring systems should not replace interpersonal care. It is unclear if this concern could be addressed by providing a more personalized experience, such as telehealth or telephone sessions, or additional contact from their provider. Wound care after surgery using telehealth reduced the need for in-person follow-ups, which was preferred by patients [
Our conclusions should be considered within the context of several limitations. The sample size for the current study was within the recommended size for usability studies [
The results of this study provide several points of feedback to advance modern methods for monitoring mental health recovery after a trauma. The need for personalized feedback, the type of feedback provided, and how patients view such a system has broad implications for other conditions. These recommendations should guide the refinement of current systems and the development of new strategies that leverage novel technology. Although technology changes rapidly, the principles obtained from this study and related projects are applicable to systems that address mental health. Such work is essential to the development of systems that will be used by patients to improve outcomes.
posttraumatic stress disorder
Perceived Useful and Ease of Use Survey
Technology Acceptance Model
The authors would like to thank Joshua Barry, Rebecca Norton, Jenna Bushor, and Tosha Hawley for their assistance in developing the app.
None declared.