This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.
The Measurement Based Care in Mental Health Initiative launched by the Department of Veterans Affairs in 2016 is an example of an evidence-based practice that uses patient-reported outcome measures (PROMs) to improve patient outcomes. The acceptance of measurement-based care (MBC) among Veterans Affairs providers is relatively high. However, there are barriers to MBC for telehealth providers. Health information technologies might afford opportunities to address some of the barriers related to the uptake of MBC.
This paper reports on an implementation effort to integrate MBC into mental health care telehealth practice using eHealth solutions.
Qualitative data were generated from 22 semistructured interviews with psychiatrists (n=4), psychologists (n=3), social workers (n=3), nurses (n=6), a pharmacist (n=1), and administrative staff (n=5) who provide telemental health care through a community-based outpatient clinic in the rural Midwestern United States. The interviews were conducted during the pilot phase of an implementation initiative to increase the adoption of MBC by revising clinic workflows to integrate the use of eHealth technologies. Data were analyzed using thematic analysis.
Time burden and workflow issues were the most common barrier to provider adoption of MBC; sharing and reviewing pencil-and-paper measures and results in the same room was no longer possible in novel telehealth workflows necessitated by the COVID-19 pandemic. Providers voiced concerns about how long it would take to collect, adequately score, interpret, share, and document the PROMs during the telehealth visit. Concerns about time might also correspond to a gap in providers’ familiarity with these assessments, greater comfort in assessing symptoms through clinical interviews, and being accustomed to using the assessments as screening tools more so than longitudinal outcome measures. Capacities associated with eHealth technologies may address workflow concerns and promote providers’ understanding and use of the measures as tracking tools.
The need to use limited appointment time well was a top priority for telemental health providers. eHealth technologies provided operative supports that protect time in appointments by shifting when and how PROMs are collected. Bolstering providers’ familiarity with how to use PROMs in the course of treatment may impact providers’ buy-in by encouraging them to reconsider how sharing and acting on PROMs could be time well spent.
The Measurement Based Care in Mental Health (MBC in MH) Initiative launched by the Department of Veterans Affairs (VA) in 2016 is an example of an evidence-based practice that uses patient-reported outcome measures (PROMs) to improve patient outcomes [
VA providers are agreeable to MBC, but use is low and varies by discipline [
Health information technologies might address some of the barriers related to the uptake of MBC especially in the context of telehealth [
The VA is promoting the use of electronic capture of PROMs via eHealth solutions. Using these technologies, providers can administer PROMs electronically. The results are immediately scored and available for integration into the medical record without requiring synchronous administration or data entry by providers. This paper reports on an implementation effort to integrate RMBC into clinical telehealth practice using eHealth solutions. Our analysis explores the full meaning of providers’ perceptions of time and workflow barriers and makes a novel finding about how the visualizations that the eHealth applications produce offer providers the opportunity to develop a narrative of treatment.
The MBC in MH Initiative in VA launched in 2016; the goal of the initiative was to rapidly implement MBC and position MBC as the new standard of care [
The software platforms that were available at the VA in 2016 (Mental Health Assistant, VA Office of Information and Technology; and Behavioral Health Lab, Capital Solution Design) lacked both patient- and provider-facing features that would have facilitated a robust MBC practice [
The VA has only recently started exploring patient-facing eHealth applications such as BHL Touch (Capital Solution Design) and MH CheckUp (VA Mobile). MH CheckUp was first rolled out in December 2020 and BHL Touch was rolled out in July 2021. Our implementation effort entailed understanding how to use these eHealth applications to facilitate the workflow processes associated with MBC, including collecting, sharing, and acting on PROMs. Human/user-centered design processes structured our implementation according to 4 phases: “Discover,” “Design,” “Build,” and “Test” [
We conducted a rapid ethnographic assessment [
This work supported the operational goals of the Office of Mental Health and Suicide Prevention and the Office of Rural Health within the VA.
Our project procedures were reviewed by the University of Iowa Institutional Review Board (# 202009601) and determined to be non–human subjects research. Additionally, the project was reviewed by the Iowa City VA Research and Development Committee.
We recruited staff members (n=26) who provide telemental health services to patients at 2 CBOCs in the rural Midwestern United States. A total of 22 staff members agreed to participate; they included psychiatrists (n=4), psychologists (n=3), social workers (n=3), nurses (n=6), a pharmacist (n=1), clerical staff (n=2), program managers (n=2), and the regional telehealth point of contact (n=1). Although we only report on interview data with psychiatrists, psychologists, social workers, pharmacists, and the telehealth point of contact in this paper, it was important to speak with clerical staff, nurses, and program managers to understand the organizational climate of the CBOCs. Not only did interviews with these individuals help us ask better questions of the providers, but they will also inform the decisions we make in subsequent phases of the study, including “Design” and “Build.” Moreover, some of these supporting staff members were part of the workflow related to collecting PROMS from patients.
We conducted direct observations of weekly web-based team meetings and semistructured interviews over videoconferencing. The program managers on each of the teams facilitated our entry into the weekly team meetings. We started observing weekly team meetings in December 2020 and continued through 2022. We took detailed field notes of the discussion of workflows that shaped the day-to-day work of the teams. Attending weekly team meetings served four purposes: (1) it acclimated the qualitative analysts (who are both trained in anthropology and not clinicians) to the technical language that the team members used when talking about their work; (2) it helped in the development of rapport, so that it was ultimately easier to recruit participants for interviews, and during those interviews, both the participant and interviewer had a sense of shared experience; (3) it helped us notice potential workflow and organizational barriers to the implementation of the eHealth technologies; and (4) in helping us notice those potential barriers, we were able to tailor our interview probes to the context. We maintained our presence in weekly meetings even after the interviews were completed to track emerging workflow and organizational barriers as we moved into the “Design” phase of our implementation. We used this method as a workaround when our original plan to conduct site visits was derailed due to COVID-19 travel restrictions.
Our early field notes informed the development and refinement of our interview guide (
In all, 2 qualitative analysts conducted a thematic analysis [
The barriers that have been documented for in-person care were also barriers to MBC in the context of telehealth. We heard providers voice concerns about how long it would take and how much coordination it would entail to adequately score, interpret, share results with patients and document the PROMs. We also found that the providers who were already engaging with eHealth applications noticed how these technologies can help resolve some of the time burden and workflow barriers in both telehealth and in-person visits.
In asking providers to reflect on their experiences with MBC, we surfaced latent barriers couched within the larger umbrellas of workflow and time. Concerns about time might correspond to providers’ training and preference in assessing symptoms through clinical interviews rather than standard assessments. These characteristics, combined with being accustomed to using the assessments as screening tools more so than longitudinal outcome measures, might contribute to providers’ hesitancy to use MBC as a tool for shared decision-making and treatment planning.
In our discussions with providers, we made a novel finding about the potential of the visualizations that the eHealth technologies can create. Researchers have noted how tracking patient’s responses over time using visualizations is an unexplored potential benefit of RMBC with eHealth technologies [
At the time of the interviews, providers were still adjusting to changes in their workflow precipitated by the COVID-19 pandemic. The workflows providers had been using to administer PROMs were no longer possible. One psychiatrist explained how, prior to the pandemic,
Usually the patients...would check in in a kiosk and that would flag them to fill out or to receive a paper copy of a questionnaire, And then the patient [would] fill it out on paper and then someone physically on site would upload that into...our electronic medical record, and then we would get it that way. And then the pandemic hit and everyone’s doing video visits to home, which is very exciting. And now trying to catch up and find a way to capture um, that workflow, but all electronically.
When we spoke with providers, they had been using several telehealth formats, including phone visits and video-based visits such as video telemedicine and VA Video Connect (VA Mobile) visits. For video telemedicine visits, patients come into the CBOC and have their visit via video chat with a provider who is either in another room or at another clinic location. For VA Video Connect visits, patients stay at home and have their visit with a provider who is at a clinic or at home. Providers anticipated workflow barriers to integrating PROMs into telehealth appointments, including questions about when to distribute the PROM, how to get it back, and how to get it back in time for it to be used during the appointment. One psychiatrist described their experience in using MBC during in person visits and how “there’s a lot of pieces.” They said,
So when I think about Tele Health, there’s the phone which we’re trying to go away from, and then there’s [VA’s telehealth platform]. So some of my patients like I log in, it’s the start of the appointment we’re both there, fantastic. We’re having an appointment that’s great and I’m trying to get through everything that I need to do to actually take care of a patient. I’m trying to throw in some clinical reminders when I remember. I’m trying to schedule them for their next appointment. And then how I would actually do the measurement based care part? I’m not sure because I’m used to being able to like throw them a measure while I multitask and now I can’t...in a perfect scenario, if they show up, it’s possible that if we had a way for [our teleadministrative staff] to know, like “hey, this person needs XY and Z,” then maybe [the Veteran] could be handed that, but then they’re like trying to do it during the appointment. Do they turn it in after? Where does it go? Uhm, is there something that they could be sent ahead of time that they could do at home and it could be ready in time for the appointment?
The use of electronic applications to conduct assessments prior to scheduled visits addressed many of the workflow issues that this provider described. Some providers were already making use of available eHealth technologies on an individual basis. With these technologies, providers schedule assessments in advance of appointments. Patients receive a link over email or SMS text message, and they complete the assessments; the results are shared and discussed in the upcoming mental health visit. Some applications allow the provider to sync results directly into the electronic medical record, so a patient’s responses to assessments are easily documented, integrated into the visit note, and shared with all the different providers that care for that patient within that institution. A few providers we spoke to had already started using some of the eHealth applications. One of these psychiatrists said,
When I ask Veterans, “Oh how did it go on your end?” the majority answer on their phones...So it seems like text is the best--, has had the most success or the easiest for them to respond. I like that I don’t have to do anything. I can just upload it and it’s right there.
A social worker reported a similar experience:
What I like about it is I can schedule as many...I can have them do more than one...like if I’m seeing somebody weekly and I want to know how they’re doing with whatever one of the assessments I choose to use. I’m able to put it on a weekly basis without having to go back in there and redo it. So that’s what I like about it and then I’m able to see it in [the electronic medical record]...so that morning...before their appointment I can see what the results were.
Providers who had already started using some of the eHealth solutions helped us notice how the functionality of the applications could solve some of the workflow barriers identified by providers who had not yet tried to use the eHealth technology.
Confirming findings already published about the use of MBC, many providers mentioned how time burden was a potential barrier. However, we also noticed that when providers talked about time, they often talked about how they already used the clinical interview to generate the same information the assessments would capture. The relative value of MBC to clinical interviews was not clear. A psychologist described how,
It is time consuming and...it’s either, it doesn’t add as much as I would want it to the clinical interview...and I have to ask those questions anyway, um, PHQ-9 [9-item Patient Health Questionnaire] and GAD [General Anxiety Disorder], I feel like I do them and they don’t always lead to a more in-depth conversation.
Providers felt that it took a long time to do the assessments because they were not sure why they were doing them, or what value the assessments added. For example, one psychiatrist talked about how,
It takes time to do it, and then once you get it, you have to then take more time to understand...what it’s actually saying. Meanwhile, you could just do like a narrative thing or subjective interview where you’re...you talk to them about their sleep and you immediately get to, ‘I'm sleeping too much’ and then you have the conversation about sleeping.
eHealth technologies addressed this concern somewhat by moving some of this work outside of the appointment, as patients were encouraged to complete the PROMs on their own time.
Helping to resolve workflow and time barriers would not address providers’ preferences for understanding a person’s lived experience through clinical interview. On the one hand, providers voiced concern that the score on an assessment is a snapshot in time and often decontextualized from a person’s lived experience. One psychiatrist worried how,
They come in and they’re under a lot of stress because their dog died, and their scores are gonna shoot up...with time it should come, you know, back down, and I do the treatment, it will keep improving. But that’ll take time to track.
The use of the same PROMs as both a screening tool and an MBC tool led to persistent confusion. Rather than using the assessments to shape a meaningful treatment plan and inform clinical decision-making, most providers continue to view standard assessments as screening tools rather than tools for longitudinal assessments of patient-reported treatment outcomes. Moreover, many providers lacked training about the specific content validity and psychometrics of the available measures. Another psychiatrist remembered how,
I think initially I had to like look [many of the scales] up, --not all of them, but lots of them. Like I [didn’t] know what [some were]...like are these useful scales?...So that took a lot of time and then [it] ended up being like most of them [were] not really useful for my purposes...and also I don’t think...the information [for] interpreting the results...you have to look that up separately if needed.
Providers that used the assessments as more than screening tools talked about using the patient’s responses to shape a narrative about treatment. One social worker, adept at cognitive processing therapy for posttraumatic stress disorder, explained how the assessments help her understand how treatment was going. She described how,
I’m doing something like cognitive processing therapy, and I do a consistent uh, PCL [posttraumatic stress disorder symptom rating scale]...I expect those symptoms when we start to increase and spike, and then I expect to see them decrease and so if I don’t see that happening, if I don’t see that spike, then I-I know we’re not really touching on things that are bothering them, that we’re still really surface and if over time I don’t see a decrease, I know that we’re really not scratching what we need to hit to get them to process the information well enough.
eHealth technologies offer providers the ability to visualize data from the assessments to track the trajectory of a patient’s symptoms over time. The graph might be very useful for helping providers build an account of the course of treatment. One social worker reflected on their experience with visualizations provided by eHealth technologies prior to working with the VA; they remembered how,
I had one guy...it was actually pretty amazing...he was super depressed and then...you could see...right along with his mood...his chart totally changed, and he actually got to a point where [he said] “I’d rather actually pay for going to jujitsu classes than therapy. At this point, I’m feeling pretty good.”...I thought that was a cool way to be able to do it.
The visual functionality reinforces the assessments as tracking tools, as the graph potentially facilitates conversations about “trends over time.” Providers no longer have to analyze and create a way to present the long-term data, making it easier to visualize a patient’s progress over time. One psychiatrist who had already started using one of the eHealth technologies reflected how,
I like how quickly it shows the trend...very user-friendly...I think a big value is having the response before the start of the appointment...it’s really nice going into the appointment knowing at least on paper things look better, things look worse, than when I last saw them...So you’re already kind of thinking a little bit about what the next step might be...it kind of helps tailor [my plan] right off the bat a little.
Our findings indicate that not all providers were sure how to use the assessments as a method for developing an understanding of a patient’s lived experience and may have perceived the time spent doing the assessments as time not spent effectively. The above reflection, from a psychiatrist who felt successful using the PROMs to shape their clinical decisions, suggests that eHealth technologies can help providers develop a narrative of treatment that they can use to tailor their treatment plans.
Promoting the uptake of MBC in telehealth requires addressing the issue of time burden, which necessitates both (1) acknowledging the limited time in appointments by facilitating the administration of PROMs via SMS text messaging or email before appointments and (2) satisfying telemental health providers’ need to use time well by increasing their familiarity with how to use the assessments to measure response to treatment. eHealth technologies facilitate the administration of assessments prior to appointments and, thus, pose a pragmatic solution to concerns about time, especially in the context of telehealth. The additional functionality associated with eHealth technologies (eg, graphs that visualize patients’ responses over time) has the potential to increase providers’ awareness of assessments as tracking tools that can facilitate setting goals and following progress toward those goals rather than simply as screening tools.
Our findings confirm and extend findings in the extant literature about how time burden is a formidable barrier to the adoption of MBC [
Although we recruited a diversity of providers from different disciplines and roles within the clinic, our sample is small and only represents 2 CBOCs, both of which were part of the VA health care system. Although our findings would be strengthened by comparison to more diverse clinic settings, our findings reflect previous studies’ findings in different settings; moreover, our qualitative methods allowed us to expand upon and clarify this previous work. Our sample size will grow as our implementation effort continues and we continue to report on our findings.
The adoption of MBC into existing professional practice and the implementation of such programming into a telehealth workflow is a complex process. Promoting the uptake of MBC in telehealth requires addressing the issue of time burden, which necessitates both (1) acknowledging the limited time in appointments by facilitating the administration of PROMs before appointments and (2) satisfying telemental health providers’ need to use time well by increasing their familiarity with how to use the assessments to set treatment goals. eHealth technologies facilitate the administration of assessments prior to appointments and, thus, pose a pragmatic solution to concerns about time. The additional functionality (eg, graphs that visualize patients’ responses over time) has the potential to increase providers’ awareness of assessments as tracking tools that can facilitate setting goals and following progress toward those goals.
Interview guide.
community-based outpatient clinic
measurement-based care
Measurement Based Care in Mental Health
patient-reported outcome measure
remote measurement-based care
Department of Veterans Affairs
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs (VA) or the United States Government.
This project was funded by the VA Quality Enhancement Research Initiative (#QUE 20-007) from the US VA Health Services Research and Development Service and the VA Office of Rural Health.
None declared.