This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.
People with intellectual and developmental disabilities are at increased health-related risk due to the COVID-19 pandemic. Virtual training programs that support providers in caring for the physical and mental health needs of this population, as well provide psychological support to the providers themselves, are needed during the pandemic.
This paper describes the design, delivery, and evaluation of a virtual educational COVID-19–focused Extension for Community Healthcare Outcomes program to support providers during the COVID-19 pandemic in caring for the mental health of people with intellectual and developmental disabilities.
A rapid design thinking approach was used to develop a 6-session program that incorporates mindfulness practice, a wellness check, COVID-19–related research and policy updates, a didactic presentation on a combination mental health and COVID-19 related topic, and a case-based discussion to encourage practical learning. We used the first 5 outcome levels of Moore’s evaluation framework—focusing on participation, satisfaction, learning, self-efficacy, and change in practice—which were rated (out of 5) by care providers from health and disability service sectors, as well as additional reflection measures about innovations to the program. Qualitative feedback from open-text responses from participants were analyzed using modified manifest content analysis.
A total of 104 care providers from health and disability service sectors participated in the program. High levels of engagement (81 participants per session on average) and satisfaction (overall satisfaction score: mean 4.31, SD 0.17) were observed. Self-efficacy (score improvement: 19.8%), support, and coping improved. Participants also rated the newly developed COVID-19 program and its innovative components highly. Open text feedback showed participants felt that the Extension for Community Healthcare Outcomes program expanded their knowledge and competency and created a sense of being part of a community of practice; provided value for the COVID-19 innovations; supported resource-sharing within and beyond program participants; and facilitated changes to participants’ approaches to client care in practice and increased participants’ confidence in supporting clients and families.
The Extension for Community Healthcare Outcomes program is an effective model for capacity-building programs with a shared-learning approach. Future iterations should include targeted evaluation of long-term outcomes such as staff burnout.
Countries across the world have responded to the COVID-19 pandemic with rapid deployment of public health measures and hospital-based care for the acutely unwell. Various population groups, including people with intellectual and developmental disabilities, are marginalized and underserved in the health care system, both during and prior to COVID-19 [
Experiences from previous pandemics suggest the need to support health care workers by increasing their mental health awareness and encouraging self-care [
Virtual education programs can be used to overcome barriers to healthcare and social service provider training in a pandemic situation arising from physical distancing, quarantine, and other isolation measures [
Globally, organizations have adapted the ECHO model for their respective COVID-19–specific needs. A recent study [
COVID-19 was declared a pandemic on March 11, 2020. As a result of the pandemic, social service providers and health care providers who support people with intellectual and developmental disabilities faced new and unprecedented challenges in community settings. In response, we leveraged the ECHO model to develop ECHO Ontario Adult Intellectual and Developmental Disabilities: Mental Health in the Time of COVID-19 (ECHO AIDD-COVID), a targeted virtual education program to support care providers, working together, from the disability and health sectors. The purpose of the program was to share best practices in caring for the mental health of the intellectual and developmental disabilities population during the COVID-19 pandemic and to reduce feelings of isolation and burnout by making new connections and sharing resources. In this paper, we aim to describe the rapid development and evaluation of the ECHO AIDD-COVID program. We hypothesized that this program would improve participants’ self-efficacy in supporting and managing the mental health issues of people with intellectual and developmental disabilities during the COVID-19 pandemic.
The rampant increase in COVID-19 infections in the winter of 2020, and the subsequent need to enhance and strengthen the skills of care workers, led to the rapid planning and development of a COVID-19–focused ECHO program. This program was an adaptation of an existing 12-session ECHO program launched prior to the COVID-19 pandemic, ECHO Ontario Adult Intellectual and Developmental Disabilities (ECHO Ontario AIDD), which focused on caring for the mental health of people with intellectual and developmental disabilities [
A rapid design thinking approach [
The inspiration stage—the problem or opportunity at hand—comprised the challenges created for providers caring for people with intellectual and developmental disabilities by the COVID-19 pandemic, such as the negative impact on mental health, for both clients and providers alike, and the need for rapid capacity building and connection in the community.
This stage involved brainstorming and refining ideas and solutions. We met several times with project leads, hub members, and members of the ECHO program team to explore ways to leverage existing operational structures and the collective expertise of the multidisciplinary ECHO Ontario AIDD team members.
In this stage, potential solutions were developed and shared with target users, who provided feedback. A prototype—a description used in rapid design framework to develop best possible solutions for the identified problems [
Participation in the ECHO AIDD-COVID program was open to all care providers, both social service providers and health care providers, working with people with intellectual and developmental disabilities in Ontario, Canada. We recruited potential participants by emailing invitation flyers to all previous participants of ECHO programs at the Centre for Addiction and Mental Health, as well to developmental service agencies, community mental health organizations, professional accrediting colleges, and primary care sites in Ontario. Providers who were interested in participating completed a web-based application form. All applicants independently providing care were accepted in to the program.
We assembled a hub team that comprised a psychiatrist, psychologist, primary care physician, behavior therapist, occupational therapist, nurse, social worker, and family advisor (the parent of an adult with intellectual and developmental disabilities). The strengths of the team included expertise in primary care, mental health, and intellectual and developmental disabilities; experience working directly with people with intellectual and developmental disabilities the during COVID-19; and experience managing psychological distress.
The curriculum was developed by triangulating sources, including feedback from a prior needs’ assessment [
Weekly 1.5-hour-long sessions were conducted from April 17, 2020 to May 22, 2020 over 6 weeks, which is half the duration of ECHO Ontario AIDD. Each session included introductions, a mindfulness exercise led by the family advisor, a wellness check, COVID-19–related research and policy updates, a didactic presentation based on the curriculum topic for the day, and a case-based discussion, in which a participant (care provider) presented an anonymized case from their practice, for which they required support, to illustrate the complexities in caring for people with intellectual and developmental disabilities in conjunction with the impact of COVID-19. Additionally, a web-based ECHO AIDD-COVID resource portal, with reference materials related to the ECHO program, was available to the participants for use during and after the sessions. Sessions had a dual focus— participant skill development, for addressing the mental health issues faced by people with intellectual and developmental disabilities, and support for the psychological well-being of participants. The innovations described earlier were integrated in to the ECHO sessions seamlessly. Evidence-based resources were shared with participants and could be accessed after the course.
Our evaluation strategy was informed by the Evaluation framework for continuing professional development, specifically, levels 1 to 5 (participation, satisfaction, learning, self-efficacy and change in practice) [
Basic participant demographic information (profession, practice setting, and attendance) was collected throughout the duration of the program.
Participant satisfaction was measured weekly using web-based postsession satisfaction questionnaires. Statements were rated on a 5-point Likert scale, from 1 (strongly disagree) to 5 (strongly agree), and focused on expanded knowledge and skills, reduced professional isolation, addressed learning needs, recommend session to others, and overall satisfaction. We obtained qualitative feedback with open-text responses to questions that asked for suggestions (for curriculum topics) and overall comments or feedback.
Perceived self-efficacy was assessed for 4 core program competencies, with respect to providing care for people with intellectual and developmental disabilities during the COVID-19 pandemic, before and after program participation, with a previously established 100-point confidence scale (a higher number indicated higher confidence) [
Participants responded, using a binary scale (1, yes; 0, no), to a question that asked whether participation in the program resulted in a change in their practice; participants were also prompted to provide examples with open-text feedback.
Feedback in this area was collected before and after the program with 2 items—having professional support and being equipped to cope with stressors (ie, fear of contagion; rapid spread of virus; risk to self, client, family, or friends, etc) related to the pandemic—using a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).
Reflection questions about innovations to the program (combining social service providers and health care providers, COVID-19 strategy sharing, including of mindfulness practice, including support from the community of practice, and sharing COVID-19 updates) were asked after the program, with responses captured using a 5-point Likert scale. Participants were also asked to comment on how participation in the ECHO AIDD-COVID program impacted the challenges that they experienced during the COVID-19 pandemic with open-text responses.
Evaluation measures and data sets generated and analyzed in this report were reviewed and deemed to be part of program evaluation at the Centre for Addiction and Mental Health.
Quantitative data were analyzed using either Excel (Microsoft Inc) or SPSS software (version 21; IBM Corp). Proportions, frequencies, and percentages were calculated for categorical variables (profession, practice setting, change in practice, and attendance). Means and standard deviations were calculated for continuous variables (satisfaction scores, self-efficacy scores, scores for reflections about innovations to the program, and scores for experiences with COVID-19 pandemic). Pre- and postprogram data about experiences during the COVID-19 pandemic and self-efficacy were analyzed using either paired
Preliminary modified manifest content analysis was conducted using NVivo software (version 12, QSR International) to evaluate open-text responses about program participation and the impact to challenges experienced during COVID-19. A project team member uploaded all open-text responses into NVivo, and then reviewed and performed open coding on all text. The project team met on a regular basis to review and discuss coding to develop and refine a coding matrix with definitions. This coding matrix was applied to all text; references for each code were reviewed, and frequencies were calculated. Finally, all codes were summarized and organized for interpretation [
A total of 104 care providers, with a variety of professional backgrounds, from 56 organizations (
Weekly satisfaction scores were high, ranging from a mean 4.07 (SD 0.18) to a mean 4.32 (SD 0.14); the overall mean satisfaction score was 4.31 (SD 0.17;
Participant demographics.
Demographic group | Value, n (%) | ||
|
104 (100) | ||
|
Access coordinator or service navigator | 2 (1.9) | |
|
Administrator | 5 (4.8) | |
|
Behavior analyst | 6 (5.8) | |
|
Case worker or manager | 23 (22.1) | |
|
Developmental services professional | 16 (15.4) | |
|
Physician | 8 (7.7) | |
|
Nursing professional (registered nurse, registered practical nurse, nurse practitioner) | 8 (7.7) | |
|
Occupational therapist | 2 (1.9) | |
|
Other (pharmacist, speech language pathologist) | 4 (3.8) | |
|
Psychologist or psychotherapist | 2 (1.9) | |
|
Social worker | 19 (18.3) | |
|
Support worker | 9 (8.7) | |
|
56 (100) | ||
|
Academic hospital | 3 (5) | |
|
Community health center | 3 (5) | |
|
Community mental health agency | 6 (11) | |
|
Community mental health and addictions agency | 1 (2) | |
|
Community support services agency | 10 (18) | |
|
Developmental services community agency | 28 (50) | |
|
Family health group | 1 (2) | |
|
Family health team | 1 (2) | |
|
Other | 1 (2) | |
|
Private practice or solo practitioner | 2 (4) |
Ratings for satisfaction survey items.
Item | Rating out of 5 (n=228)a, mean (SD) |
The session content expanded my existing skills and knowledge. | 4.07 (0.18) |
This session has addressed my learning needs. | 4.15 (0.07) |
This session has reduced my professional isolation. | 4.21 (0.18) |
I would recommend this session to others. | 4.32 (0.14 ) |
Overall, I was satisfied with the session. | 4.31 (0.17) |
aTotal number of completed weekly surveys received.
In total, 42 participants completed both pre- and postprogram self-efficacy and experiences with the COVID-19 pandemic questionnaires. Mean self-efficacy scores prior to participation in ECHO were 61.3 (SD 18.2), and after the program, self-efficacy scores were 81.1 (SD 9.8); there was a statistically significant improvement (t41= –9.035,
Change in change in pre- and postprogram confidence and experience with COVID-19 participation in the ECHO AIDD-COVID program.
Items | Score (n=42), mean (SD) | |||||
|
Pre | Post | Difference |
|
||
|
|
|
|
|
||
|
Communicate effectively and prepare for person and family-centered care for adults with intellectual and developmental disabilities during the COVID-19 pandemic | 65.38 (20.22) | 81.83 (9.79) | 16.45 (17.72) | <.001a | |
|
Support and manage the mental health of individuals with or suspected of having intellectual and developmental disabilities during the COVID-19 pandemic | 56.47 (20.06) | 77.88 (13.54) | 21.40 (16.39) | <.001b | |
|
Manage burnout and build resilience in myself, other health care and developmental service professionals, and caregivers during the COVID-19 pandemic | 57.57 (22.34) | 78.52 (12.08) | 20.95 (21.44) | <.001c | |
|
Work effectively in/with interprofessional and intraprofessional teams across health and social systems during the COVID-19 pandemic to support the care of clients with intellectual and developmental disabilities | 69.33 (18.04) | 86.07 (10.09) | 16.74 (15.72) | <.001d | |
|
|
|
|
|
||
|
I feel I have enough professional support and resources for myself to continue caring for my clients during this time | 3.45 (0.89) | 4.10 (0.62) | 0.64. (0.82) | <.001e | |
|
I feel equipped to cope with stressors (ie fear of contagion, rapid spread of virus, risk to self/client/family/friends, etc) related to the COVID-19 pandemic | 3.17 (0.93) | 4.10 (0.66) | 0.93 (0.87) | <.001f |
aThe Wilcoxon signed rank test was used (
bA paired
cA paired
dA paired
eA paired
fA paired
The analysis of 53 open-text responses about the impact of ECHO participation on challenges experienced by participants during the COVID-19 pandemic is summarized in
Key areas that emerged from open-text responses.
Key areas | Participants (n=53), n (%)a | ||
|
29 (55) | ||
|
Benefits of case-based learning | 9 | |
|
Improvements to knowledge and awareness | 13 | |
|
Increased learning through interprofessional education | 5 | |
|
25 (47) | ||
|
Supporting and learning from one another | 9 | |
|
Validation from others | 2 | |
|
21 (40) | ||
|
Benefits of mindfulness | 4 | |
|
Increases in COVID-19 knowledge | 12 | |
|
Value of family perspective | 5 | |
|
18 (34) | ||
|
Sharing resources with broader teams and organizations | 3 | |
|
10 (19) | ||
|
Application of knowledge in client care | 7 | |
Increased confidence in supporting clients and families | 3 (6) |
aPercentages do not add to 100%, and only n values are provided for subthemes.
Participants were also asked about participation in ECHO having an impact on practice. Participants reported participation in ECHO AIDD-COVID resulted in a change in their practice and an equal number were in favor of this program being run again (46/53, 87%). Almost all of the participants reported their learning needs were met in the program (51/53, 96%). The reflection questionnaire around prototype innovations in the program was completed by 53 participants. An overwhelming 98% of participants (52/53) agreed ECHO AIDD-COVID made them feel supported and part of a virtual community of practice. One participant commented that it was
so nice to know that we are not alone in this strange time and share the same challenges
and that they would
miss this weekly touch point with professionals.
A similar percentage of participants (52/53, 98%) reported COVID-19 updates and resources as valuable; a participant shared,
there were innovative strategies suggested in each session as well as content in the presentations that I believe helped improve my approach day to day with the clients I have been supporting.
Furthermore, 94% of participants (50/53) agreed or strongly agreed that having both interprofessional health care providers and social service providers enhanced their learning. Most participants (51/53, 96%) also reported that having a family member in the hub enhanced their learning. The impact of the family perspective is best illustrated by a participant who shared that
...the most powerful experiences I had was whenever the family member spoke. I think we all can hypothetically understand caregivers’ perspectives, but we cannot understand the full emotional toll or the personal thoughts and worries that caregivers have.
A similar percentage (51/53, 96%) of participants appreciated the opportunity to share strategies in the community. The weekly mindfulness exercise led by the family advisor was reported to be helpful by 77% of the participants (41/53) and functioned to
remind us to take care of ourselves.
We described the successful development and evaluation of a COVID-19–focused ECHO program for workers caring for people with intellectual and developmental disabilities. A rapid design thinking approach was used to develop the ECHO AIDD-COVID program. Evaluation findings showed high levels of engagement and satisfaction with the program, with the majority of participants reporting changing their practice because of the program. To the best of our knowledge, this paper is the first to document use of the ECHO model and its significant improvement in perceived self-efficacy in caring for people with intellectual and developmental disabilities during the COVID-19 pandemic for a cohort that encompassed both social service providers and health care providers. Improved confidence in all the core program competencies shows the ECHO model is an effective way to improve provider skills for supporting the mental health needs of people with intellectual and developmental disabilities. Additionally, there was a cascading effect from knowledge shared by participants beyond the program—use of the program can be an effective share-and-spread strategy during the pandemic. Participant feedback from open-text responses shows that the program helped expand knowledge and facilitated changes in practice. The ECHO model was conceptually designed to develop a community of practice. This was validated by the qualitative feedback from participants. The results of the pre- and postprogram evaluations suggest that participants felt that group participation helped to support their own well-being, especially their ability to cope with COVID-19 stressors.
Because of the pandemic, uncertainty and unprecedented challenges arose in social support and health care sectors, and there was a need for the swift deployment of a capacity-building program to support the needs of people with intellectual and developmental disabilities and those of the health and social service providers who work with this underserved population. The rapid design thinking framework was instrumental in incorporating lessons learned from our previous ECHO capacity-building project [
In contrast to other ECHOs designed to strictly focus on working with a population [
There are some limitations to consider when reviewing our findings. First, the data and measures used in this evaluation are from a single cycle of an ECHO program that was specifically focused on mental health in intellectual and developmental disabilities populations in Ontario during COVID-19; as such, our findings may not be generalizable to other cycles or settings. Future evaluation and research efforts will seek to replicate these findings with other health conditions and settings. Additionally, findings for satisfaction, changes in confidence, and experience with COVID-19 participation were informed by data collected from individuals who completed satisfaction surveys and questionnaires. This may introduce a response bias, whereby those who participated in these data collection activities may have been more engaged and likely to respond with higher scores; however, we recognize this challenge is not unique to our evaluation and exists for anyone collecting data via surveys.
Future iterations should incorporate targeted outcome measures to evaluate the role of ECHO in addressing the mental health needs of care workers who are supporting people with intellectual and developmental disabilities in the community [
Although this study addressed self-efficacy and the competency of learners, focus on retention and its medium- to long-term effects would be helpful for designing future programs. In addition, the evaluation of implementation outcomes would be helpful to understand the impact of this educational intervention. Ethical implications and dynamics involved in coproducing educational content are important future considerations [
Extension for Community Healthcare Outcomes
Extension for Community Healthcare Outcomes Ontario Adult Intellectual and Developmental Disabilities: Mental Health in the Time of COVID-19
ECHO Ontario Adult Intellectual and Developmental Disabilities
We would like to thank the ECHO AIDD-COVID team, who contributed to the design and launch of this initiative: Renisha Iruthayanathan, BSc; Lee Steel, family advisor; Liz Grier, MD, CCFP; Kendra Thomson, PhD, BCBA-D; Angela Gonzales, RN, MN; and Nadia Mia, MSW, RSW.
None declared.