Background: Barriers to recruiting and retaining people with psychosis and their families in research are well-established, potentially biasing clinical research samples. Digital research tools, such as online platforms, mobile apps, and text messaging, have the potential to address barriers to research by facilitating remote participation. However, there has been limited research on leveraging these technologies to engage people with psychosis and their families in research.
Objective: The objective of this study was to assess the uptake of digital tools to engage patients with provisional psychosis and their families in research and their preferences for different research administration methods.
Methods: This study used Research Electronic Data Capture (REDCap)—a secure web-based platform with built-in tools for data collection and storage—to send web-based consent forms and surveys on service engagement via text message or email to patients and families referred to early psychosis intervention services; potential participants were also approached or reminded about the study in person. We calculated completion rates and timing using remote and in-person methods and compensation preferences.
Results: A total of 447 patients with provisional psychosis and 187 of their family members agreed to receive the web-based consent form, and approximately half of the patients (216/447, 48.3%) and family members (109/187, 58.3%) consented to participate in the survey. Most patients (182/229, 79.5%) and family members (75/116, 64.7%) who completed the consent form did so remotely, with more family members (41/116, 35.3%) than patients (47/229, 20.5%) completing it in person. Of those who consented, 77.3% (167/216) of patients and 72.5% (79/109) of family members completed the survey, and most did the survey remotely. Almost all patients (418/462, 90.5%) and family members (174/190, 91.6%) requested to receive the consent form and survey by email, and only 4.1% (19/462) and 3.2% (6/190), respectively, preferred text message. Just over half of the patients (91/167, 54.5%) and family members (42/79, 53.2%) preferred to receive electronic gift cards from a coffee shop as study compensation. Most surveys were completed on weekdays between 12 PM and 6 PM.
Conclusions: When offered the choice, most participants with psychosis and their families chose remote administration methods, suggesting that digital tools may enhance research recruitment and participation in this population, particularly in the context of the COVID-19 global pandemic.
While research is critically important to advancing the assessment and treatment of psychosis, conducting research with people with psychosis and their families can be challenging. Barriers to recruiting and retaining participants in psychosis research include study burden (eg, frequent and long study visits), illness severity, poverty, reluctance to accept diagnosis and treatment, low interest in participating among patients, and an unwillingness among care providers to refer patients to research [- ]. Given the impairments associated with psychotic disorders and the challenges in recruiting and retaining participants [ - ], samples may be biased toward those who are higher functioning and more highly engaged in clinical services. Low recruitment rates are common and can impact study quality, resulting in small convenience samples or clinical trials that are not adequately powered [ , , - ]. When patients and families do participate in research, studies are vulnerable to high attrition rates and missed follow-up assessments, leading to missing data [ , - ]. Together, these issues pose barriers to conducting robust clinical research with people with psychosis and their families.
Digital research tools, such as online platforms, mobile apps, and text messaging, have the potential to reduce barriers to conducting research with this population, improving recruitment and retention rates and leading to more robust, representative, and high-quality research. High rates of digital engagement have been found among young people, including those with psychosis [- ], with email and text message cited as their preferred modes of communication [ ]. Although evidence suggests that many people with psychosis own mobile phones and computers and have access to the internet [ , , - ], digital equity concerns remain for those living in poverty or experiencing homelessness [ ]. Still, these technologies have been used to deliver mobile- and internet-based clinical interventions to those with psychosis, with substantial research examining their effects on symptoms, appointment attendance, and medication adherence [ , , ]. However, few studies have examined how these technologies can be leveraged to engage people with psychosis in research. The body of research on use of electronic questionnaires does not specifically consider the preferences and digital literacy of people with serious mental illness [ ]. Furthermore, given the lack of research in this area, institutional review boards (IRBs) tasked with assessing the risks associated with emerging digital tools have little guidance, which may lead to a slow approval process [ ]. While their high rates of acceptability and utility as health interventions suggest that digital technologies may be effective research tools for people with psychosis and their families, there is little evidence to guide their use. This research gap has become even more apparent in the context of the COVID-19 global pandemic, where digital tools may offer a way to continue research that was postponed to accommodate physical distancing.
This exploratory paper examines the uptake of digital research tools among patients with psychosis and their families, as well as more traditional methods of recruitment and survey administration. We used digital research tools to obtain consent, administer a survey, and compensate patients and their families recruited at the time of their referral to early psychosis intervention (EPI) services. We also offered traditional alternatives, including in-person consent, a pen-and-paper survey option, and compensation with hard copy gift cards. The survey explored patient and family views on facilitators, barriers, and ideas to improve service engagement. These are critical aspects of EPI care given high rates of service disengagement  and gaps in the literature on patient- and family-reported perspectives [ ]. Digital research tools were selected for their potential to remotely capture the perspectives of patients and families who were not well-engaged in clinical services. The objective of this study was to assess the uptake of digital tools to engage patients with provisional psychosis and their families in research, and their preferences for different research administration methods.
The Slaight Centre Early Intervention Service (SCEIS) is an EPI program at the Centre for Addiction and Mental Health in Toronto, Canada. Following the EPI model, it aims to provide comprehensive treatment delivered by a multidisciplinary team early in the course of illness . It provides assessment for young people aged 16-29 years with provisional psychosis and offers treatment for up to 3 years for those with confirmed affective, nonaffective, and substance-induced psychosis. The program receives approximately 600 new referrals each year. Research is integrated into clinical care through a centralized and coordinated research recruitment process. Young people referred to the program between July 2018 and February 2020 and their family members were invited to complete a survey exploring patient- and family-reported facilitators and barriers to engagement in EPI services.
Recruitment and Procedures
This study used Research Electronic Data Capture (REDCap)—a secure, web-based platform with built-in tools for data collection and storage—to send web-based consent forms and surveys to patients and families referred to EPI services [- ]. REDCap self-sufficiently stores email addresses and phone numbers in a secure participant list to send forms directly by email or through SMS text message using a plug-in called Twilio. When the clinic administrator from SCEIS phoned to book the consultation appointment, verbal consent was sought from the patient to send them a link to a web-based consent form via text message or email. After obtaining verbal consent from the patient, when possible, the clinic administrator would ask to speak to a family member to send them the family consent form. Less typically, the family member was consented first if they booked the consultation appointment on behalf of the patient.
The web-based consent form, including a description of the survey's purpose and length, data storage, and investigator, as well as the purpose of the study, was sent after verbal consent was obtained. Two reminders were sent 2 days apart following the survey. In some cases, the appointment was arranged without a direct phone call (eg, by leaving voice-mail messages back and forth or emailing a discharge planner on the inpatient unit). Patients and family members who were missed for recruitment over the phone but presented in person were approached by the research team to explain the study and obtain web-based consent on a tablet or written consent on paper, depending on their preference. Participants who agreed to receive the consent form but had not yet consented were also approached in person. In addition to reminding patients and family members about the research project, in-person administration served to answer questions and provide technical support.
Web-based surveys were sent automatically through REDCap 30 days after participants consented to the study to capture early experiences in the EPI program. Surveys were approximately 12 pages in length, with 1-8 questions per page, and took less than 15 minutes to complete. If the survey was not completed remotely after 2 automated reminders, participants who were attending services were approached to complete the survey in person using a tablet or pen and paper, based on their preference. Participants were also contacted by phone and reminded to complete the survey. Those who completed the survey remotely were given the option to select a Can $10 (US $8.27) electronic gift (e-gift) card from their choice of a major chain of coffee shops, bookstores, or movie theatres upon survey completion; those who completed the survey in person were offered an e-gift or hard copy gift card. Before concluding the study, participants who had consented but not yet completed the survey were sent 2 automated messages notifying them that the study was ending and the link would be expiring. The recruitment process is outlined in. In addition to answering questions about service engagement, participants were asked to self-report their sociodemographic characteristics according to locally developed standardized questions designed to capture health equity factors [ ]. This study was approved by the Research Ethics Board at the Centre for Addiction and Mental Health and is reported in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES; [ ]).
We compiled descriptive statistics for consent and survey completion rates, the demographic characteristics of patients and family members who completed the survey, time and day of the week the survey was completed, and communication and gift card preferences. We also tracked the responses of participants who were contacted by phone to complete the survey. Data were analyzed with descriptive statistics including means, standard deviations, ranges, and percentages using Stata statistical software (StataCorp LLC ).
Consent and survey completion rates, as well as communication and gift card preferences, are outlined in. Of the 801 patients who were identified as being eligible for the study, 447 patients and 187 family members agreed to receive the consent form electronically from the clinic administrator or in person from the research team. Only 8 patients and 7 family members who were approached declined to receive the consent form; the remaining eligible patients were not reached by the clinic administrator directly (eg, they did not return calls to book their intake appointment or communicated by leaving voice mails back and forth), and if they came in person, they were missed by the research team. Approximately half of the patients (216/447, 48.3%) and family members (109/187, 58.3%) consented to participate in the survey. A small number of patients (13/447, 2.9%) and family members (7/187, 3.7%) responded to the consent form but did not wish to participate in the survey. Most patients (182/229, 79.5%) and, to a lesser extent, family members (75/116, 64.7%) who completed the consent form did so remotely as opposed to in person. Even when the consent form was administered in person, all patients and family members who participated in the study opted to complete it on a tablet in REDCap rather than using pen and paper.
|Completion rates and digital preferences||Patients||Family members|
|n (%)||N||n (%)||N|
|Completion rates: consent form|
|Yes||216 (48.3)||447||109 (58.3)||187|
|No||13 (2.9)||447||7 (3.7)||187|
|Did not respond||218 (48.8)||447||71 (38.0)||187|
|Format in which consent form was completed|
|Remote||182 (79.5)||229||75 (64.7)||116|
|In person||47 (20.5)||229||41 (35.3)||116|
|Completion rates: survey|
|Yes||167 (77.3)||216||79 (72.5)||109|
|Did not respond||49 (22.7)||216||30 (27.5)||109|
|Partially completed survey||11 (22.4)||49||7 (23.3)||30|
|Left survey blank or did not open it||38 (77.6)||49||22 (73.3)||30|
|Had technical problems with REDCapa||0 (0)||49||1 (3.3)||30|
|Format in which survey was completed|
|Remote||139 (83.2)||167||68 (86.1)||79|
|In person||28 (16.8)||167||11 (13.9)||79|
|418 (90.5)||447||174 (91.6)||187|
|Text message||19 (4.1)||447||6 (3.2)||187|
|No preference||16 (3.5)||447||3 (1.6)||187|
|Switched between preferences||9 (1.9)||447||7 (3.7)||187|
|Gift card preference|
|Coffee shop||91 (54.5)||167||42 (53.2)||79|
|Bookstore||39 (23.4)||167||20 (25.3)||79|
|Movie theatre||33 (19.8)||167||15 (19.0)||79|
|Lost to follow-up or declined||4 (2.4)||167||2 (2.5)||79|
aREDCap: Research Electronic Data Capture.
Of those who consented, 77.3% (167/216) of patients and 72.5% (79/109) of family members completed the survey. The demographic characteristics of patients and family members who participated in the study are highlighted in. Among the patients, the mean age was 22.8 years, 46.7% (78/167) were female, 63.5% (106/167) identified with racial groups other than White, and 63.5% (106/167) were born in Canada. Most patients (138/167, 82.6%) were single, 64.1% (107/167) were living with family, and 75.5% (126/167) had English as a first language. Almost half of the patients (80/167, 47.9%) attended or completed university, 33.5% (56/167) were in full-time work or school, and 25.7% (43/167) were receiving some form of income assistance. Over one-third of patients (54/167, 32.3%) reported weekly substance use in the past month.
Among family members, the mean age was 47.8 years, 59.5% (47/79) were mothers, 72.2% (57/79) lived with the patient, and 48.1% (38/79) identified with racial groups other than White. Half of the family members (40/79, 50.6%) were born in Canada, 67.1% (53/79) had English as a first language, and 62.0% (49/79) were married or in a relationship. Over one-third (32/79, 40.5%) of family members attended or completed university, 64.6% (51/79) were in full-time work or school, and 70.9% (56/79) were receiving earnings from regular work.
|Characteristics||Patients (N=167)||Family members (N=79)|
|Age (years), mean (SD)||22.8 (3.5)||47.8 (12.6)|
|Relationship to patient (mother), n (%)||N/Aa||47 (59.5)|
|Gender, n (%)|
|Male||76 (45.5)||18 (22.8)|
|Female||78 (46.7)||61 (77.2)|
|Trans, nonbinary, 2-spirit, or unknown||13 (7.8)||0 (0)|
|Racial group, n (%)|
|East, South, or South East Asian||41 (24.6)||12 (15.2)|
|Black African, Caribbean, or North American||23 (13.8)||13 (16.5)|
|Latin American||11 (6.6)||6 (7.6)|
|Middle Eastern||6 (3.6)||0 (0)|
|White European or North American||61 (36.5)||41 (51.9)|
|Other including First Nations, Inuit, Metis, or Indigenous/Aboriginal not included elsewhere; mixed heritage; Indian Caribbean; decline to answer; or do not knowb||25 (15.0)||7 (8.9)|
|Born in Canada, n (%)||106 (63.5)||40 (50.6)|
|Relationship status, n (%)|
|Single, never marriedc||138 (82.6)||17 (21.5)|
|Married, partner, or significant other||29 (17.4)||49 (62.0)|
|Separated, widowed, or divorced||0 (0)||13 (16.5)|
|Living with the patient, n (%)||N/A||57 (72.2)|
|Housing, n (%)|
|Own an apartment/housing||26 (15.5)||N/A|
|Live with parents or other family members||107 (64.1)||N/A|
|Share a place with friends or peers||17 (10.2)||N/A|
|Emergency shelter or couch surfing||4 (2.4)||N/A|
|Supportive/transitional housing, treatment facility, or other||6 (3.6)||N/A|
|Rooming or boarding house||7 (4.2)||N/A|
|English as a first language, n (%)||126 (75.5)||53 (67.1)|
|Education, n (%)|
|Attended or completed high schoold||59 (34.3)||9 (11.4)|
|Attended or completed college, or a trade or technical school||21 (12.6)||21 (26.6)|
|Attended or completed university||80 (47.9)||32 (40.5)|
|Attended or completed graduate school||7 (4.2)||17 (21.5)|
|Currently in full-time school or work, n (%)||56 (33.5)||51 (64.6)|
|Receiving earnings from regular work, n (%)||48 (28.7)||56 (70.9)|
|Receiving income assistance, n (%)||43 (25.7)||5 (6.3)|
aN/A: not applicable.
bRespondents who answered “other”; First Nations, Inuit, Metis, or Indigenous/Aboriginal not included elsewhere; mixed heritage; Indian Caribbean; “decline to answer,” or “do not know” were grouped together to supress small cells.
cRespondents who were separated, widowed, or divorced were grouped in with “single, never married” to suppress small cells.
dRespondents who answered “other,” “don’t know,” or “decline to answer” were included in the group who attended or completed high school to suppress small cells.
In total, 83.2% (139/167) of patients and 86.1% (68/79) of family members completed the survey remotely. Most surveys were completed on weekdays and more than half were completed between 12 PM and 6 PM, as shown in. Almost all patients (418/462, 90.5%) and family members (174/190, 91.6%) requested to receive the consent form and survey by email, and only 4.1% (19/462) and 3.2% (6/190), respectively, preferred text message. Over half of the patients and family members preferred to receive e-gift cards from a coffee shop as study compensation (91/167, 54.5%, and 42/79, 53.2%, respectively). As with the consent form, all patients and family members who completed the survey in person opted to use a tablet rather than pen and paper.
Approximately one quarter of patients (49/216, 22.7%) and family members (30/109, 27.5%) did not complete the survey. Of these, 22.4% (11/49) of patients and 23.3% (7/30) of family members partially completed the survey, whereas the remainder did not open the survey or left it blank. One family member was not able to complete the survey because REDCap was having technical problems. Participants who had consented but not yet completed the survey were contacted by phone; those who were reached reported that they were either no longer interested in participating or asked to be sent the survey again. Participants who were unreachable by phone had voice mails that were not activated, phone numbers that were no longer in service, and incorrect contact information.
To our knowledge, this is one of the first studies to examine the use of digital tools to engage people with psychosis and their families in research. Patients and family members referred to EPI services were generally agreeable to receiving and completing the web-based consent form and survey and preferred these remote online administration methods to in-person and pen-and-paper–based methods. However, in-person engagement was still a useful strategy for reminding patients and family members about the research, explaining the research, and providing technical support. Patients and family members preferred to engage in research by email rather than text message and receive e-gift cards from a coffee shop as compensation for their participation. Our results suggest that digital tools can support effective recruitment, consenting, and survey administration among people with psychosis and their family members. They can also be used to capture the perspectives of those who may have been hard to engage with more traditional research methods. These are important findings that have the potential to optimize study recruitment and retention, reduce research burden, and improve the quality and representativeness of psychosis research.
Many patients and family members agreed to receive the web-based consent form, and those who consented had high survey completion rates. Although few studies have used online recruitment and survey administration methods with this population, our completion rates for this one-time survey were substantially higher than those found in a study that delivered regular surveys through a mobile app to youth receiving EPI care . Our high completion rates could be explained by the low burden and low-risk nature of the study. Thus, online methods may not be effective at engaging patients and families in research if the study is perceived as burdensome or the risk is perceived as high. IRBs should assess the risks and benefits of participating in remote research while considering decision-making capacity, stigma, and data privacy concerns [ ]. It is also likely that completion rates are influenced by the study setting; in this study, the setting was an EPI program that integrates clinical care and research and uses a coordinated recruitment process that provides all patients with an opportunity to participate in research. Completion rates may also be influenced by the timing of study recruitment, in that patients and families may feel pressure to participate in research when they are entering a mental health program because of worries that their refusal may impact their care. Future studies should examine the feasibility and effectiveness of using digital research tools in more complex studies that recruit participants at different stages in their care journey or other mental health services, particularly those that may not have the infrastructure to support a robust and coordinated recruitment system.
Close to half of the patients who completed the survey were female. This is more than the average for EPI programs, in which the proportion of females is closer to one-quarter to one-third , and while the majority identified with racial groups other than White, the proportion (less than 60%) was lower than our patient population as a whole (closer to 70%) [ ]. This suggests that further work is needed to ensure that psychosis research is representative of the population served in clinical programs.
We found that more than one-third of family members and almost one-quarter of patients completed the consent form in person. These findings suggest that providing options for in-person recruitment methods is still a meaningful strategy to engage patients with psychosis and their families in research, especially during the consent process. Family members in particular relied more on in-person support to consent to the study but almost all went on to complete the survey remotely, suggesting that once family members understand and consent to a study, there is a high likelihood that they will feel comfortable using online survey administration methods. This in-person support may also have acted as a technical orientation to REDCap. While digital tools can augment current methods of recruitment, they may also bias results to individuals from specific backgrounds. In-person engagement strategies may be particularly meaningful for urban EPI programs that serve multicultural populations with language barriers or vulnerable groups, including people experiencing homelessness, who may not have consistent access to technology [, , , ].
Patients and family members in our study preferred to receive research materials by email rather than text message. This finding was somewhat surprising, given the high rates of mobile phone ownership and text messaging among young people with psychosis [, ]. While previous studies suggested that youth with psychosis prefer text messaging and social media, our findings suggest that for the purposes of research, patients with psychosis and their families prefer to receive consent forms and surveys via email [ , , ]. Although the reasons for this are unclear, one possibility is that online surveys are more user-friendly to complete by email using a computer or tablet than by texted link on a smartphone. Also, patients and family members may have felt that email was a more secure communication tool than text message [ ]. We also found that most surveys were completed on weekday afternoons, and patients and family members preferred to receive e-gift cards from a coffee shop as study compensation. This finding highlights factors that may yield better recruitment and completion rates in future studies that engage people with psychosis and their families in research.
Similar to past studies, we had challenges recruiting patients with psychosis and their families in our digital research study and retaining participants between consenting and completing the survey. Many patients and family members did not respond to the web-based consent form and survey, some partially completed it, and one family member experienced technical difficulties. Engaging this population via phone call reminders also posed challenges, including incorrect contact information and phone numbers that were no longer in service. Similar barriers were encountered in prior studies that relied on in-person research visits and exemplify some of the difficulties in recruiting and retaining this population in research [, , ]. While online administration methods are feasible and acceptable among patients with psychosis and their families and may increase research engagement, challenges are still present.
These findings have timely implications in the context of the COVID-19 pandemic as the health care system navigates the shift to virtual care and remote research . Digital tools such as those used in this study can facilitate the adaptation and continuation of current projects that are on hold as well as the initiation of new research. The rapid adoption of digital tools to support research amid COVID-19 has put pressure on IRBs to establish standard procedures and safety protocols for remote research, including consent and assessment frameworks. As digital tools become widely deployed to facilitate different types of research in varied populations, it is incumbent on researchers adopting them to study their uptake and safety. New frameworks for remote research and data supporting its acceptability, feasibility, and safety may allow it to persist beyond the pandemic as a way to improve research engagement.
Strengths and Limitations of the Study
Our study has notable strengths. While digital tools are increasingly being used to support research with populations with mental illness, and particularly young people [, ], we are not aware of studies that have specifically examined the uptake of digital research tools with people with psychosis and their families. This is an important advance given the established barriers to research recruitment in psychosis and the paucity of literature studying novel approaches to increase research engagement. This is particularly timely given the impact of the COVID-19 pandemic on clinical research across disorders, when evidence to guide the implementation of tools to conduct research remotely is so critical. Establishing feasibility of this approach to remote research in family members provides additional value, given the importance of capturing caregiver perspectives in psychosis research [ ].
There are some study limitations to consider. First, while we attempted to approach all referred patients and their family members, those who were not approached or did not complete the consent form or survey may have been systematically different from survey completers. Unfortunately, we were unable to gather information on the patients and family members who did not participate. Approaching potential participants in person did allow us to reach some individuals who may otherwise have declined participation due to problems accessing a mobile device or the internet. Family member participation may have been subject to additional sampling bias if the family member was not present to verbally consent to receive the web-based consent form, as recruitment mostly relied on their in-person attendance at clinic appointments. Some additional factors may limit the generalizability of our findings, as the study was conducted at a well-resourced EPI program in Toronto, Canada, with an integrated clinical research program that may have been more conducive to innovative research methods.
Recommendations for Conducting Remote Research in Psychosis
Our results suggest that future studies would benefit from including both in-person and virtual engagement strategies, particularly for family members who may want to discuss the study in person and require additional support from the research team. Further, prolonged engagement in the form of telephone and email reminders may be necessary to increase participation rates. Given that in-person administration may have also provided technical support, we suggest that the selected consent and survey platform be tested with a target audience before it is launched so that functionality issues can be addressed. Outlining potential usability challenges in the consent form or a tip sheet could help participants orient themselves to the platform and make the online methods more user-friendly. Consent forms and surveys may have the highest yield if they are sent to participants on weekday afternoons. Participant attrition between consent and survey completion suggests that participants completing surveys remotely may have restrictions on time, and platforms that allow them to partially complete surveys and then return to complete them later may yield higher completion rates. To continue developing and using digital research methods more broadly, researchers should build relationships with their IRBs and work together to develop ethical standards for conducting remote research safely.
Digital tools offer a low-burden way of engaging people with psychosis and their families in research. These findings demonstrate that virtual consent and survey administration methods are feasible and acceptable and can be used to capture the perspectives of those who may have been hard to engage with more traditional research methods. Leveraging data on preferences and usage patterns to guide the use of a variety of digital and in-person research tools can help optimize study recruitment and retention, improving the quality and representativeness of psychosis research during the COVID-19 global pandemic and beyond.
The authors wish to acknowledge Royce Jean-Louis for additional data management support and Augustina Ampofo for providing feedback on the recruitment materials and survey. Dr. Kozloff’s work was supported by an award from the 2017 Slaight Family Centre for Youth in Transition Internal Funding Competition through the CAMH Foundation and Dr. Polillo’s work was supported by a Mitacs Elevate Postdoctoral Fellowship. NK conceptualized the study. AP and NK drafted and revised sections of the paper. GF, ANV, SAK, SH, SS, AS, and VS made critical revisions and approved the final version.
Conflicts of Interest
Study recruitment process.PNG File , 114 KB
Checklist for Reporting Results of Internet E-Surveys (CHERRIES).DOCX File , 19 KB
- Furimsky I, Cheung AH, Dewa CS, Zipursky RB. Strategies to enhance patient recruitment and retention in research involving patients with a first episode of mental illness. Contemp Clin Trials 2008 Nov;29(6):862-866. [CrossRef] [Medline]
- Woodall A, Howard L, Morgan C. Barriers to participation in mental health research: findings from the Genetics and Psychosis (GAP) Study. Int Rev Psychiatry 2011;23(1):31-40. [CrossRef] [Medline]
- Jørgensen R, Munk-Jørgensen P, Lysaker PH, Buck KD, Hansson L, Zoffmann V. Overcoming recruitment barriers revealed high readiness to participate and low dropout rate among people with schizophrenia in a randomized controlled trial testing the effect of a Guided Self-Determination intervention. BMC Psychiatry 2014 Feb 03;14:28 [FREE Full text] [CrossRef] [Medline]
- Doyle R, Turner N, Fanning F, Brennan D, Renwick L, Lawlor E, et al. First-episode psychosis and disengagement from treatment: a systematic review. Psychiatr Serv 2014 May 01;65(5):603-611. [CrossRef] [Medline]
- Penttilä M, Jääskeläinen E, Hirvonen N, Isohanni M, Miettunen J. Duration of untreated psychosis as predictor of long-term outcome in schizophrenia: systematic review and meta-analysis. Br J Psychiatry 2014 Aug;205(2):88-94. [CrossRef] [Medline]
- Morgan VA, Waterreus A, Carr V, Castle D, Cohen M, Harvey C, et al. Responding to challenges for people with psychotic illness: Updated evidence from the Survey of High Impact Psychosis. Aust N Z J Psychiatry 2017 Feb;51(2):124-140. [CrossRef] [Medline]
- Farris MS, Devoe DJ, Addington J. Attrition rates in trials for adolescents and young adults at clinical high-risk for psychosis: A systematic review and meta-analysis. Early Interv Psychiatry 2020 Oct;14(5):515-527. [CrossRef] [Medline]
- Loughland CM, Carr VJ, Lewin TJ, Barnard RE, Chapman JL, Walton JM. Potential sampling and recruitment source impacts in schizophrenia research. Psychiatry Res 2004 Feb 15;125(2):117-127. [CrossRef] [Medline]
- Addington J, McGregor L, Marulanda D, Raedler T. Recruitment strategies for the detection of individuals at clinical high risk of developing psychosis. Epidemiol Psychiatr Sci 2013 Jun;22(2):181-185 [FREE Full text] [CrossRef] [Medline]
- Lee H, Schepp KG. Lessons learned from research with adolescents with schizophrenia and their families. Arch Psychiatr Nurs 2013 Aug;27(4):198-203. [CrossRef] [Medline]
- Hernandez M, Franco R, Kopelowicz A, Hernandez MY, Mejia Y, Barrio C, et al. Lessons Learned in Clinical Research Recruitment of Immigrants and Minority Group Members with First-Episode Psychosis. J Immigr Minor Health 2019 Feb;21(1):123-128 [FREE Full text] [CrossRef] [Medline]
- Gilbody S, Wahlbeck K, Adams C. Randomized controlled trials in schizophrenia: a critical perspective on the literature. Acta Psychiatr Scand 2002 Apr;105(4):243-251. [CrossRef] [Medline]
- Szymczynska P, Walsh S, Greenberg L, Priebe S. Attrition in trials evaluating complex interventions for schizophrenia: Systematic review and meta-analysis. J Psychiatr Res 2017 Jul;90:67-77. [CrossRef] [Medline]
- Villeneuve K, Potvin S, Lesage A, Nicole L. Meta-analysis of rates of drop-out from psychosocial treatment among persons with schizophrenia spectrum disorder. Schizophr Res 2010 Aug;121(1-3):266-270. [CrossRef] [Medline]
- Ennis L, Rose D, Denis M, Pandit N, Wykes T. Can't surf, won't surf: the digital divide in mental health. J Ment Health 2012 Aug;21(4):395-403 [FREE Full text] [CrossRef] [Medline]
- Firth J, Cotter J, Torous J, Bucci S, Firth JA, Yung AR. Mobile Phone Ownership and Endorsement of "mHealth" Among People With Psychosis: A Meta-analysis of Cross-sectional Studies. Schizophr Bull 2016 Mar;42(2):448-455 [FREE Full text] [CrossRef] [Medline]
- Firth J, Torous J. Smartphone Apps for Schizophrenia: A Systematic Review. JMIR Mhealth Uhealth 2015 Nov 06;3(4):e102 [FREE Full text] [CrossRef] [Medline]
- D'Arcey J, Collaton J, Kozloff N, Voineskos AN, Kidd SA, Foussias G. The Use of Text Messaging to Improve Clinical Engagement for Individuals With Psychosis: Systematic Review. JMIR Ment Health 2020 Apr 02;7(4):e16993 [FREE Full text] [CrossRef] [Medline]
- Lal S, Dell'Elce J, Tucci N, Fuhrer R, Tamblyn R, Malla A. Preferences of Young Adults With First-Episode Psychosis for Receiving Specialized Mental Health Services Using Technology: A Survey Study. JMIR Ment Health 2015;2(2):e18 [FREE Full text] [CrossRef] [Medline]
- Abdel-Baki A, Lal S, D-Charron O, Stip E, Kara N. Understanding access and use of technology among youth with first-episode psychosis to inform the development of technology-enabled therapeutic interventions. Early Interv Psychiatry 2017 Feb;11(1):72-76. [CrossRef] [Medline]
- Aref-Adib G, O'Hanlon P, Fullarton K, Morant N, Sommerlad A, Johnson S, et al. A qualitative study of online mental health information seeking behaviour by those with psychosis. BMC Psychiatry 2016 Jul 11;16:232 [FREE Full text] [CrossRef] [Medline]
- McInnes DK, Li AE, Hogan TP. Opportunities for engaging low-income, vulnerable populations in health care: a systematic review of homeless persons' access to and use of information technologies. Am J Public Health 2013 Dec;103 Suppl 2:e11-e24. [CrossRef] [Medline]
- Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res 2014 Jun;156(1):96-106. [CrossRef] [Medline]
- Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009 Jul 08(3):MR000008. [CrossRef] [Medline]
- Torous J, Nebeker C. Navigating Ethics in the Digital Age: Introducing Connected and Open Research Ethics (CORE), a Tool for Researchers and Institutional Review Boards. J Med Internet Res 2017 Feb 08;19(2):e38 [FREE Full text] [CrossRef] [Medline]
- Lal S, Malla A. Service Engagement in First-Episode Psychosis: Current Issues and Future Directions. Can J Psychiatry 2015 Aug;60(8):341-345 [FREE Full text] [CrossRef] [Medline]
- Early Psychosis Intervention Program Standards. Ontario Ministry of Health and Long-Term Care. Toronto; 2011 Mar 31. URL: https://www.health.gov.on.ca/english/providers/pub/mental/epi_program_standards.pdf [accessed 2021-05-15]
- REDCap (Research Electronic Data Capture). Vanderbilt University. 2018. URL: https://www.project-redcap.org/ [accessed 2020-04-02]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
- Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, REDCap Consortium. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019 Jul;95:103208 [FREE Full text] [CrossRef] [Medline]
- Wray R, Agic B, Bennett-AbuAyyash C, Kanee M, Lam R, Mohamed A. We ask because we care: The Tri-Hospital + TPH Health Equity Data Collection Research Project Report. Measuring Health Equity. 2013 Jun. URL: http://torontohealthequity.ca/wp-content/uploads/2017/05/We-Ask-Because-We-Care-Summary-Report.pdf [accessed 2020-07-23]
- Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [CrossRef] [Medline]
- StataCorp. Stata Statistical Software. 16 ed. College Station, TX: StataCorp LLC; 2019.
- Kumar D, Tully LM, Iosif A, Zakskorn LN, Nye KE, Zia A, et al. A Mobile Health Platform for Clinical Monitoring in Early Psychosis: Implementation in Community-Based Outpatient Early Psychosis Care. JMIR Ment Health 2018 Feb 27;5(1):e15 [FREE Full text] [CrossRef] [Medline]
- Anderson KK, Mukherjee SD. The need for additional safeguards in the informed consent process in schizophrenia research. J Med Ethics 2007 Nov;33(11):647-650 [FREE Full text] [CrossRef] [Medline]
- Anderson KK, Norman R, MacDougall A, Edwards J, Palaniyappan L, Lau C, et al. Effectiveness of Early Psychosis Intervention: Comparison of Service Users and Nonusers in Population-Based Health Administrative Data. Am J Psychiatry 2018 May 01;175(5):443-452. [CrossRef] [Medline]
- Kozloff N, Voineskos A, Foussias G, Polillo A, Kidd S, Bromley S, et al. Factors Associated with Early Risk of Disengagement from Early Psychosis Intervention Services. In: Schizophrenia Bulletin. 2020 Presented at: Congress of the Schizophrenia International Research Society; April 2020; Virtual Conference (Originally Scheduled for Florence, IT) p. S130-S131 URL: https://academic.oup.com/schizophreniabulletin/article/46/Supplement_1/S130/5839422 [CrossRef]
- Anderson KK, Cheng J, Susser E, McKenzie KJ, Kurdyak P. Incidence of psychotic disorders among first-generation immigrants and refugees in Ontario. CMAJ 2015 Jun 16;187(9):E279-E286 [FREE Full text] [CrossRef] [Medline]
- Bonet L, Llácer B, Hernandez-Viadel M, Arce D, Blanquer I, Cañete C, et al. Differences in the Use and Opinions About New eHealth Technologies Among Patients With Psychosis: Structured Questionnaire. JMIR Ment Health 2018 Jul 25;5(3):e51 [FREE Full text] [CrossRef] [Medline]
- Bucci S, Morris R, Berry K, Berry N, Haddock G, Barrowclough C, et al. Early Psychosis Service User Views on Digital Technology: Qualitative Analysis. JMIR Ment Health 2018 Oct 31;5(4):e10091 [FREE Full text] [CrossRef] [Medline]
- Kannisto KA, Koivunen MH, Välimäki MA. Use of mobile phone text message reminders in health care services: a narrative literature review. J Med Internet Res 2014 Oct 17;16(10):e222 [FREE Full text] [CrossRef] [Medline]
- Kozloff N, Mulsant BH, Stergiopoulos V, Voineskos AN. The COVID-19 Global Pandemic: Implications for People With Schizophrenia and Related Disorders. Schizophr Bull 2020 Jul 08;46(4):752-757 [FREE Full text] [CrossRef] [Medline]
- Lal S, Starcevic DJ, Fuhrer R. Youth Experiences With Referrals to Mental Health Services in Canada: Protocol for a Web-Based Cross-Sectional Survey Study. JMIR Res Protoc 2020 Mar 24;9(3):e16945 [FREE Full text] [CrossRef] [Medline]
- Hawke LD, Barbic SP, Voineskos A, Szatmari P, Cleverley K, Hayes E, et al. Impacts of COVID-19 on Youth Mental Health, Substance Use, and Well-being: A Rapid Survey of Clinical and Community Samples: Répercussions de la COVID-19 sur la santé mentale, l'utilisation de substances et le bien-être des adolescents : un sondage rapide d'échantillons cliniques et communautaires. Can J Psychiatry 2020 Oct;65(10):701-709 [FREE Full text] [CrossRef] [Medline]
- Soklaridis S, McCann M, Waller-Vintar J, Johnson A, Wiljer D. Where is the family voice? Examining the relational dimensions of the family- healthcare professional and its perceived impact on patient care outcomes in mental health and addictions. PLoS One 2019;14(4):e0215071 [FREE Full text] [CrossRef] [Medline]
|CHERRIES: Checklist for Reporting Results of Internet E-Surveys|
|e-gift: electronic gift|
|EPI: early psychosis intervention|
|IRB: institutional review board|
|REDCap: Research Electronic Data Capture|
|SCEIS: Slaight Centre Early Intervention Services|
Edited by G Eysenbach; submitted 04.10.20; peer-reviewed by S Lal, C García, P Wicks; comments to author 15.11.20; revised version received 10.01.21; accepted 04.04.21; published 31.05.21Copyright
©Alexia Polillo, Aristotle N Voineskos, George Foussias, Sean A Kidd, Andreea Sav, Steve Hawley, Sophie Soklaridis, Vicky Stergiopoulos, Nicole Kozloff. Originally published in JMIR Mental Health (https://mental.jmir.org), 31.05.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.