Informing the Future of Integrated Digital and Clinical Mental Health Care: Synthesis of the Outcomes From Project Synergy

Introduction

Mental Health Service Reform

In 2014, the National Mental Health Commission invited the Young and Well Cooperative Research Centre (CRC) to provide advice about digital mental health solutions for young people [1]. Following interviews with mental health and industry leaders, the Young and Well CRC promoted the view that there was an urgent need to broaden research and development (R&D) focus from standalone mental health apps, digital health records, and other simple technologies (Digital Health 1.0) to highly coordinated integration of health information technologies (HITs) within more effective health care delivery systems (HIT Digital Health 2.0). That is, new systems of care would integrate both conventional and novel in-clinic and web-based mental health services, thereby broadening the rapid provision of scalable forms of high-quality care.

As a result, the National Mental Health Commission recommended that HITs be considered a cornerstone of mental health service reform in Australia. This would require that the mental health system be overhauled to embed digital health within conventional in-clinic services [2]. More recently, this recommendation was echoed by the Australian Productivity Commission review of mental health. It calls for increased use of technology through a national digital platform, emphasizing initial assessment and referral processes, as well as increasing service user access to a broader range of interventions and supports [3]. However, there is still a considerable need for increased practical knowledge as to how to best incorporate HITs into clinical services, as well as how to capitalize on the aggregate information available from HITs that could be used to drive more effective coordination of care, enhanced allocation of clinical resources, faster identification of risk, and greater emphasis on achieving a higher proportion of improved health, social, and economic outcomes.

Over the past 20 years, our research group has actively advocated for the use of technology to drive enhanced mental health care, and broader system reform, in Australia [4-8]. Although there is a growing evidence base supporting the use of digital technologies, the integration of digital solutions in mental health services remains relatively rare. As an example, Titov et al [9] have reported success in integrating internet-delivered cognitive behavioral therapy as part of clinical care with a mental health professional in 5 clinics internationally. Unfortunately, far more studies highlight barriers to successful implementations of digital health solutions [10-13], and further research is required to identify the most effective ways in which to design and implement evidence-based HITs as part of clinical care at scale.

Project Synergy

The Australian Government Department of Health commissioned Project Synergy in 2014 to be conducted by the Young and Well CRC in partnership with the University of Sydney’s Brain and Mind Centre. However, following the completion of the Young and Well CRC’s work in 2016, InnoWell, a joint venture enterprise between the University of Sydney and PwC (Australia), was established to deliver the remainder of the work. The overarching objective of Project Synergy is to explore how best to use such new HITs to drive mental health service reform toward delivering better outcomes for people experiencing mental ill-health [4].

Because of its scale, Project Synergy had 2 phases. Phase I (2014-2016; Aus $5.5 million [US $3.97 million]) investigated the use of digital (including smartphone) technologies to transform the provision of mental health services for Australian young people through co-designing, building, implementing, and evaluating prototypes with representative end user populations. These included young people along with their supportive others (eg, family, carers, and friends), health professionals, and service providers [4]. Four research studies were conducted on prototypical software and concepts that included the following: (1) a university-based health and well-being prototype [14]; (2) a synergized web-based system including a triage e-tool, health and well-being e-tool, and a Mental Health eClinic with a multidimensional web-based assessment, shared mental health dashboard, and video visit capabilities [15]; (3) wireframes for a tiered suicide risk management protocol [16]; (4) functionality for trusted supportive others [16-18]; (5) and a proof of concept study for implementing technologies into primary youth mental health services [19]. The details of these preliminary studies have been published previously [4].

The remainder of this paper focuses on phase II (2017-2021; Aus $30 million [US $21.63 million]), which concluded in June 2021. In phase II, prototypes from phase I were iterated into a beta version of a single web-based platform named the InnoWell Platform and implemented into both conventional in-clinic and web-based primary mental health services. The fundamental aim was articulated as delivering the right level of care to individuals upon their first presentation to service (Right care, First time) as underpinned by clinical stage, level of clinical need, and personal choice, with the aim of preventing progression to more serious complex and severe forms of illness.

InnoWell Platform

As detailed by Davenport et al [20], the InnoWell Platform collects both multidimensional self-report and informant (eg, supportive others and health professionals) information via questionnaires to capture a holistic understanding of a service user’s clinical presentation, level of need, and personal preferences, to then monitor their progress over time. Specifically, survey data were collected across a range of biopsychosocial health domains, such as psychological distress, depressed mood, sleep-wake cycle, physical health, and social and occupational functioning. Further objective behavioral data can also be collected via third-party integrations (eg, Fitbit fitness trackers [Google LLC]). As shown in Figure 1, the multidimensional assessment results are presented back to service users in a dashboard color coded with a traffic light system to ease interpretability (ie, green, yellow, orange, and red are used to denote increasing degrees of symptom severity or level of impairment for each health domain). Service users are also provided with a description of each health domain and a brief explanation of the result they received (Figure 2).

The assessment results can then be used collaboratively by the service user and their health professional to ensure both parties are informed and actively engaged in treatment decisions in a responsive way. This is designed to occur not only at the outset of the care journey but also on an ongoing basis, as service users and clinicians jointly track progress over time. Importantly, as shown in Figure 3, service users can also access recommended nonclinical care options (eg, fact sheets, apps, e-tools, and other web-based systems) to support their mental health and well-being proactively (eg, before receiving and in conjunction with the clinical care received through the service).

‎View this figure

‎View this figure

‎View this figure

Objective

The InnoWell Platform was co-designed and implemented in diverse service settings across different age groups and populations. The Project Synergy effectiveness trials aimed to deliver critical insights about how HITs, exemplified by the InnoWell Platform, can be optimally configured and implemented as part of standard care to meet the needs of the populations under focus, their immediate care team (health professionals and service providers), and supportive others. Furthermore, a mixed methods approach was established to evaluate the web-based platform and the impact of its implementation (where available) in the individual services to iteratively refine the InnoWell Platform and the technology-enabled service model into which it was embedded.

As shown in Figure 4 [14-16,18-19,21-22,23], this paper seeks to summarize the methodologies used in phase II of Project Synergy and highlight their associated research outcomes. Specifically, we describe the process by which the original beta version of the InnoWell Platform was (1) iteratively co-designed, developed, and refined through participatory design workshops with individuals with lived experience of mental ill-health, health professionals, and service administrators to create service-specific configurations of the digital platform; (2) tested by end users to determine acceptance for each participating service or service user group; (3) implemented within a technology-enabled service model that had been co-designed with service staff through service mapping; and (4) evaluated to determine its impact at the level of the service user, health professional, and service. We provide an overview of the key outcomes from this research, highlight tools that can be used to drive the uptake of HITs by both service users and health professionals, and provide examples of how smart, digital technologies can drive innovation in models of care at the service and system level to better address demand management, early intervention, and rapid assessment and triage of at-risk users to specialized care.

‎View this figure

Methods

Overview of the Project Synergy Research Streams

Phase II of Project Synergy aimed to conduct 4 rolling research streams to evaluate the engagement with and efficiency and effectiveness of the InnoWell Platform. The research streams focused on four populations across the life span, namely, veterans and their families, young children and their families, young people, and older adults. Importantly, through the internal R&D process, these rolling research streams enabled the iterative development and redevelopment, evaluation, and refinement of essential functionality within the InnoWell Platform (eg, assessment, dashboard, algorithms, real time data tracking, aggregate service indicator dashboard, immediate staged care recommendations, and service pathways) over time. One or more research studies were planned in relation to each research stream, in collaboration with relevant mental health services when feasible or with specific participant groups (ie, young children and families). In relation to the former, potential participating services were identified based on the alignment between their service user group or groups and the research streams. After agreeing to participate in Project Synergy, a research agreement was formalized between the University of Sydney and the participating service. As described in this Methods section, the methodology that underpins Project Synergy’s research trials includes co-design activities (ie, participatory design workshops, user acceptance testing, and service mapping sessions), implementation science, and a mixed methods evaluation of the impact of the InnoWell Platform and the associated technology-enabled model of care at the level of the individual, health professional, and service based on key performance indicators.

Participating Services and Service User Groups

The InnoWell Platform was co-designed with and tested by the target end users, including individuals seeking or engaged in mental health care. Digital health solutions are widely recognized as a potential means by which to address issues of demand management in mental health globally, and, to a degree, the COVID-19 pandemic fast-tracked the uptake of technology as part of mental health care. However, it is critical to recognize that factors such as age, technical skills, literacy, and health literacy affect the ways in which individuals engage with technology [24]. Despite the proliferation of digital health solutions, such as apps, in recent years, little attention has been paid to how these tools will be used by individuals from diverse populations [25]. In addition, only 30% of apps developed by established digital health companies have tested their apps with individuals with clinical conditions [26]. To ensure that the InnoWell Platform and technology-enabled model of care met the needs of the target end users, it was crucial to ensure that the research studies were conducted with individuals engaged in clinical care. Participating services and the service users to whom they provide care are presented in Textbox 1.

Additional co-design and preliminary evaluation research studies were conducted with specific service user groups not affiliated with a mental health service including older adults (aged ≥50 years) and children (aged ≥8 years) and families to inform future iterations of the InnoWell Platform. These participants were recruited through active research clinics at the University of Sydney’s Brain and Mind Centre as well as via nongovernmental and private organizations associated with the Brain and Mind Centre. The co-design studies were advertised via posters and postcards at the advertising site as well as via their websites and social media pages. Importantly, although a diverse grouping of services and service users participated in Project Synergy, further investigation is still required to determine the usability, acceptability, and relevance of the InnoWell Platform for individuals who are from a culturally and linguistically diverse background, identify as Aboriginal and Torres Strait Islanders, or identify as having a disability.

Finally, approximately 50 local stakeholders from the North Coast NSW Primary Health Network, including representatives from health and social policy agencies, nongovernment organizations, primary care providers, emergency services, research institutions, community groups, and, importantly, people with lived experience of suicide, contributed to the development of a system dynamics model. Using a broad systems perspective, stakeholders identified key outcomes of interest and subsequently provided feedback to iteratively refine the model.

Ethics Approval

This research required multiple ethics approvals by various human research ethics committees (HRECs) owing to the diverse organizational structures governing each of the participating services. The clinical trial allowing users of the InnoWell Platform to share their deidentified data for research purposes was approved by the Northern Sydney Local Health District HREC (reference HREC/17/HAWKE/480) and was registered with the Australian New Zealand Clinical Trial Registry (ACTRN12618001676202). Separate ethics were sought for the co-design and impact evaluation research for each participating service. The governing bodies of some mental health services required applications to be submitted through their own internal HRECs (ie, the Department of Defence and Veterans’ Affairs HREC Projects 056-18 and 017-17), whereas others preferred that required applications were submitted through the University of Sydney HREC (Projects 2017/944, 2018/849, 2018/041, 2018/962, 2019/172, 2019/683, 2019/767, 2020/141, and 2020/246). The development of the system dynamics model did not require ethics approval as it involved secondary analysis of a broad range of aggregate data sets and synthesis of existing research. Stakeholders were invited to contribute their expertise to this model but were not the subject of the research activity. Figure 5 outlines the diverse research methodologies that were used as part of Project Synergy, underpinning all aspects of the project from co-design and development of the InnoWell Platform through to impact evaluation and dissemination of outcomes.

‎View this figure

Co-design

An R&D cycle, described by Davenport et al [17], was established and used to explicitly position end users as empowered participants in all stages of the research—from planning and design through to implementation and evaluation. Central to the R&D cycle are participatory design workshops to discover, prototype, and evaluate potential design solutions through the use of prompted discussion, review of working prototypes (wireframes) and existing InnoWell Platform functionality, personas (ie, a fictitious potential user for whom the product is being designed), and user journeys. Direct engagement with potential end users as well as other key stakeholders (ie, representatives from funding agencies) through targeted participatory workshops is essential to ensure the development and build of acceptable, usable, and scalable digital tools. The primary objective of the workshops was to co-design and configure the InnoWell Platform for each participating service or user group to ensure that it met the needs of the target end users, including people with lived experience accessing care and their supportive others as well as health professionals, service managers, and administrators.

Workshop agendas were tailored to each service by the research management team, comprising researchers, health professionals, and service administrators, to determine how best to discover how the InnoWell Platform might enhance standard care. Areas of focus included increased accessibility to care, waitlist management, risk identification, coordination of care with other services, and the potential for digital tools to support self-management and well-being. Participants worked together to create mockups of the technology, including the landing page, assessments, and dashboard, highlighting key features and functions that they thought would address identified pain points for the service and service users. Only after having captured participants’ unbiased ideas were they then provided with paper wireframes or screenshots of the InnoWell Platform on which to provide specific feedback on prototyped or existing features and functionality. The final outcome of the workshops was a service-specific configuration of the InnoWell Platform.

Importantly, the service-specific version of the InnoWell Platform was then tested by end users in one-to-one, face-to-face, 90-minute user testing sessions. Each session was facilitated by a member of the research team, and a scribe was present to take notes to capture all feedback accurately. The aim of the user testing sessions was to (1) learn if participants were able to navigate the InnoWell Platform intuitively, (2) identify any features of the platform that the user would want to see changed before implementation of the platform into the service (ie, showstoppers), (3) gain insight into participants’ experience of using the platform, (4) identify possibilities for improving the platform to better meet user needs and wants, and (5) learn how best to implement the platform into the participating service. Testers were observed interacting with the InnoWell Platform and asked to think aloud when exploring its features, providing both positive and negative feedback about the user experience. In addition to freely exploring the features and functionality of the InnoWell Platform, testers were also asked to complete specific scenarios relevant to their role (eg, service user, health professional, and service manager). For example, a health professional might be asked to try entering data into the InnoWell Platform about a service user and then to comment on their experience (ie, ease of completing the action and gaps in the functionality). The feedback was integrated back into the iterative R&D cycle to inform the ongoing refinement of the product by our collaborators at InnoWell. Furthermore, user testing was also used to achieve acceptance of the technology, signally that it was ready for implementation within the service.

Finally, in parallel with the participatory design workshops and user testing sessions, our team also used service mapping, a process mapping methodology, to understand how best to integrate the InnoWell Platform within participating mental health services. As published in LaMonica et al [27], we have used the term service mapping to refer to the structured approach to understanding the needs, existing processes, gaps in care, and performance of a mental health service mapped against an individual’s journey through the service. More specifically, participants worked collaboratively with the researchers to first map the current service user journey through the participating service on a whiteboard or butcher’s paper. The gaps or limitations of the current care pathway were reviewed relative to the key features of high-quality mental health care to identify areas for improvement or reform. The existing map was then revised to include technical element supported by the InnoWell Platform (eg, web-based initial assessment and clinical and nonclinical care options). The outcome was a co-designed technology-enabled model of care highlighting the intersection and impact of the InnoWell Platform at key points in care (eg, referral, assessment, treatment planning, and routine outcome monitoring).

Table 1 summarizes the research aims and co-design activities for each of our participating services and service user groups.

Implementation and the Clinical Trial

The effectiveness of the InnoWell Platform is being assessed in a naturalistic, prospective, quasiexperimental (ie, uncontrolled, nonrandomized) clinical trial. It is important to note that this methodology is rapidly gaining recognition as a practical and efficient method of assessing change over time in users under ecologically valid conditions (ie, real-world conditions of constantly changing apps and technologies) [29,30]. Importantly, by conducting group comparisons based on clinical stage, demographics, and locality, it is possible to infer strong evidence for the usefulness (or lack thereof) of the technology-enabled blended model of care.

Regarding implementation and conduct of the clinical trial, the co-designed and configured InnoWell Platform was implemented at each of the participating services within the redesigned, technology-enabled service model as part of standard practice in accordance with our established implementation science strategy [31]. Importantly, standard practice was defined by the service; in other words, participating services had the opportunity to implement the InnoWell Platform in varying ways depending on what best suited their service and their service user base. All service users aged ≥14 years who presented to a participating service were eligible to use the InnoWell Platform as part of their care. Upon creating an account in the InnoWell Platform, users were presented with a Terms of Use agreement informing them that their deidentified data collected by the InnoWell Platform would be used for research purposes unless they opt out [32]. In 2007, the Australian National Statement of Ethical Conduct in Human Research was updated to enable HRECs to approve an opt-out consent for low-risk clinical research [33]. Opt-out approaches require researchers to ensure that participants are aware of their potential inclusion in research and given the opportunity to indicate that they do not want their data used in this way (ie, thus opting out). Importantly, service users were able to update their data sharing permissions in the InnoWell Platform at any time, enabling them to, for example, withdraw from the clinical trial should they wish to do so. As the InnoWell Platform is an exempt medical device, it is not required to be included in the Australian Register of Therapeutic Goods and can be used by service users regardless of whether they opt out of the clinical trial.

In addition to the sharing of data for research purposes, service users are also informed that their data will be shared with the participating service at which they are receiving care. This information is made explicit to the user at the point at which the InnoWell Platform is originally explained to them by a staff member from the participating service using a simple script delivered via telephone, email, chat, or in person, and in the subsequent email invitation they receive inviting them to use the InnoWell Platform. In addition, this information is documented in the InnoWell Privacy Policy, which is available on the landing page, before creating an account and providing any personal information.

The primary objective of the above-mentioned health services trial is to validate an automated decision protocol for allocating care based on severity and persistence of mental health symptoms (ie, based on the clinical stage) [34] relative to the clinical stage as allocated by health professionals, multidisciplinary teams via consensus, and an expert clinical reference group [20]. The protocol aims to automate critical decision points to triage service users based on clinical stage. Specifically, the first decision determines whether there is clear evidence of at least one full-threshold, major discrete disorder, or a persistent and recurrent syndrome or syndromes, whereas the second decision aims to determine whether the syndrome is nonspecific or attenuated. By identifying the clinical stage and level of need, service users can then be automatically triaged to self-management strategies including self-directed apps and e-tools, ambulatory care services, or acute or specialized services at the point of service entry. As detailed by Sawrikar et al [35], clinical staging avoids the fail first approach typified by stepped care in favor of allocating the Right care, First time. The clinical trial allows for the collection of deidentified data for research purposes, and cross-sectional findings from the participating services are provided in Multimedia Appendix 1 [36-40]; however, the findings from the clinical trial are beyond the scope of this paper and will be published separately.

An automated suicide escalation protocol is embedded within the InnoWell Platform, designed to identify service users reporting suicidal thoughts or behaviors and, importantly, suggest pathways to appropriate and timely care [21]. Specifically, when a service user expresses suicidal thoughts or behaviors on the InnoWell Platform’s multidimensional assessment, they are immediately provided with a pop-up message containing contact information for relevant 24-hour crisis support services (ie, Lifeline and Kids Helpline). In addition, to enable a rapid clinical response, a notification is also sent to the clinical service when the degree of risk meets a predetermined severity threshold established by the service for their service user base.

Impact Evaluation

Traditional clinical science approaches to the development and implementation of interventions rely on a lengthy linear process, from basic science through to effectiveness trials and dissemination [41], which can result in delays of up to 17 years for research translation into clinical practice [42]. Given the rapid advances that occur in the technology sector, this is not a practical approach to the evaluation of HITs or technology-enabled models of care. To that end, Mohr et al [29] highlight the benefits of evaluating the challenges and successes associated with tools such as the InnoWell Platform in targeted clinical settings, such as through the Project Synergy participating services, to iteratively refine and optimize usability, acceptability, and effectiveness to ensure that it meets identified clinical objectives. Therefore, as detailed previously by LaMonica et al [10], we used a mixed methods impact evaluation study, including web-based surveys, semistructured interviews, and workshops, to identify potential barriers to and facilitators of implementation and to evaluate the impact of the InnoWell Platform on the participating mental health services over time, including (1) digital literacy and competence of the service staff in relation to the implementation of the HIT in the service; (2) changes in the service in association with the implementation of the HIT-enabled solution; and (3) the quality, usability, and acceptability of the solution. All participating staff involved in the implementation of the InnoWell Platform at each of the respective services, including health professionals, service managers, and administrators, were invited to participate in the impact evaluation study. Where staff from the services’ funding or governing bodies (ie, service providers or primary health networks) were associated with implementation, these staff were also invited to participate and were required to provide written informed consent. The impact evaluation data were collected at baseline and every 3 months thereafter for the duration of the implementation, the length of which varied based on agreements made with participating services. Importantly, the use of mixed methods allowed the service staff to provide feedback in a manner that suited their work schedule (ie, web-based helpline operating on a 16-hour schedule vs traditional face-to-face counseling services operating during normal business hours), thus ensuring both breadth (via surveys) and depth (via the semistructured interviews and workshop) of evaluation from a broad range of participants.

Participatory Systems Modeling

As referenced previously, a system dynamics model was developed using a participatory approach that involved approximately 50 local stakeholders, including representatives from health and social policy agencies, nongovernment organizations, primary care providers, emergency services, research institutions, community groups, and, importantly, people with lived experience of suicide [43]. Over a series of 3 workshops, stakeholders were asked to draw on the deep tacit knowledge and diverse perspectives of these system actors to map the local health system and the role that social determinants play in psychological distress and suicidal behaviors. These workshops focused on prioritization of the key outcomes of interest for the model, the mapping of pathways and drivers of those outcomes, and the prioritization of interventions to be included in the model. A full description of this process, the model, and its results can be found in the study by Occhipinti et al [43].

Results

Co-design

The results of most of our team’s co-design work have been published previously and will therefore not be presented in detail [27,28,36-38,44]. However, a summary of the key insights derived from the participatory design workshops is provided in Table 2, including findings regarding the (1) support for HITs as part of clinical care, (2) critical factors to consider in relation to the design and content of HITs, and (3) potential for incongruity between health professional and service manager attitudes and beliefs toward HITs during co-design relative to implementation.

In addition to the participatory design outcomes, service mapping also proved an effective way to engage key stakeholders in reflecting on existing service and system pathways and, in turn, identifying current barriers to and facilitators of quality mental health care. As highlighted in LaMonica et al [27], by working collaboratively with participating services, service mapping highlighted key gaps in care that might be improved through the implementation of the InnoWell Platform, while also identifying aspects of care that could be improved through changes to specific service-level processes (ie, intake). In this way, the service mapping process prevented a one-size-fits-all approach to service reform, ensuring that specific contextual factors impacting the quality of care at each service were considered, thus optimizing the service delivery. Each participating service was able to determine how best to integrate the InnoWell Platform as part of standard care as it pertained to the respective services. For example, at Open Arms and most headspace services, service users were invited to complete the InnoWell Platform’s multidimensional assessment before their first appointment, whereas at Kildare Road Medical Centre (KRMC), a general practice, service users were only invited to the InnoWell Platform after having an appointment with a mental health nurse who determined whether the digital tool was appropriate for their individual care needs. Importantly, our research shows that the process of mapping technology-enabled service models can lead to improvements in service delivery pathways, both in relation to and independent of the technology [28].

Implementation and the Clinical Trial

Overview

To address known barriers, as well as those that might be specific to individual services, the R&D team developed a protocol for the implementation of digital solutions into mental health services, including digital tools and technology-enabled models of care [31]. The protocol supports active collaboration between the researchers, health services, and service providers and users and is intended to promote mitigation strategies that might enable the successful translation of new innovations into standard clinical practice. This not only includes a strong emphasis on co-design but also acknowledges that a well-designed technology product is not sufficient to drive improved outcomes or mental health reform. Therefore, the protocol highlights the need to (1) upskill the mental health workforce regarding the evidence-based digital, clinical, service, and safety elements essential to quality mental health care, as well as how digital solutions can support the delivery of such care; (2) provide on-the-ground support to both service users and staff in the early stages of implementation to troubleshoot any issues with the technology and to help interpret and report clinical data in a meaningful way; and (3) refine iteratively the digital solution, as well as the technology-enabled model of care, in response to service-level changes in outcomes and processes or user feedback on the quality, usability, and acceptability of the solution. In relation to the latter, it is crucial to continue to refine digital solutions to ensure they become an integral part of standard care, enabling improved outcomes on key service performance indicators. Concurrently, minimizing increased burden on service staff associated with the implementation is challenging. However, striking this balance may afford the best opportunity for the sustainability and scalability of the solution.

The naturalistic, quasiexperimental clinical trial to validate the InnoWell Platform is in progress. At the time of this publication, 1644 service users aged ≥14 years consented to share their data for research purposes. Cross-sectional data from the initial multidimensional assessment for each participating service are provided in Multimedia Appendix 1. Further analysis of the data collected by the InnoWell Platform will be conducted at the service-specific level and is beyond the scope of this paper. Further, the data to validate the automated decision protocol for allocating care based on clinical stage will be analyzed and published separately. Importantly, however, the automated suicide escalation protocol embedded within the InnoWell Platform has been shown to effectively identify service users requiring urgent risk assessment. For example, 22.6% (24/106) of the young people using the InnoWell Platform as part of their care at a headspace service in the regional North Coast NSW Primary Health Network footprint reported high levels of suicidal thoughts and behaviors (STBs), thus triggering an immediate notification to the service. A similar proportion (18/62, 29%) of users from Open Arms underwent a rapid risk assessment after the service received a notification of moderate to high levels of STB as self-reported via the assessments in the InnoWell Platform. Although the management of such notifications can be challenging, services recognized the need for this functionality, as it fast-tracks care.

Innovative Models of Enhanced Primary Care

The implementation science strategies resulted in an innovative model of enhanced primary care, best suited to local implementation. Specifically, KRMC is a large Western Sydney family practice with 26 general practitioners (GPs) seeing an average of 25 to 30 service users per day. As part of their engagement in Project Synergy, KRMC used a mental health nurse to support their GPs with assessment and triage, treatment planning and intervention, and case review of those who presented with symptoms suggesting mental ill-health. The mental health nurse was a critical facilitator of the co-designed technology-enabled service delivery model at KRMC. For example, the application of the InnoWell Platform’s suicide escalation protocol [21] identified 13% (17/131) of those service users as having STBs, thus enabling the mental health nurse to triage these individuals rapidly to appropriate care. The mental health nurse also reviewed the initial assessment results in collaboration with each service user to guide a discussion about available care options, considering service user preferences. Importantly for KRMC, the technology-enabled model of care, as led by the mental health nurse, resulted in expanded treatment options, including the availability of brief psychological therapies conducted by the mental health nurse as well as the apps and e-tools embedded within the InnoWell Platform. Furthermore, the mental health nurse was also able to use the multidimensional assessment to draft a mental health care plan for later review by the GP. Consequently, this reduced the assessment time required by the GP, allowing them to focus instead on delivering value-added care to more service users. In light of the success of this pilot study, KRMC sought funding to support the ongoing implementation of this innovative technology-enabled, mental health nurse-led service delivery model.

An Innovative Model of Enhanced Community-Based Assessment and Triage

As previously reported [39], the R&D team partnered with Neami National (who deliver the central community-based intake service of the NSW North Coast Primary Health Network—Connect to Wellbeing North Coast) to co-design and implement a new technology-enabled assessment and triage system for the range of services offered [39]. Specifically, the InnoWell Platform’s multidimensional assessments were used to (1) identify service users with risk factors and symptoms or impairment (stage 1a) who were immediately triaged to self-management including self-directed and clinician-supported apps and e-tools, (2) assist those with attenuated or subthreshold major disorders (stage 1b+) to clinical care within ambulatory care services including general practice or another primary care service, and (3) provide immediate clinical assessment for those with a risk of self-harm and determine the need for acute care or hospitalization. This model could serve as a prototype for a direct-to-service user preclinic triage system (Figure 6). Such a system is scalable and directly addresses the Productivity Commission’s call to develop a sustainable national digital platform to facilitate assessment and referral processes that match levels of mental health care more directly with service users’ actual needs [3].

‎View this figure

Impact Evaluation

As shown in Table 3, data from the impact evaluation results highlight consistent agreement regarding the potential for HITs to improve outcomes for service users and mental health services alike. However, participants, including health professionals, service managers, and administrators, indicated that technologies are generally not well integrated into current service delivery models. Service readiness for change (eg, existing technology infrastructure and the digital literacy of staff and service users) was noted to be a potential barrier to successful implementation, with less than half of respondents indicating that their service was ready to implement new technologies to enhance mental health care. Furthermore, there was considerable variability among clinical staff as to whether it was their responsibility to recommend technology as part of standard care.

The impact evaluation findings were used to iteratively refine the technology-enabled model of care throughout the implementation of the InnoWell Platform at each participating service. For example, based on findings from the evaluation activities, services modified (1) the point in the care journey at which the InnoWell Platform was offered to participants, (2) the required qualifications for the person presenting the platform to service users for the first time (eg, mental health nurse, counselor, or clinical psychologist rather than administrative staff), (3) the technology available to health professionals at the service to be able to best use the InnoWell Platform actively during sessions, and (4) processes by which the data collected by the InnoWell Platform were used to inform care. Furthermore, the findings were fed back to service leadership to help find ways to better support health professionals and administrative staff to use the InnoWell Platform effectively and efficiently as part of standard care. Finally, feedback regarding the usability and acceptability of the InnoWell Platform as well as identified gaps or limitations in its features and functionality were fed back to InnoWell as part of the internal R&D cycle to inform the development or redevelopment and refinement of the technology.

Participatory Systems Modeling of the North Coast Primary Health Network Mental Health Services

The R&D team was engaged to work with the North Coast Primary Health Network to model the mental health services available across the region and test the relative impacts of a range of interventions on mental health and prevention of suicide. This work was critical for Project Synergy, as one of the primary interventions of interest was technology-enabled care coordination. The capacity to compare the impact of this system-level intervention with other clinical and health system alternatives was a major advance. The model was validated against historic time series data (2011-2017), and then projected population-level trajectories of suicidal behavior in the region using a 20-year time horizon (2021-2041) were derived. Against the baseline trajectory, implementation of technology-enabled care coordination was projected to decrease suicide deaths by 5.6% (95% CI 4.8%-6.5%) [43]. The model also found that the most effective combination of interventions to reduce suicidal behavior was supporting social connectedness, technology-enabled coordinated care, postsuicide attempt assertive aftercare, reductions in childhood adversity, and increasing youth employment [43]. This combination of interventions was projected to reduce self-harm hospitalizations (indicative of suicide attempts) by 28.5% (95% CI 26.3%-30.8%) and suicide deaths by 29.3% (95% CI 27.1%-31.5%) among a youth population (aged 15-24 years). Introducing additional interventions beyond this best performing suite of interventions produced only marginal improvement. That is, “more is not necessarily better” [43]. Further analysis demonstrated that technology-enabled care coordination led to greater reductions in suicide deaths, the total number of self-harm hospitalizations or mental health–related ED hospitalizations, and the prevalence of high psychological distress in the population, when accompanied by increases in service capacity growth by 20% (including standard telehealth practices [48]). Findings from these simulation models indicate the significant promise offered by HITs when effective implementation is achieved.

Tools to Support the Adoption of Digital Solutions in Clinical Practice

As noted previously, service and health professional readiness for change are 2 key factors that impact the implementation of digital solutions. To help build confidence in the use of apps and e-tools in practice, the R&D team adapted the original Mobile App Rating Scale (MARS; a reliable and internationally recognized app rating system [49]) to ensure it is appropriate for health-related apps as well as e-tools (eg, websites and web-based courses). It is now known as the A-MARS [50]. A complimentary quality assurance protocol was also developed to guide health professionals in the evaluation of the quality and safety of health-related apps and e-tools to determine their appropriateness for use in clinical practice. This approach includes (1) a broad exploration for available apps and e-tools; (2) shortlisting of those health-related apps and e-tools deemed to match the needs of the service users, health professionals, or service; (3) evaluation using the newly developed A-MARS; and (4) review of the ratings compared with service-specific criteria to determine appropriateness for recommendation.

Given the consistent concerns regarding data security and privacy highlighted in our co-design work, the R&D team also developed a privacy risk assessment tool to assess whether current apps and e-tools are meeting privacy standards [46]. Our review revealed consistently poor readability, resulting in marked limitations in the transparency of the information presented. This, in turn, can undermine a service user’s trust in the privacy of their personal and health information. To assist health professionals and service providers in understanding potential privacy risks, we developed easy-to-use guidelines for their consideration before promoting individual apps and e-tools as part of care [46]. It is our recommendation that these guidelines be adopted to ensure that HITs are used to their full potential to maximize service user health outcomes while minimizing risk and users are informed of privacy and security considerations to be able to make educated decisions as to how they would like to share their personal and health information [46].

Discussion

Principal Findings

Recent enhanced funding for mental health care in Australia has been focused largely on increasing access to traditional clinic-based psychological therapy [51], with little to no emphasis on delivering quality care matched to the level of need, coordinating systems of care, or accounting for the unequal distribution of resources across urban, regional, and rural communities. Although the increased investment in mental health included in the Australian Government Federal Budget of May 2021 signals recognition of the need for system reform, there is a relative lack of funding dedicated to service innovation, health systems research, or evaluation [52]. These omissions are particularly concerning given evidence highlighting that technology-enabled care coordination has the potential to differentially improve health outcomes (ie, reductions in deaths by suicide, mental health–related hospitalizations, and rates of high psychological distress in the community) and assist with demand management over and above any impact owing to a 20% increase in service capacity and standard telehealth services [48]. Furthermore, technology-enabled coordinated care in conjunction with programs targeting the social determinants of health would appear to offer a good return on investment, particularly in regions with limited health resources [43].

With the stage set by the population-level modeling outcomes, Project Synergy has now delivered critical outcomes regarding the practicalities of designing, implementing, and evaluating digital mental health solutions (Figure 7 [4,6,15,19-20,22,27-28,31,36-38,43,53-59]). These outputs are critical pieces needed to inform delivery of effective technology-enabled and coordinated mental health care, as called for by the Productivity Commission [3] and the World Economic Forum [60]. We now have the methodologies, tools, and essential insights to build robust, dynamic, sustainable, scalable, and coordinated systems of mental health care. This has the capacity to firmly establish Australia as a leader in the delivery of digital mental health care.

‎View this figure

On the basis of the findings from Project Synergy, it is critical to ensure that all digital solutions and innovative models of technology-enhanced systems of care be developed in collaboration with the intended target audiences, including those with lived experience of mental ill-health. However, it is important to recognize that the co-design process must also consider the needs and time demands on health professionals. Historically, health professionals have been reluctant to drive service reforms, frequently arguing instead simply for more of the same (ie, endless therapy) [61]. Similarly, our results highlight the potential for health professionals to be reluctant to implement digital tools as part of standard practice. To that end, research indicates that health professionals are most likely to accept changes that they are prepared for and that they value, especially those that are likely to result in direct benefits to service users [62]. Furthermore, it is essential to provide health professionals with the necessary education and training to support their understanding and use of digital health solutions. It needs to be very clear that the primary goal is to deliver high-quality and person-centered care, not simply operational or financial efficiency. Until digital health care is included as part of the core curriculum for standard health professional education, services are also likely to benefit from including a digital navigator on their care team. These new personnel can evaluate the suitability of available digital solutions for the service, troubleshoot technical difficulties, and assist in interpreting and reporting data collected via digital tools in a meaningful way to inform care and self-management strategies [63].

As existing digital infrastructure is so poorly integrated, health professionals are frequently required to enter the same data across multiple systems, including client management software, electronic medical records, minimum data sets, and personal health records. Only by reducing this administrative burden are health professionals likely to engage with new digital solutions; therefore, there is an urgent need to ensure open application program interfaces are built into all digital solutions to enable interoperability or the transfer of relevant personal and health information between various systems. Such integration can inform shared decision-making, track progress over time in coordination with and between care teams and supportive services, and empower the service user, along with families and carers, to be informed and active partners in care.

Systematically collecting multidimensional assessment data through self-report and clinical digital tools, as well as wearables and smartphone sensors, will provide the opportunity to examine service user outcomes, health professional practices, and service-level performance in relation to key indicators, including safety, acceptability, efficiency, accessibility, effectiveness, coordination, and workforce capability [64]. More specifically, assessment data enables services to understand the clinical complexity of those service users presenting for care. This can ensure that the best care options are offered immediately, as well as promoting coordination of care with other relevant specialist or support services. This type of data may also help services provide necessary training opportunities for their workforce. Furthermore, routine outcome monitoring enables data-informed decisions regarding the effectiveness or lack thereof of treatment plans to facilitate necessary changes or referrals to specialist care as needed. Finally, data linkage at the state and national levels, using data from health records (ie, Medicare Benefit Scheme and Pharmaceutical Benefits Scheme), provides the necessary information for analysis of the costs and effectiveness of various models of care. Taken together, these methods of evaluation offer an opportunity for accountability in service delivery.

Having successfully delivered the necessary building blocks to overhaul the mental health care system through Project Synergy, it is now time to translate these insights into action. With the ever-increasing demand for more personalized and higher-quality services [65], it is clear that simply increasing investment into various clinical services delivered in isolation will not be enough to improve the outcomes for the whole system; instead, new models of integrated care, and the digital infrastructure to support them, is needed to accompany these new clinical investments. Project Synergy supports calls for increased use of digital technologies in mental health care; however, it also highlights that the way these technologies are deployed really matters. Specifically, dynamic systems modeling shows that when digital technologies are limited to standard telehealth practices (ie, by extending existing services on the web), without changing the underlying model of care, the impact is low [48]. Although standard telehealth will now be implemented more widely, following the COVID-19 pandemic, to date, little effort has been made to use these technologies in ways that promote multidimensional team-based care or maximize the benefits that these technologies can really provide. By capitalizing on digital tools and investing in digital infrastructure, we now have the opportunity to recalibrate the whole mental health system, resulting in a greater impact on outcomes as opposed to those expected by simply improving the capacity across individual components of the existing mental health system.

Conclusions

Although the Australian Government continues to invest in more clinical services, the well-documented failings of the mental health system (notably restricted and delayed access to quality care, siloed services, inadequate use of routine outcome monitoring, and care plans that do not match a service user’s level of need), may be left unaddressed. Project Synergy highlights the power of more innovative, digitally enhanced systems to increase the efficiencies of health care systems by addressing demand management, reducing delays in access to appropriate care, enabling deployment of early intervention strategies, and rapidly assisting at-risk service users to access acute and specialist care. System-level innovation can be achieved via the engagement of the appropriate service leadership and active promotion of the interoperability of the new technologies.