Background: Globally, there are fundamental shortcomings in mental health care systems, including restricted access, siloed services, interventions that are poorly matched to service users’ needs, underuse of personal outcome monitoring to track progress, exclusion of family and carers, and suboptimal experiences of care. Health information technologies (HITs) hold great potential to improve these aspects that underpin the enhanced quality of mental health care.
Objective: Project Synergy aimed to co-design, implement, and evaluate novel HITs, as exemplified by the InnoWell Platform, to work with standard health care organizations. The goals were to deliver improved outcomes for specific populations under focus and support organizations to enact significant system-level reforms.
Methods: Participating health care organizations included the following: Open Arms–Veterans & Families Counselling (in Sydney and Lismore, New South Wales [NSW]); NSW North Coast headspace centers for youth (Port Macquarie, Coffs Harbour, Grafton, Lismore, and Tweed Heads); the Butterfly Foundation’s National Helpline for eating disorders; Kildare Road Medical Centre for enhanced primary care; and Connect to Wellbeing North Coast NSW (administered by Neami National), for population-based intake and assessment. Service users, families and carers, health professionals, and administrators of services across Australia were actively engaged in the configuration of the InnoWell Platform to meet service needs, identify barriers to and facilitators of quality mental health care, and highlight potentially the best points in the service pathway to integrate the InnoWell Platform. The locally configured InnoWell Platform was then implemented within the respective services. A mixed methods approach, including surveys, semistructured interviews, and workshops, was used to evaluate the impact of the InnoWell Platform. A participatory systems modeling approach involving co-design with local stakeholders was also undertaken to simulate the likely impact of the platform in combination with other services being considered for implementation within the North Coast Primary Health Network to explore resulting impacts on mental health outcomes, including suicide prevention.
Results: Despite overwhelming support for integrating digital health solutions into mental health service settings and promising impacts of the platform simulated under idealized implementation conditions, our results emphasized that successful implementation is dependent on health professional and service readiness for change, leadership at the local service level, the appropriateness and responsiveness of the technology for the target end users, and, critically, funding models being available to support implementation. The key places of interoperability of digital solutions and a willingness to use technology to coordinate health care system use were also highlighted.
Conclusions: Although the COVID-19 pandemic has resulted in the widespread acceptance of very basic digital health solutions, Project Synergy highlights the critical need to support equity of access to HITs, provide funding for digital infrastructure and digital mental health care, and actively promote the use of technology-enabled, coordinated systems of care.
Mental Health Service Reform
In 2014, the National Mental Health Commission invited the Young and Well Cooperative Research Centre (CRC) to provide advice about digital mental health solutions for young people . Following interviews with mental health and industry leaders, the Young and Well CRC promoted the view that there was an urgent need to broaden research and development (R&D) focus from standalone mental health apps, digital health records, and other simple technologies (Digital Health 1.0) to highly coordinated integration of health information technologies (HITs) within more effective health care delivery systems (HIT Digital Health 2.0). That is, new systems of care would integrate both conventional and novel in-clinic and web-based mental health services, thereby broadening the rapid provision of scalable forms of high-quality care.
As a result, the National Mental Health Commission recommended that HITs be considered a cornerstone of mental health service reform in Australia. This would require that the mental health system be overhauled to embed digital health within conventional in-clinic services . More recently, this recommendation was echoed by the Australian Productivity Commission review of mental health. It calls for increased use of technology through a national digital platform, emphasizing initial assessment and referral processes, as well as increasing service user access to a broader range of interventions and supports [ ]. However, there is still a considerable need for increased practical knowledge as to how to best incorporate HITs into clinical services, as well as how to capitalize on the aggregate information available from HITs that could be used to drive more effective coordination of care, enhanced allocation of clinical resources, faster identification of risk, and greater emphasis on achieving a higher proportion of improved health, social, and economic outcomes.
Over the past 20 years, our research group has actively advocated for the use of technology to drive enhanced mental health care, and broader system reform, in Australia [- ]. Although there is a growing evidence base supporting the use of digital technologies, the integration of digital solutions in mental health services remains relatively rare. As an example, Titov et al [ ] have reported success in integrating internet-delivered cognitive behavioral therapy as part of clinical care with a mental health professional in 5 clinics internationally. Unfortunately, far more studies highlight barriers to successful implementations of digital health solutions [ - ], and further research is required to identify the most effective ways in which to design and implement evidence-based HITs as part of clinical care at scale.
The Australian Government Department of Health commissioned Project Synergy in 2014 to be conducted by the Young and Well CRC in partnership with the University of Sydney’s Brain and Mind Centre. However, following the completion of the Young and Well CRC’s work in 2016, InnoWell, a joint venture enterprise between the University of Sydney and PwC (Australia), was established to deliver the remainder of the work. The overarching objective of Project Synergy is to explore how best to use such new HITs to drive mental health service reform toward delivering better outcomes for people experiencing mental ill-health .
Because of its scale, Project Synergy had 2 phases. Phase I (2014-2016; Aus $5.5 million [US $3.97 million]) investigated the use of digital (including smartphone) technologies to transform the provision of mental health services for Australian young people through co-designing, building, implementing, and evaluating prototypes with representative end user populations. These included young people along with their supportive others (eg, family, carers, and friends), health professionals, and service providers . Four research studies were conducted on prototypical software and concepts that included the following: (1) a university-based health and well-being prototype [ ]; (2) a synergized web-based system including a triage e-tool, health and well-being e-tool, and a Mental Health eClinic with a multidimensional web-based assessment, shared mental health dashboard, and video visit capabilities [ ]; (3) wireframes for a tiered suicide risk management protocol [ ]; (4) functionality for trusted supportive others [ - ]; (5) and a proof of concept study for implementing technologies into primary youth mental health services [ ]. The details of these preliminary studies have been published previously [ ].
The remainder of this paper focuses on phase II (2017-2021; Aus $30 million [US $21.63 million]), which concluded in June 2021. In phase II, prototypes from phase I were iterated into a beta version of a single web-based platform named the InnoWell Platform and implemented into both conventional in-clinic and web-based primary mental health services. The fundamental aim was articulated as delivering the right level of care to individuals upon their first presentation to service (Right care, First time) as underpinned by clinical stage, level of clinical need, and personal choice, with the aim of preventing progression to more serious complex and severe forms of illness.
As detailed by Davenport et al , the InnoWell Platform collects both multidimensional self-report and informant (eg, supportive others and health professionals) information via questionnaires to capture a holistic understanding of a service user’s clinical presentation, level of need, and personal preferences, to then monitor their progress over time. Specifically, survey data were collected across a range of biopsychosocial health domains, such as psychological distress, depressed mood, sleep-wake cycle, physical health, and social and occupational functioning. Further objective behavioral data can also be collected via third-party integrations (eg, Fitbit fitness trackers [Google LLC]). As shown in , the multidimensional assessment results are presented back to service users in a dashboard color coded with a traffic light system to ease interpretability (ie, green, yellow, orange, and red are used to denote increasing degrees of symptom severity or level of impairment for each health domain). Service users are also provided with a description of each health domain and a brief explanation of the result they received ( ).
The assessment results can then be used collaboratively by the service user and their health professional to ensure both parties are informed and actively engaged in treatment decisions in a responsive way. This is designed to occur not only at the outset of the care journey but also on an ongoing basis, as service users and clinicians jointly track progress over time. Importantly, as shown in, service users can also access recommended nonclinical care options (eg, fact sheets, apps, e-tools, and other web-based systems) to support their mental health and well-being proactively (eg, before receiving and in conjunction with the clinical care received through the service).
The InnoWell Platform was co-designed and implemented in diverse service settings across different age groups and populations. The Project Synergy effectiveness trials aimed to deliver critical insights about how HITs, exemplified by the InnoWell Platform, can be optimally configured and implemented as part of standard care to meet the needs of the populations under focus, their immediate care team (health professionals and service providers), and supportive others. Furthermore, a mixed methods approach was established to evaluate the web-based platform and the impact of its implementation (where available) in the individual services to iteratively refine the InnoWell Platform and the technology-enabled service model into which it was embedded.
As shown in[ - , - , - , ], this paper seeks to summarize the methodologies used in phase II of Project Synergy and highlight their associated research outcomes. Specifically, we describe the process by which the original beta version of the InnoWell Platform was (1) iteratively co-designed, developed, and refined through participatory design workshops with individuals with lived experience of mental ill-health, health professionals, and service administrators to create service-specific configurations of the digital platform; (2) tested by end users to determine acceptance for each participating service or service user group; (3) implemented within a technology-enabled service model that had been co-designed with service staff through service mapping; and (4) evaluated to determine its impact at the level of the service user, health professional, and service. We provide an overview of the key outcomes from this research, highlight tools that can be used to drive the uptake of HITs by both service users and health professionals, and provide examples of how smart, digital technologies can drive innovation in models of care at the service and system level to better address demand management, early intervention, and rapid assessment and triage of at-risk users to specialized care.
Overview of the Project Synergy Research Streams
Phase II of Project Synergy aimed to conduct 4 rolling research streams to evaluate the engagement with and efficiency and effectiveness of the InnoWell Platform. The research streams focused on four populations across the life span, namely, veterans and their families, young children and their families, young people, and older adults. Importantly, through the internal R&D process, these rolling research streams enabled the iterative development and redevelopment, evaluation, and refinement of essential functionality within the InnoWell Platform (eg, assessment, dashboard, algorithms, real time data tracking, aggregate service indicator dashboard, immediate staged care recommendations, and service pathways) over time. One or more research studies were planned in relation to each research stream, in collaboration with relevant mental health services when feasible or with specific participant groups (ie, young children and families). In relation to the former, potential participating services were identified based on the alignment between their service user group or groups and the research streams. After agreeing to participate in Project Synergy, a research agreement was formalized between the University of Sydney and the participating service. As described in this Methods section, the methodology that underpins Project Synergy’s research trials includes co-design activities (ie, participatory design workshops, user acceptance testing, and service mapping sessions), implementation science, and a mixed methods evaluation of the impact of the InnoWell Platform and the associated technology-enabled model of care at the level of the individual, health professional, and service based on key performance indicators.
Participating Services and Service User Groups
The InnoWell Platform was co-designed with and tested by the target end users, including individuals seeking or engaged in mental health care. Digital health solutions are widely recognized as a potential means by which to address issues of demand management in mental health globally, and, to a degree, the COVID-19 pandemic fast-tracked the uptake of technology as part of mental health care. However, it is critical to recognize that factors such as age, technical skills, literacy, and health literacy affect the ways in which individuals engage with technology . Despite the proliferation of digital health solutions, such as apps, in recent years, little attention has been paid to how these tools will be used by individuals from diverse populations [ ]. In addition, only 30% of apps developed by established digital health companies have tested their apps with individuals with clinical conditions [ ]. To ensure that the InnoWell Platform and technology-enabled model of care met the needs of the target end users, it was crucial to ensure that the research studies were conducted with individuals engaged in clinical care. Participating services and the service users to whom they provide care are presented in .
Participating mental health services and their respective service user groups.
- Open Arms–Veterans & Families Counselling Sydney and Lismore, New South Wales (NSW), providing care to current and ex-serving military personnel and their families
- NSW North Coast headspace centers (Port Macquarie, Coffs Harbour, Grafton, Lismore, and Tweed Heads) providing mental health care to young people in regional NSW aged 12-25 years
- The Butterfly Foundation’s National Helpline supporting those with eating disorders and negative body image via telephone, email, and web-based chat
- Kildare Road Medical Centre—a large general practice in Blacktown, NSW, with a socioeconomically and culturally diverse service user population, including a large number of Aboriginal and Torres Strait Islander service users
- Connect to Wellbeing North Coast NSW (administered by Neami National)—a community-based primary intake and referral service located within the footprint of the North Coast NSW Primary Health Network. Within the Australian context, Primary Health Networks are operated by not-for-profit companies and are responsible for commissioning and coordinating health services to meet the needs of the people in a designated region
Additional co-design and preliminary evaluation research studies were conducted with specific service user groups not affiliated with a mental health service including older adults (aged ≥50 years) and children (aged ≥8 years) and families to inform future iterations of the InnoWell Platform. These participants were recruited through active research clinics at the University of Sydney’s Brain and Mind Centre as well as via nongovernmental and private organizations associated with the Brain and Mind Centre. The co-design studies were advertised via posters and postcards at the advertising site as well as via their websites and social media pages. Importantly, although a diverse grouping of services and service users participated in Project Synergy, further investigation is still required to determine the usability, acceptability, and relevance of the InnoWell Platform for individuals who are from a culturally and linguistically diverse background, identify as Aboriginal and Torres Strait Islanders, or identify as having a disability.
Finally, approximately 50 local stakeholders from the North Coast NSW Primary Health Network, including representatives from health and social policy agencies, nongovernment organizations, primary care providers, emergency services, research institutions, community groups, and, importantly, people with lived experience of suicide, contributed to the development of a system dynamics model. Using a broad systems perspective, stakeholders identified key outcomes of interest and subsequently provided feedback to iteratively refine the model.
This research required multiple ethics approvals by various human research ethics committees (HRECs) owing to the diverse organizational structures governing each of the participating services. The clinical trial allowing users of the InnoWell Platform to share their deidentified data for research purposes was approved by the Northern Sydney Local Health District HREC (reference HREC/17/HAWKE/480) and was registered with the Australian New Zealand Clinical Trial Registry (ACTRN12618001676202). Separate ethics were sought for the co-design and impact evaluation research for each participating service. The governing bodies of some mental health services required applications to be submitted through their own internal HRECs (ie, the Department of Defence and Veterans’ Affairs HREC Projects 056-18 and 017-17), whereas others preferred that required applications were submitted through the University of Sydney HREC (Projects 2017/944, 2018/849, 2018/041, 2018/962, 2019/172, 2019/683, 2019/767, 2020/141, and 2020/246). The development of the system dynamics model did not require ethics approval as it involved secondary analysis of a broad range of aggregate data sets and synthesis of existing research. Stakeholders were invited to contribute their expertise to this model but were not the subject of the research activity.outlines the diverse research methodologies that were used as part of Project Synergy, underpinning all aspects of the project from co-design and development of the InnoWell Platform through to impact evaluation and dissemination of outcomes.
An R&D cycle, described by Davenport et al , was established and used to explicitly position end users as empowered participants in all stages of the research—from planning and design through to implementation and evaluation. Central to the R&D cycle are participatory design workshops to discover, prototype, and evaluate potential design solutions through the use of prompted discussion, review of working prototypes (wireframes) and existing InnoWell Platform functionality, personas (ie, a fictitious potential user for whom the product is being designed), and user journeys. Direct engagement with potential end users as well as other key stakeholders (ie, representatives from funding agencies) through targeted participatory workshops is essential to ensure the development and build of acceptable, usable, and scalable digital tools. The primary objective of the workshops was to co-design and configure the InnoWell Platform for each participating service or user group to ensure that it met the needs of the target end users, including people with lived experience accessing care and their supportive others as well as health professionals, service managers, and administrators.
Workshop agendas were tailored to each service by the research management team, comprising researchers, health professionals, and service administrators, to determine how best to discover how the InnoWell Platform might enhance standard care. Areas of focus included increased accessibility to care, waitlist management, risk identification, coordination of care with other services, and the potential for digital tools to support self-management and well-being. Participants worked together to create mockups of the technology, including the landing page, assessments, and dashboard, highlighting key features and functions that they thought would address identified pain points for the service and service users. Only after having captured participants’ unbiased ideas were they then provided with paper wireframes or screenshots of the InnoWell Platform on which to provide specific feedback on prototyped or existing features and functionality. The final outcome of the workshops was a service-specific configuration of the InnoWell Platform.
Importantly, the service-specific version of the InnoWell Platform was then tested by end users in one-to-one, face-to-face, 90-minute user testing sessions. Each session was facilitated by a member of the research team, and a scribe was present to take notes to capture all feedback accurately. The aim of the user testing sessions was to (1) learn if participants were able to navigate the InnoWell Platform intuitively, (2) identify any features of the platform that the user would want to see changed before implementation of the platform into the service (ie, showstoppers), (3) gain insight into participants’ experience of using the platform, (4) identify possibilities for improving the platform to better meet user needs and wants, and (5) learn how best to implement the platform into the participating service. Testers were observed interacting with the InnoWell Platform and asked to think aloud when exploring its features, providing both positive and negative feedback about the user experience. In addition to freely exploring the features and functionality of the InnoWell Platform, testers were also asked to complete specific scenarios relevant to their role (eg, service user, health professional, and service manager). For example, a health professional might be asked to try entering data into the InnoWell Platform about a service user and then to comment on their experience (ie, ease of completing the action and gaps in the functionality). The feedback was integrated back into the iterative R&D cycle to inform the ongoing refinement of the product by our collaborators at InnoWell. Furthermore, user testing was also used to achieve acceptance of the technology, signally that it was ready for implementation within the service.
Finally, in parallel with the participatory design workshops and user testing sessions, our team also used service mapping, a process mapping methodology, to understand how best to integrate the InnoWell Platform within participating mental health services. As published in LaMonica et al , we have used the term service mapping to refer to the structured approach to understanding the needs, existing processes, gaps in care, and performance of a mental health service mapped against an individual’s journey through the service. More specifically, participants worked collaboratively with the researchers to first map the current service user journey through the participating service on a whiteboard or butcher’s paper. The gaps or limitations of the current care pathway were reviewed relative to the key features of high-quality mental health care to identify areas for improvement or reform. The existing map was then revised to include technical element supported by the InnoWell Platform (eg, web-based initial assessment and clinical and nonclinical care options). The outcome was a co-designed technology-enabled model of care highlighting the intersection and impact of the InnoWell Platform at key points in care (eg, referral, assessment, treatment planning, and routine outcome monitoring).
summarizes the research aims and co-design activities for each of our participating services and service user groups.
|Population||Service partner||Research aim||Co-design activities|
|Current and ex-serving Australian Defence Force personnel and their families||Open Arms–Veterans & Families Counselling Sydney and Lismore, NSWa||Co-design, develop, implement, and evaluate the InnoWell Platform to achieve improved outcomes|
|Young people aged 12-25 years residing in the NSW North Coast||NSW North Coast headspace centers (Port Macquarie, Coffs Harbour, Grafton, Lismore, and Tweed Heads)||Enhance quality and timeliness of mental health services for NSW North Coast youth|
|Australians with eating disorders and body image issues||The Butterfly Foundation’s National Helpline||Evaluate, prototype, and configure a digital system for a nonconventional (web-based, phone, and email) service providing support for eating disorders and body image issues through the use of co-design processes|
|NSW North Coast residents||Connect to Wellbeing North Coast NSW (administered by Neami National)||Configure and implement the InnoWell Platform to support community-based primary intake and referrals across the North Coast NSW Primary Health Network|
|Service users in a community general practice setting||Kildare Road Medical Centre, Blacktown, NSW||Configure and implement the InnoWell Platform to support Kildare Road Medical Centre’s delivery of mental health care|
|Older adults (aged ≥50 years)||N/Ad||Co-design and configure the InnoWell Platform to evaluate the accessibility, engagement, and appropriateness of the technology for an older age group|
|Younger children and families||N/A||Co-design a configuration of the InnoWell Platform that is appropriate for younger children and their families|
aNSW: New South Wales.
bA total of 9 cofacilitated participatory design workshops, including 2 service mapping workshops, were conducted in 2017 with current and former military personnel and their families as well as Open Arms’ health professionals, service managers, and administrators from Sydney, Canberra, Maitland, Singleton, and Port Stephens . Importantly, the outcomes from this collaboration served as the basis for the configuration of the InnoWell Platform with supplementary participatory workshops being conducted in 2018 specifically as part of Project Synergy.
cFurther information informing the service-specific configurations and user acceptance of the InnoWell Platform was captured during routine project management meetings with each of the participating services.
dN/A: not applicable.
Implementation and the Clinical Trial
The effectiveness of the InnoWell Platform is being assessed in a naturalistic, prospective, quasiexperimental (ie, uncontrolled, nonrandomized) clinical trial. It is important to note that this methodology is rapidly gaining recognition as a practical and efficient method of assessing change over time in users under ecologically valid conditions (ie, real-world conditions of constantly changing apps and technologies) [, ]. Importantly, by conducting group comparisons based on clinical stage, demographics, and locality, it is possible to infer strong evidence for the usefulness (or lack thereof) of the technology-enabled blended model of care.
The primary objective of the above-mentioned health services trial is to validate an automated decision protocol for allocating care based on severity and persistence of mental health symptoms (ie, based on the clinical stage)  relative to the clinical stage as allocated by health professionals, multidisciplinary teams via consensus, and an expert clinical reference group [ ]. The protocol aims to automate critical decision points to triage service users based on clinical stage. Specifically, the first decision determines whether there is clear evidence of at least one full-threshold, major discrete disorder, or a persistent and recurrent syndrome or syndromes, whereas the second decision aims to determine whether the syndrome is nonspecific or attenuated. By identifying the clinical stage and level of need, service users can then be automatically triaged to self-management strategies including self-directed apps and e-tools, ambulatory care services, or acute or specialized services at the point of service entry. As detailed by Sawrikar et al [ ], clinical staging avoids the fail first approach typified by stepped care in favor of allocating the Right care, First time. The clinical trial allows for the collection of deidentified data for research purposes, and cross-sectional findings from the participating services are provided in [ - ]; however, the findings from the clinical trial are beyond the scope of this paper and will be published separately.
An automated suicide escalation protocol is embedded within the InnoWell Platform, designed to identify service users reporting suicidal thoughts or behaviors and, importantly, suggest pathways to appropriate and timely care . Specifically, when a service user expresses suicidal thoughts or behaviors on the InnoWell Platform’s multidimensional assessment, they are immediately provided with a pop-up message containing contact information for relevant 24-hour crisis support services (ie, Lifeline and Kids Helpline). In addition, to enable a rapid clinical response, a notification is also sent to the clinical service when the degree of risk meets a predetermined severity threshold established by the service for their service user base.
Traditional clinical science approaches to the development and implementation of interventions rely on a lengthy linear process, from basic science through to effectiveness trials and dissemination , which can result in delays of up to 17 years for research translation into clinical practice [ ]. Given the rapid advances that occur in the technology sector, this is not a practical approach to the evaluation of HITs or technology-enabled models of care. To that end, Mohr et al [ ] highlight the benefits of evaluating the challenges and successes associated with tools such as the InnoWell Platform in targeted clinical settings, such as through the Project Synergy participating services, to iteratively refine and optimize usability, acceptability, and effectiveness to ensure that it meets identified clinical objectives. Therefore, as detailed previously by LaMonica et al [ ], we used a mixed methods impact evaluation study, including web-based surveys, semistructured interviews, and workshops, to identify potential barriers to and facilitators of implementation and to evaluate the impact of the InnoWell Platform on the participating mental health services over time, including (1) digital literacy and competence of the service staff in relation to the implementation of the HIT in the service; (2) changes in the service in association with the implementation of the HIT-enabled solution; and (3) the quality, usability, and acceptability of the solution. All participating staff involved in the implementation of the InnoWell Platform at each of the respective services, including health professionals, service managers, and administrators, were invited to participate in the impact evaluation study. Where staff from the services’ funding or governing bodies (ie, service providers or primary health networks) were associated with implementation, these staff were also invited to participate and were required to provide written informed consent. The impact evaluation data were collected at baseline and every 3 months thereafter for the duration of the implementation, the length of which varied based on agreements made with participating services. Importantly, the use of mixed methods allowed the service staff to provide feedback in a manner that suited their work schedule (ie, web-based helpline operating on a 16-hour schedule vs traditional face-to-face counseling services operating during normal business hours), thus ensuring both breadth (via surveys) and depth (via the semistructured interviews and workshop) of evaluation from a broad range of participants.
Participatory Systems Modeling
As referenced previously, a system dynamics model was developed using a participatory approach that involved approximately 50 local stakeholders, including representatives from health and social policy agencies, nongovernment organizations, primary care providers, emergency services, research institutions, community groups, and, importantly, people with lived experience of suicide . Over a series of 3 workshops, stakeholders were asked to draw on the deep tacit knowledge and diverse perspectives of these system actors to map the local health system and the role that social determinants play in psychological distress and suicidal behaviors. These workshops focused on prioritization of the key outcomes of interest for the model, the mapping of pathways and drivers of those outcomes, and the prioritization of interventions to be included in the model. A full description of this process, the model, and its results can be found in the study by Occhipinti et al [ ].
The results of most of our team’s co-design work have been published previously and will therefore not be presented in detail [, , - , ]. However, a summary of the key insights derived from the participatory design workshops is provided in , including findings regarding the (1) support for HITs as part of clinical care, (2) critical factors to consider in relation to the design and content of HITs, and (3) potential for incongruity between health professional and service manager attitudes and beliefs toward HITs during co-design relative to implementation.
In addition to the participatory design outcomes, service mapping also proved an effective way to engage key stakeholders in reflecting on existing service and system pathways and, in turn, identifying current barriers to and facilitators of quality mental health care. As highlighted in LaMonica et al , by working collaboratively with participating services, service mapping highlighted key gaps in care that might be improved through the implementation of the InnoWell Platform, while also identifying aspects of care that could be improved through changes to specific service-level processes (ie, intake). In this way, the service mapping process prevented a one-size-fits-all approach to service reform, ensuring that specific contextual factors impacting the quality of care at each service were considered, thus optimizing the service delivery. Each participating service was able to determine how best to integrate the InnoWell Platform as part of standard care as it pertained to the respective services. For example, at Open Arms and most headspace services, service users were invited to complete the InnoWell Platform’s multidimensional assessment before their first appointment, whereas at Kildare Road Medical Centre (KRMC), a general practice, service users were only invited to the InnoWell Platform after having an appointment with a mental health nurse who determined whether the digital tool was appropriate for their individual care needs. Importantly, our research shows that the process of mapping technology-enabled service models can lead to improvements in service delivery pathways, both in relation to and independent of the technology [ ].
|Key insights||Specific findings|
|Overwhelming support for integrating HITsa into mental health care across populations and service settings [, , , - , ].|
|Service users and carers further emphasized the importance of being provided reputable (academic, government, or nonprofit mental health organization) care options and information .|
|Concerns with data privacy and security of personal and health information were prevalent across all service user groups [, , ]; however, health professionals questioned whether all users would be wary of security risks.|
|There is a gap in what is expressed during the co-design process and actual implementation; often, clinicians are very active and willing supporters during the co-design process, however not in practice .|
aHIT: health information technology.
bPHN: Primary Health Network.
Implementation and the Clinical Trial
To address known barriers, as well as those that might be specific to individual services, the R&D team developed a protocol for the implementation of digital solutions into mental health services, including digital tools and technology-enabled models of care . The protocol supports active collaboration between the researchers, health services, and service providers and users and is intended to promote mitigation strategies that might enable the successful translation of new innovations into standard clinical practice. This not only includes a strong emphasis on co-design but also acknowledges that a well-designed technology product is not sufficient to drive improved outcomes or mental health reform. Therefore, the protocol highlights the need to (1) upskill the mental health workforce regarding the evidence-based digital, clinical, service, and safety elements essential to quality mental health care, as well as how digital solutions can support the delivery of such care; (2) provide on-the-ground support to both service users and staff in the early stages of implementation to troubleshoot any issues with the technology and to help interpret and report clinical data in a meaningful way; and (3) refine iteratively the digital solution, as well as the technology-enabled model of care, in response to service-level changes in outcomes and processes or user feedback on the quality, usability, and acceptability of the solution. In relation to the latter, it is crucial to continue to refine digital solutions to ensure they become an integral part of standard care, enabling improved outcomes on key service performance indicators. Concurrently, minimizing increased burden on service staff associated with the implementation is challenging. However, striking this balance may afford the best opportunity for the sustainability and scalability of the solution.
The naturalistic, quasiexperimental clinical trial to validate the InnoWell Platform is in progress. At the time of this publication, 1644 service users aged ≥14 years consented to share their data for research purposes. Cross-sectional data from the initial multidimensional assessment for each participating service are provided in. Further analysis of the data collected by the InnoWell Platform will be conducted at the service-specific level and is beyond the scope of this paper. Further, the data to validate the automated decision protocol for allocating care based on clinical stage will be analyzed and published separately. Importantly, however, the automated suicide escalation protocol embedded within the InnoWell Platform has been shown to effectively identify service users requiring urgent risk assessment. For example, 22.6% (24/106) of the young people using the InnoWell Platform as part of their care at a headspace service in the regional North Coast NSW Primary Health Network footprint reported high levels of suicidal thoughts and behaviors (STBs), thus triggering an immediate notification to the service. A similar proportion (18/62, 29%) of users from Open Arms underwent a rapid risk assessment after the service received a notification of moderate to high levels of STB as self-reported via the assessments in the InnoWell Platform. Although the management of such notifications can be challenging, services recognized the need for this functionality, as it fast-tracks care.
Innovative Models of Enhanced Primary Care
The implementation science strategies resulted in an innovative model of enhanced primary care, best suited to local implementation. Specifically, KRMC is a large Western Sydney family practice with 26 general practitioners (GPs) seeing an average of 25 to 30 service users per day. As part of their engagement in Project Synergy, KRMC used a mental health nurse to support their GPs with assessment and triage, treatment planning and intervention, and case review of those who presented with symptoms suggesting mental ill-health. The mental health nurse was a critical facilitator of the co-designed technology-enabled service delivery model at KRMC. For example, the application of the InnoWell Platform’s suicide escalation protocol  identified 13% (17/131) of those service users as having STBs, thus enabling the mental health nurse to triage these individuals rapidly to appropriate care. The mental health nurse also reviewed the initial assessment results in collaboration with each service user to guide a discussion about available care options, considering service user preferences. Importantly for KRMC, the technology-enabled model of care, as led by the mental health nurse, resulted in expanded treatment options, including the availability of brief psychological therapies conducted by the mental health nurse as well as the apps and e-tools embedded within the InnoWell Platform. Furthermore, the mental health nurse was also able to use the multidimensional assessment to draft a mental health care plan for later review by the GP. Consequently, this reduced the assessment time required by the GP, allowing them to focus instead on delivering value-added care to more service users. In light of the success of this pilot study, KRMC sought funding to support the ongoing implementation of this innovative technology-enabled, mental health nurse-led service delivery model.
An Innovative Model of Enhanced Community-Based Assessment and Triage
As previously reported , the R&D team partnered with Neami National (who deliver the central community-based intake service of the NSW North Coast Primary Health Network—Connect to Wellbeing North Coast) to co-design and implement a new technology-enabled assessment and triage system for the range of services offered [ ]. Specifically, the InnoWell Platform’s multidimensional assessments were used to (1) identify service users with risk factors and symptoms or impairment (stage 1a) who were immediately triaged to self-management including self-directed and clinician-supported apps and e-tools, (2) assist those with attenuated or subthreshold major disorders (stage 1b+) to clinical care within ambulatory care services including general practice or another primary care service, and (3) provide immediate clinical assessment for those with a risk of self-harm and determine the need for acute care or hospitalization. This model could serve as a prototype for a direct-to-service user preclinic triage system ( ). Such a system is scalable and directly addresses the Productivity Commission’s call to develop a sustainable national digital platform to facilitate assessment and referral processes that match levels of mental health care more directly with service users’ actual needs [ ].
As shown in, data from the impact evaluation results highlight consistent agreement regarding the potential for HITs to improve outcomes for service users and mental health services alike. However, participants, including health professionals, service managers, and administrators, indicated that technologies are generally not well integrated into current service delivery models. Service readiness for change (eg, existing technology infrastructure and the digital literacy of staff and service users) was noted to be a potential barrier to successful implementation, with less than half of respondents indicating that their service was ready to implement new technologies to enhance mental health care. Furthermore, there was considerable variability among clinical staff as to whether it was their responsibility to recommend technology as part of standard care.
The impact evaluation findings were used to iteratively refine the technology-enabled model of care throughout the implementation of the InnoWell Platform at each participating service. For example, based on findings from the evaluation activities, services modified (1) the point in the care journey at which the InnoWell Platform was offered to participants, (2) the required qualifications for the person presenting the platform to service users for the first time (eg, mental health nurse, counselor, or clinical psychologist rather than administrative staff), (3) the technology available to health professionals at the service to be able to best use the InnoWell Platform actively during sessions, and (4) processes by which the data collected by the InnoWell Platform were used to inform care. Furthermore, the findings were fed back to service leadership to help find ways to better support health professionals and administrative staff to use the InnoWell Platform effectively and efficiently as part of standard care. Finally, feedback regarding the usability and acceptability of the InnoWell Platform as well as identified gaps or limitations in its features and functionality were fed back to InnoWell as part of the internal R&D cycle to inform the development or redevelopment and refinement of the technology.
|Key outcome||Specific findings|
|Staff across multiple service settings consistently support the use of technology as part of their work; however, they also list digital literacy of both service users and health professionals as well as service readiness for change as potential barriers to widespread adoption .|
|There must be organizational leadership (PHNa-level decision-making) as well as a local champion at the service level to support a successful implementation .|
|Health professionals are often confident about the effectiveness of their current service models (ie, business as usual) and express reluctance to change their usual practices .|
aPHN: Primary Health Network.
Participatory Systems Modeling of the North Coast Primary Health Network Mental Health Services
The R&D team was engaged to work with the North Coast Primary Health Network to model the mental health services available across the region and test the relative impacts of a range of interventions on mental health and prevention of suicide. This work was critical for Project Synergy, as one of the primary interventions of interest was technology-enabled care coordination. The capacity to compare the impact of this system-level intervention with other clinical and health system alternatives was a major advance. The model was validated against historic time series data (2011-2017), and then projected population-level trajectories of suicidal behavior in the region using a 20-year time horizon (2021-2041) were derived. Against the baseline trajectory, implementation of technology-enabled care coordination was projected to decrease suicide deaths by 5.6% (95% CI 4.8%-6.5%) . The model also found that the most effective combination of interventions to reduce suicidal behavior was supporting social connectedness, technology-enabled coordinated care, postsuicide attempt assertive aftercare, reductions in childhood adversity, and increasing youth employment [ ]. This combination of interventions was projected to reduce self-harm hospitalizations (indicative of suicide attempts) by 28.5% (95% CI 26.3%-30.8%) and suicide deaths by 29.3% (95% CI 27.1%-31.5%) among a youth population (aged 15-24 years). Introducing additional interventions beyond this best performing suite of interventions produced only marginal improvement. That is, “more is not necessarily better” [ ]. Further analysis demonstrated that technology-enabled care coordination led to greater reductions in suicide deaths, the total number of self-harm hospitalizations or mental health–related ED hospitalizations, and the prevalence of high psychological distress in the population, when accompanied by increases in service capacity growth by 20% (including standard telehealth practices [ ]). Findings from these simulation models indicate the significant promise offered by HITs when effective implementation is achieved.
Tools to Support the Adoption of Digital Solutions in Clinical Practice
As noted previously, service and health professional readiness for change are 2 key factors that impact the implementation of digital solutions. To help build confidence in the use of apps and e-tools in practice, the R&D team adapted the original Mobile App Rating Scale (MARS; a reliable and internationally recognized app rating system ) to ensure it is appropriate for health-related apps as well as e-tools (eg, websites and web-based courses). It is now known as the A-MARS [ ]. A complimentary quality assurance protocol was also developed to guide health professionals in the evaluation of the quality and safety of health-related apps and e-tools to determine their appropriateness for use in clinical practice. This approach includes (1) a broad exploration for available apps and e-tools; (2) shortlisting of those health-related apps and e-tools deemed to match the needs of the service users, health professionals, or service; (3) evaluation using the newly developed A-MARS; and (4) review of the ratings compared with service-specific criteria to determine appropriateness for recommendation.
Given the consistent concerns regarding data security and privacy highlighted in our co-design work, the R&D team also developed a privacy risk assessment tool to assess whether current apps and e-tools are meeting privacy standards . Our review revealed consistently poor readability, resulting in marked limitations in the transparency of the information presented. This, in turn, can undermine a service user’s trust in the privacy of their personal and health information. To assist health professionals and service providers in understanding potential privacy risks, we developed easy-to-use guidelines for their consideration before promoting individual apps and e-tools as part of care [ ]. It is our recommendation that these guidelines be adopted to ensure that HITs are used to their full potential to maximize service user health outcomes while minimizing risk and users are informed of privacy and security considerations to be able to make educated decisions as to how they would like to share their personal and health information [ ].
Recent enhanced funding for mental health care in Australia has been focused largely on increasing access to traditional clinic-based psychological therapy , with little to no emphasis on delivering quality care matched to the level of need, coordinating systems of care, or accounting for the unequal distribution of resources across urban, regional, and rural communities. Although the increased investment in mental health included in the Australian Government Federal Budget of May 2021 signals recognition of the need for system reform, there is a relative lack of funding dedicated to service innovation, health systems research, or evaluation [ ]. These omissions are particularly concerning given evidence highlighting that technology-enabled care coordination has the potential to differentially improve health outcomes (ie, reductions in deaths by suicide, mental health–related hospitalizations, and rates of high psychological distress in the community) and assist with demand management over and above any impact owing to a 20% increase in service capacity and standard telehealth services [ ]. Furthermore, technology-enabled coordinated care in conjunction with programs targeting the social determinants of health would appear to offer a good return on investment, particularly in regions with limited health resources [ ].
With the stage set by the population-level modeling outcomes, Project Synergy has now delivered critical outcomes regarding the practicalities of designing, implementing, and evaluating digital mental health solutions ([ , , , - , , - , , - , , - ]). These outputs are critical pieces needed to inform delivery of effective technology-enabled and coordinated mental health care, as called for by the Productivity Commission [ ] and the World Economic Forum [ ]. We now have the methodologies, tools, and essential insights to build robust, dynamic, sustainable, scalable, and coordinated systems of mental health care. This has the capacity to firmly establish Australia as a leader in the delivery of digital mental health care.
On the basis of the findings from Project Synergy, it is critical to ensure that all digital solutions and innovative models of technology-enhanced systems of care be developed in collaboration with the intended target audiences, including those with lived experience of mental ill-health. However, it is important to recognize that the co-design process must also consider the needs and time demands on health professionals. Historically, health professionals have been reluctant to drive service reforms, frequently arguing instead simply for more of the same (ie, endless therapy) . Similarly, our results highlight the potential for health professionals to be reluctant to implement digital tools as part of standard practice. To that end, research indicates that health professionals are most likely to accept changes that they are prepared for and that they value, especially those that are likely to result in direct benefits to service users [ ]. Furthermore, it is essential to provide health professionals with the necessary education and training to support their understanding and use of digital health solutions. It needs to be very clear that the primary goal is to deliver high-quality and person-centered care, not simply operational or financial efficiency. Until digital health care is included as part of the core curriculum for standard health professional education, services are also likely to benefit from including a digital navigator on their care team. These new personnel can evaluate the suitability of available digital solutions for the service, troubleshoot technical difficulties, and assist in interpreting and reporting data collected via digital tools in a meaningful way to inform care and self-management strategies [ ].
As existing digital infrastructure is so poorly integrated, health professionals are frequently required to enter the same data across multiple systems, including client management software, electronic medical records, minimum data sets, and personal health records. Only by reducing this administrative burden are health professionals likely to engage with new digital solutions; therefore, there is an urgent need to ensure open application program interfaces are built into all digital solutions to enable interoperability or the transfer of relevant personal and health information between various systems. Such integration can inform shared decision-making, track progress over time in coordination with and between care teams and supportive services, and empower the service user, along with families and carers, to be informed and active partners in care.
Systematically collecting multidimensional assessment data through self-report and clinical digital tools, as well as wearables and smartphone sensors, will provide the opportunity to examine service user outcomes, health professional practices, and service-level performance in relation to key indicators, including safety, acceptability, efficiency, accessibility, effectiveness, coordination, and workforce capability . More specifically, assessment data enables services to understand the clinical complexity of those service users presenting for care. This can ensure that the best care options are offered immediately, as well as promoting coordination of care with other relevant specialist or support services. This type of data may also help services provide necessary training opportunities for their workforce. Furthermore, routine outcome monitoring enables data-informed decisions regarding the effectiveness or lack thereof of treatment plans to facilitate necessary changes or referrals to specialist care as needed. Finally, data linkage at the state and national levels, using data from health records (ie, Medicare Benefit Scheme and Pharmaceutical Benefits Scheme), provides the necessary information for analysis of the costs and effectiveness of various models of care. Taken together, these methods of evaluation offer an opportunity for accountability in service delivery.
Having successfully delivered the necessary building blocks to overhaul the mental health care system through Project Synergy, it is now time to translate these insights into action. With the ever-increasing demand for more personalized and higher-quality services , it is clear that simply increasing investment into various clinical services delivered in isolation will not be enough to improve the outcomes for the whole system; instead, new models of integrated care, and the digital infrastructure to support them, is needed to accompany these new clinical investments. Project Synergy supports calls for increased use of digital technologies in mental health care; however, it also highlights that the way these technologies are deployed really matters. Specifically, dynamic systems modeling shows that when digital technologies are limited to standard telehealth practices (ie, by extending existing services on the web), without changing the underlying model of care, the impact is low [ ]. Although standard telehealth will now be implemented more widely, following the COVID-19 pandemic, to date, little effort has been made to use these technologies in ways that promote multidimensional team-based care or maximize the benefits that these technologies can really provide. By capitalizing on digital tools and investing in digital infrastructure, we now have the opportunity to recalibrate the whole mental health system, resulting in a greater impact on outcomes as opposed to those expected by simply improving the capacity across individual components of the existing mental health system.
Although the Australian Government continues to invest in more clinical services, the well-documented failings of the mental health system (notably restricted and delayed access to quality care, siloed services, inadequate use of routine outcome monitoring, and care plans that do not match a service user’s level of need), may be left unaddressed. Project Synergy highlights the power of more innovative, digitally enhanced systems to increase the efficiencies of health care systems by addressing demand management, reducing delays in access to appropriate care, enabling deployment of early intervention strategies, and rapidly assisting at-risk service users to access acute and specialist care. System-level innovation can be achieved via the engagement of the appropriate service leadership and active promotion of the interoperability of the new technologies.
This research was conducted on behalf of the Australian Government Department of Health as part of Project Synergy (2017-2021). InnoWell was formed by the University of Sydney and PricewaterhouseCoopers (PwC) Australia to deliver the Aus $30 million (US $21.63 million) Australian Government–funded Project Synergy.
Project Synergy was made possible by many researchers over the lifetime of the project. Sincerest thanks go to Amelia English, William Capon, Dominic FitzGerald, Tanya Jackson, Cristina Ricci, Anna E Roberts, Vilas Sawrikar, Joan Torony, and Hannah Yee for their contributions. The authors would also like to thank participants from Open Arms–Veterans & Families Counselling (Sydney and Lismore), the North Coast Primary Health Network, the headspace centers (Coffs Harbour, Grafton, Lismore, Port Macquarie, and Tweed Heads), Neami National, Connect to Wellbeing, Butterfly Foundation’s National Helpline 1800 ED HOPE, and Kildare Road Medical Centre as well as older adults, young children, and family members for their involvement in this research. In addition, the authors acknowledge and thank the Project Synergy Lived Experience Advisory Function led by Ms Sue Muller for their contribution to this research. Finally, the authors thank their collaborators at InnoWell, including former CEOs Ayse Burke and Will Fellowes, as well as the Board of Directors chaired by Michael Ronaldson with Deputy Alan Griffin, Kristin Stubbins from PwC (Australia), and Professor Laurent Rivory from the University of Sydney, for supporting the research objectives of Project Synergy.
Authors IBH, TAD, SC, and JB were integral in securing funding to support the study. The studies encompassed by Project Synergy were designed by HML, FI, JO, TAD, SC, AM, LOP, LW, SCR, MD, ES, and TO with subsequent contributions by GYL, SP, SH, and EMS. Data analyses were conducted by HML, FI, AM, SCR, and MD, with contributions from VWSC and oversight and advice from IBH. The system dynamics modeling was undertaken by Dr Adam Skinner (Research Fellow, Brain and Mind Centre, University of Sydney) with contributions from authors JO and FI. All authors contributed to and have approved the final manuscript.
Conflicts of Interest
IBH is codirector of Health and Policy at the Brain and Mind Centre (BMC) at the University of Sydney. The BMC operates an early-intervention youth services at Camperdown under contract to headspace. He is the Chief Scientific Advisor to, and a 5% equity shareholder in, InnoWell Pty Ltd. InnoWell was formed by the University of Sydney (45% equity) and PwC (Australia; 45% equity) to deliver the Aus $30 million (US $21.63 million) Australian Government–funded Project Synergy (2017-2020; a 3-year program for the transformation of mental health services) and to lead the transformation of mental health services internationally through the use of innovative technologies. JO is both Head of Systems Modelling, Simulation & Data Science at the University of Sydney’s BMC and Managing Director of Computer Simulation & Advanced Research Technologies. TAD is now Director (Research and Evaluation) of the Design and Strategy Division, Australian Digital Health Agency. EMS is Principal Research Fellow at the BMC, University of Sydney. She is Discipline Leader of Adult Mental Health, School of Medicine, University of Notre Dame, and a consultant psychiatrist. She was the Medical Director at the Young Adult Mental Health Unit, St Vincent’s Hospital Darlinghurst, until January 2021. She has received honoraria for educational seminars related to the clinical management of depressive disorders supported by Servier and Eli-Lilly pharmaceuticals. She has participated in a national advisory board for the antidepressant compound Pristiq, manufactured by Pfizer. She was the National Coordinator of an antidepressant trial sponsored by Servier. The source of funding does not entail any potential conflict of interest for the other members of the Project Synergy Research and Development Team.
Data from the InnoWell Platform for Participating Services.DOCX File , 469 KB
- Burns J. NMHC Advice-on-innovative-technologies-in-eMental-health. National Mental Health Commission Review. 2015. URL: https://www.researchgate.net/publication/283010660_NMHC_Advice-on-Innovative-Technologies-in-eMental-Health [accessed 2022-02-20]
- National Review of Mental Health Programmes and Services – Preliminary analysis. National Mental Health Commission. URL: http://mhaustralia.org/fact-sheets/national-review-mental-health-programmes-and-services-preliminary-analysis [accessed 2022-02-20]
- Productivity Commission Report no.95. Canberra, ACT, Australia: Productivity Commission; 2020.
- Hickie IB, Davenport TA, Burns JM, Milton AC, Ospina-Pinillos L, Whittle L, et al. Project Synergy: co-designing technology-enabled solutions for Australian mental health services reform. Med J Aust 2019 Oct;211 Suppl 7:S3-39. [CrossRef] [Medline]
- Groom G, Hickie I, Davenport T. 'Out of Hospital, Out of Mind!' : A Report Detailing Mental Health Services in Australia in 2002 and Community Priorities for National Mental Health Policy for 2003-2008. Canberra: Mental Health Council of Australia; 2003.
- Hickie IB. Moving beyond stepped care to staged care using a novel, technology-enabled care model for youth mental health. Med J Aust 2019 Nov 03;211(9):404-405. [CrossRef] [Medline]
- Burns JM, Birrell E, Bismark M, Pirkis J, Davenport TA, Hickie IB, et al. The role of technology in Australian youth mental health reform. Aust Health Rev 2016 Nov;40(5):584-590. [CrossRef] [Medline]
- Rosenberg S, Hickie IB, Mendoza J. National mental health reform: less talk, more action. Med J Aust 2009 Feb 16;190(4):193-195. [CrossRef] [Medline]
- Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, et al. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv 2018 Sep;13:108-115 [FREE Full text] [CrossRef] [Medline]
- LaMonica HM, Milton A, Braunstein K, Rowe SC, Ottavio A, Jackson T, et al. Health information technology-enabled solutions for Australian mental health services reform: impact evaluation. JMIR Form Res 2020 Mar (forthcoming). [CrossRef]
- Granja C, Janssen W, Johansen MA. Factors determining the success and failure of eHealth interventions: systematic review of the literature. J Med Internet Res 2018 May 01;20(5):e10235 [FREE Full text] [CrossRef] [Medline]
- Vis C, Mol M, Kleiboer A, Bührmann L, Finch T, Smit J, et al. Improving implementation of eMental health for mood disorders in routine practice: systematic review of barriers and facilitating factors. JMIR Ment Health 2018 Mar 16;5(1):e20 [FREE Full text] [CrossRef] [Medline]
- Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 2017 Nov 01;19(11):e367 [FREE Full text] [CrossRef] [Medline]
- Davenport T, Ospina-Pinillos L, Ricci C, Milton A, Burns J, Hickie I. Study 1: Fit Uni Life to thrive: an online health and wellbeing prototype for young people. Med J Aust 2019;211(7 Suppl).
- Ospina-Pinillos L, Davenport TA, Ricci CS, Milton AC, Scott EM, Hickie IB. Developing a mental health eClinic to improve access to and quality of mental health care for young people: using participatory design as research methodologies. J Med Internet Res 2018 May 28;20(5):e188 [FREE Full text] [CrossRef] [Medline]
- Davenport T, Milton A, Whittle L, McLoughlin L, Ospina-Pinillos L, Mendoza J, et al. Study 3: co-designing, user testing and evaluating digital suicide prevention functionality. Med J Aust 2019;7:S24-S29.
- Davenport TA, Milton AC, Ospina-Pinillos L, Whittle L, Ricci CS, Burns JM, et al. Project Synergy research and development cycle: iterative processes of participatory design, user testing, implementation and feasibility testing. Med J Aust 2019 Oct;211 Suppl 7.
- Davenport T, Whittle L, Ospina-Pinillos L, Milton A, Burns J, Hickie I. Evaluating engagement with a prototypic online platform to improve the mental health and wellbeing of young people living in disadvantaged communities. Med J Aust 2019(Suppl (7)):S17-S23.
- Cross S, Piper S, Davenport T, Milton A, Iorfino F, Rici C, et al. Implementation study of a prototypic e-clinic being integrated into youth mental health services: staff experiences and reported service quality improvements. Med J Aust 2019;7:S30-S36.
- Davenport TA, LaMonica HM, Whittle L, English A, Iorfino F, Cross S, et al. Validation of the InnoWell platform: protocol for a clinical trial. JMIR Res Protoc 2019 May 31;8(5):e13955 [FREE Full text] [CrossRef] [Medline]
- Iorfino F, Davenport TA, Ospina-Pinillos L, Hermens DF, Cross S, Burns J, et al. Using new and emerging technologies to identify and respond to suicidality among help-seeking young people: a cross-sectional study. J Med Internet Res 2017 Jul 12;19(7):e247 [FREE Full text] [CrossRef]
- Cross S, Davenport T, Scott E. A service delivery model to support highly personalised and measurement-based care in youth mental health. Med J Aust 2019;211(9):S42-S46.
- Hickie IB. The role of new technologies in monitoring the evolution of psychopathology and providing measurement-based care in young people. World Psychiatry 2020 Feb;19(1):38-39 [FREE Full text] [CrossRef] [Medline]
- Hall AK, Bernhardt JM, Dodd V, Vollrath MW. The digital health divide: evaluating online health information access and use among older adults. Health Educ Behav 2015 Apr 25;42(2):202-209 [FREE Full text] [CrossRef] [Medline]
- Sarkar U, Gourley GI, Lyles CR, Tieu L, Clarity C, Newmark L, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 2016 Dec;31(12):1417-1426 [FREE Full text] [CrossRef] [Medline]
- Winkle BV, Carpenter N, Moscucci M. Why aren't our digital solutions working for everyone? AMA J Ethics 2017 Nov 01;19(11):1116-1124 [FREE Full text] [CrossRef] [Medline]
- LaMonica HM, Davenport TA, Ottavio A, Rowe SC, Cross SP, Iorfino F, et al. Optimising the integration of technology-enabled solutions to enhance primary mental health care: a service mapping study. BMC Health Serv Res 2021 Jan 15;21(1):68 [FREE Full text] [CrossRef] [Medline]
- LaMonica HM, Davenport TA, Burns J, Cross S, Hodson S, Veitch J, et al. Technology-enabled mental health service reform for open arms - veterans and families counselling: participatory design study. JMIR Form Res 2019 Sep 19;3(3):e13662 [FREE Full text] [CrossRef] [Medline]
- Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res 2015 Jul 08;17(7):e166 [FREE Full text] [CrossRef] [Medline]
- Fleming TM, de Beurs D, Khazaal Y, Gaggioli A, Riva G, Botella C, et al. Maximizing the impact of e-therapy and serious gaming: time for a paradigm shift. Front Psychiatry 2016;7:65 [FREE Full text] [CrossRef] [Medline]
- LaMonica HM, Davenport TA, Braunstein K, Ottavio A, Piper S, Martin C, et al. Technology-enabled person-centered mental health services reform: strategy for implementation science. JMIR Ment Health 2019 Sep 19;6(9):e14719 [FREE Full text] [CrossRef] [Medline]
- The Australian Research Council and Universities Australia. National Statement on Ethical Conduct in Human Research 2007 (Updated 2018). Canberra: National Health and Medical Research Council; 2007.
- Iorfino F, Cheng VW, Cross SP, Yee HF, Davenport TA, Scott EM, et al. Right care, first time: developing a theory-based automated protocol to help clinically stage young people based on severity and persistence of mental illness. Front Public Health 2021;9:621862 [FREE Full text] [CrossRef] [Medline]
- Sawrikar V, Stewart E, LaMonica HM, Iorfino F, Davenport TA, Cross S, et al. Using staged care to provide "right care first time" to people with common affective disorders. Psychiatr Serv 2021 Jun;72(6):691-703. [CrossRef] [Medline]
- Milton AC, Hambleton A, Dowling M, Roberts AE, Davenport T, Hickie I. Technology-enabled reform in a nontraditional mental health service for eating disorders: participatory design study. J Med Internet Res 2021 Feb 16;23(2):e19532 [FREE Full text] [CrossRef] [Medline]
- LaMonica HM, Davenport TA, Roberts AE, Hickie IB. Understanding technology preferences and requirements for health information technologies designed to improve and maintain the mental health and well-being of older adults: participatory design study. JMIR Aging 2021 Jan 06;4(1):e21461 [FREE Full text] [CrossRef] [Medline]
- LaMonica HM, Roberts AE, Davenport TA, Hickie IB. Evaluation of the usability and acceptability of the InnoWell platform as rated by older adults: survey study. JMIR Aging 2021 Apr 21;4(2):e25928 [FREE Full text] [CrossRef] [Medline]
- Davenport TA, LaMonica HM, Rowe S, Sturgess J, Scott EM, Lee G, et al. Innovative preclinic triage system to guide Australians to the right mental health care first time. Aust Health Rev 2021;45(5):578. [CrossRef]
- Weathers FW, Litz BT, Keane TM, Palmieri PA, Marx BP, Schnurr PP. The PTSD Checklist for DSM-5 (PCL-5). US Department of Veterans Affairs. 2013. URL: https://www.ptsd.va.gov/professional/assessment/adult-sr/ptsd-checklist.asp [accessed 2022-02-11]
- Onken LS, Carroll KM, Shoham V, Cuthbert BN, Riddle M. Reenvisioning clinical science: unifying the discipline to improve the public health. Clin Psychol Sci 2014 Jan 01;2(1):22-34 [FREE Full text] [CrossRef] [Medline]
- Balas E, Boren S. Managing clinical knowledge for health care improvement. Yearb Med Inform 2018 Mar 07;09(01):65-70. [CrossRef]
- Occhipinti J, Skinner A, Iorfino F, Lawson K, Sturgess J, Burgess W, et al. Reducing youth suicide: systems modelling and simulation to guide targeted investments across the determinants. BMC Med 2021 Mar 12;19(1):61 [FREE Full text] [CrossRef] [Medline]
- Rowe SC, Davenport TA, Easton MA, Jackson TA, Melsness J, Ottavio A, et al. Co-designing the InnoWell Platform to deliver the right mental health care first time to regional youth. Aust J Rural Health 2020 Apr;28(2):190-194. [CrossRef] [Medline]
- National community consultation program final report. InnoWell Lived Experience Advisory Function Working Group. 2020. URL: https://www.innowell.org/reports/project-synergy/nccp-june2020-final.pdf [accessed 2022-02-20]
- LaMonica HM, Roberts AE, Jackson TA, Rowe S, Hickie IB. The InnoWell Platform: Challenges in integrating and implementing health information technologies into youth regional mental health care. 2021 Presented at: Digital Health & Informatics Network; Feb 10, 2021; Virtual conference.
- Iorfino F, Occhipinti J, Skinner A, Davenport T, Rowe S, Prodan A, et al. The impact of technology-enabled care coordination in a complex mental health system: a local system dynamics model. J Med Internet Res 2021 Jun 30;23(6):e25331 [FREE Full text] [CrossRef] [Medline]
- Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015 Mar 11;3(1):e27 [FREE Full text] [CrossRef] [Medline]
- Roberts A, Davenport T, Wong T, Moon C, Hickie I, LaMonica H. Evaluating the quality and safety of health-related apps and e-tools: adapting the Mobile App Rating Scale and developing a quality assurance protocol. Internet Interv 2021 Apr;24:100379 [FREE Full text] [CrossRef] [Medline]
- Budget 2020-21: prioritising mental health – doubling better access initiative sessions. Australian Government Department of Health. 2020. URL: https://www.health.gov.au/resources/publications/budget-2020-21-prioritising-mental-health-doubling-better-access-initiative-sessions [accessed 2022-02-20]
- Budget 2021 - 2022: budget strategy and outlook. Commonwealth of Australia. URL: https://parlinfo.aph.gov.au/parlInfo/search/display/display.w3p;query=Id%3A%22library%2Fbudget%2F2021_01%22 [accessed 2022-02-20]
- Burns JM, Davenport TA, Durkin LA, Luscombe GM, Hickie IB. The internet as a setting for mental health service utilisation by young people. Med J Aust 2010 Jun 07;192(S11):S22-S26. [CrossRef] [Medline]
- Ospina-Pinillos L, Davenport T, Iorfino F, Tickell A, Cross S, Scott EM, et al. Using new and innovative technologies to assess clinical stage in early intervention youth mental health services: evaluation study. J Med Internet Res 2018 Sep 10;20(9):e259 [FREE Full text] [CrossRef] [Medline]
- Davenport TA, Cheng VW, Iorfino F, Hamilton B, Castaldi E, Burton A, et al. Flip the clinic: a digital health approach to youth mental health service delivery during the COVID-19 pandemic and beyond. JMIR Ment Health 2020 Dec 15;7(12):e24578 [FREE Full text] [CrossRef] [Medline]
- Ellis LA, McCabe KL, Rahilly KA, Nicholas MA, Davenport TA, Burns JM, et al. Encouraging young men's participation in mental health research and treatment: perspectives in our technological age. Clin Investigation 2014 Oct;4(10):881-888. [CrossRef]
- Ellis LA, Collin P, Hurley PJ, Davenport TA, Burns JM, Hickie IB. Young men's attitudes and behaviour in relation to mental health and technology: implications for the development of online mental health services. BMC Psychiatry 2013 Apr 20;13:119 [FREE Full text] [CrossRef] [Medline]
- Cross S, Davenport T, LaMonica H, Hickie I. Potential of real-time and integrated clinical data to drive continuous quality improvement in youth mental health services. Med J Aust 2019;211(7):S37-S39.
- Ospina-Pinillos L, Davenport T, Mendoza Diaz A, Navarro-Mancilla A, Scott EM, Hickie IB. Using participatory design methodologies to co-design and culturally adapt the Spanish version of the mental health eClinic: qualitative study. J Med Internet Res 2019 Aug 02;21(8):e14127 [FREE Full text] [CrossRef] [Medline]
- Empowering 8 billion minds: enabling better mental health for all via the ethical adoption of technologies. World Economic Forum. URL: https://www.weforum.org/whitepapers/empowering-8-billion-minds-enabling-better-mental-health-for-all-via-the-ethical-adoption-of-technologies [accessed 2022-02-20]
- Rosenberg S, Hickie I, Rock D. Rethinking mental health in Australia. The University of Sydney. URL: https://www.sydney.edu.au/content/dam/corporate/documents/brain-and-mind-centre/youthe/rethinking-the-mental-health-of-australia.pdf [accessed 2022-02-20]
- Nilsen P, Seing I, Ericsson C, Birken SA, Schildmeijer K. Characteristics of successful changes in health care organizations: an interview study with physicians, registered nurses and assistant nurses. BMC Health Serv Res 2020 Feb 27;20(1):147 [FREE Full text] [CrossRef] [Medline]
- Wisniewski H, Torous J. Digital navigators to implement smartphone and digital tools in care. Acta Psychiatr Scand 2020 Apr;141(4):350-355 [FREE Full text] [CrossRef] [Medline]
- National Mental Health Performance Subcommittee. Key performance indicators for Australian public mental health services. National Mental Health Strategy. 2013. URL: https://www.aihw.gov.au/getmedia/f9bb1a07-a43b-458a-9b73-64ef19d8aedd/Key-Performance-Indicators-for-Australian-Public-Mental-Health-Services-Third-Edition.pdf.aspx [accessed 2022-02-20]
- Morris P. The Australian mental health crisis: a system failure in need of treatment. Australas Psychiatry 2006 Sep;14(3):332-333. [CrossRef] [Medline]
|CRC: Cooperative Research Centre|
|GP: general practitioner|
|HIT: health information technology|
|HREC: human research ethics committee|
|KRMC: Kildare Road Medical Centre|
|MARS: Mobile App Rating Scale|
|NSW: New South Wales|
|R&D: research and development|
|STB: suicidal thoughts and behavior|
Edited by J Torous; submitted 26.08.21; peer-reviewed by V Mylonopoulou, AM Stupinski; comments to author 31.10.21; revised version received 14.12.21; accepted 30.12.21; published 09.03.22Copyright
©Haley M LaMonica, Frank Iorfino, Grace Yeeun Lee, Sarah Piper, Jo-An Occhipinti, Tracey A Davenport, Shane Cross, Alyssa Milton, Laura Ospina-Pinillos, Lisa Whittle, Shelley C Rowe, Mitchell Dowling, Elizabeth Stewart, Antonia Ottavio, Samuel Hockey, Vanessa Wan Sze Cheng, Jane Burns, Elizabeth M Scott, Ian B Hickie. Originally published in JMIR Mental Health (https://mental.jmir.org), 09.03.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.