This paper is in the following e-collection/theme issue:

Psychotic Disorders (160) Innovations in Mental Health Systems (390) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1514) Web-based and Mobile Health Interventions (3297) Mental Health Issues in Adolescence (442) Peer-to-Peer Support and Online Communities (636) Users' and Patients' Needs for Mental Health Services (403)

Published on in Vol 9, No 4 (2022): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/29211, first published .
Characterizing Use of a Multicomponent Digital Intervention to Predict Treatment Outcomes in First-Episode Psychosis: Cluster Analysis

Characterizing Use of a Multicomponent Digital Intervention to Predict Treatment Outcomes in First-Episode Psychosis: Cluster Analysis

Characterizing Use of a Multicomponent Digital Intervention to Predict Treatment Outcomes in First-Episode Psychosis: Cluster Analysis

Authors of this article:

Shaunagh O'Sullivan1, 2 Author Orcid Image ;   Lianne Schmaal1, 2 Author Orcid Image ;   Simon D'Alfonso1, 3 Author Orcid Image ;   Yara Jo Toenders1, 2 Author Orcid Image ;   Lee Valentine1, 2 Author Orcid Image ;   Carla McEnery1, 2 Author Orcid Image ;   Sarah Bendall1, 2 Author Orcid Image ;   Barnaby Nelson1, 2 Author Orcid Image ;   John F Gleeson4, 5 Author Orcid Image ;   Mario Alvarez-Jimenez1, 2 Author Orcid Image
Article Authors Cited by (11) Tweetations (32) Metrics

    Original Paper

    1Orygen, Parkville, Australia

    2Centre for Youth Mental Health, University of Melbourne, Melbourne, Australia

    3School of Computing and Information Systems, University of Melbourne, Melbourne, Australia

    4Health Brain and Mind Research Centre, Australian Catholic University, Melbourne, Australia

    5School of Behavioural and Health Sciences, Australian Catholic University, Melbourne, Australia

    Corresponding Author:

    Shaunagh O'Sullivan, BA, MPsychSc

    Orygen

    35 Poplar Road

    Parkville, 3052

    Australia

    Phone: 61 428 282 470

    Email: shaunagh.osullivan@orygen.org.au


    Background: Multicomponent digital interventions offer the potential for tailored and flexible interventions that aim to address high attrition rates and increase engagement, an area of concern in digital mental health. However, increased flexibility in use makes it difficult to determine which components lead to improved treatment outcomes.

    Objective: This study aims to identify user profiles on Horyzons, an 18-month digital relapse prevention intervention for first-episode psychosis that incorporates therapeutic content and social networking, along with clinical, vocational, and peer support, and to examine the predictive value of these user profiles for treatment outcomes. A secondary objective is to compare each user profile with young people receiving treatment as usual (TAU).

    Methods: Participants comprised 82 young people (aged 16-27 years) with access to Horyzons and 84 receiving TAU, recovering from first-episode psychosis. In addition, 6-month use data from the therapy and social networking components of Horyzons were used as features for K-means clustering for joint trajectories to identify user profiles. Social functioning, psychotic symptoms, depression, and anxiety were assessed at baseline and 6-month follow-up. General linear mixed models were used to examine the predictive value of user profiles for treatment outcomes and between each user profile with TAU.

    Results: A total of 3 user profiles were identified based on the following system use metrics: low use, maintained use of social components, and maintained use of both therapy and social components. The maintained therapy and social group showed improvements in social functioning (F2,51=3.58; P=.04), negative symptoms (F2,51=4.45; P=.02), and overall psychiatric symptom severity (F2,50=3.23; P=.048) compared with the other user profiles. This group also showed improvements in social functioning (F1,62=4.68; P=.03), negative symptoms (F1,62=14.61; P<.001), and overall psychiatric symptom severity (F1,63=5.66; P=.02) compared with the TAU group. Conversely, the maintained social group showed increases in anxiety compared with the TAU group (F1,57=7.65; P=.008). No differences were found between the low use group and the TAU group on treatment outcomes.

    Conclusions: Continued engagement with both therapy and social components might be key in achieving long-term recovery. Maintained social use and low use outcomes were broadly comparable with TAU, emphasizing the importance of maintaining engagement for improved treatment outcomes. Although the social network may be a key ingredient to increase sustained engagement, as users engaged with this more consistently, it should be leveraged as a tool to engage young people with therapeutic content to bring about social and clinical benefits.

    JMIR Ment Health 2022;9(4):e29211

    doi:10.2196/29211

    Keywords

    digital intervention; digital health; youth mental health; psychotic disorders; clustering; usage metrics; log data; social networking 


    Background

    Evidence indicates the efficacy of specialist early intervention (SEI) services for first-episode psychosis (FEP) in achieving symptomatic remission during the first 2 years of treatment [1,2]. Despite this, the critical period for relapse extends to 5 years from the onset of psychosis, with 55% to 70% of individuals relapsing after 2 years [3,4]. Research has indicated that some treatment effects may not be sustained at 5 years, after 2 years of SEI has ceased [5,6].

    Recently, 2 clinical trials addressed these limitations by evaluating the effects of extending SEI up to 5 years. Malla et al [7] found that clinical gains, in terms of remission of positive and negative psychotic symptoms, may be sustained if lower-intensity SEI is offered for an additional 3 years (on top of the 2 years already provided). However, findings from the trial by Albert et al [8] failed to demonstrate any additional benefits from extending SEI by 3 years, but this may be attributable to the high level of treatment provided to control participants in that study.

    Although SEI has reported success in improving symptoms during the first 2 years of treatment, many young people with FEP continue to experience poor social and functional outcomes [9,10]. Although social and functional recovery is regarded by young people as the most important aspect of recovery [11], few FEP interventions have made this a primary target [12]. Fowler et al [10] addressed this by evaluating the effectiveness of social therapy in combination with early intervention services, with findings showing increases in structured activity, indicative of improved social functioning after 9 months. Furthermore, a randomized controlled trial (RCT) by Chang et al [13] found improvements in functional outcomes when SEI was extended by 1 year, but this was not sustained at 1- and 2-year follow-ups. Therefore, further research is needed to establish the effectiveness of longer-term interventions focusing on social and functional outcomes.

    Digital interventions for FEP provide a unique opportunity to overcome the current limitations of treatment by providing continuous, engaging, and sustainable support to maintain long-term treatment effects [14]. It has been proposed that digital technologies can enhance care in FEP specifically by increasing access, enhancing current treatment, offering better predictive models, and accounting for clinical heterogeneity [15]. Some studies on the effectiveness of digital interventions for treating FEP and those with more established or sustained psychotic disorders have reported improvements in treatment outcomes such as social functioning [16], positive psychotic symptoms [17], negative psychotic symptoms [18], general psychopathology [18,19], overall psychiatric symptom severity [18], vocational outcomes [20], hallucination severity [19,21], hospital admissions [20,22], subjective well-being [16], social support [17], social connectedness [21], medication adherence [21], depression [1], and stress [14,17].

    Although digital interventions have been associated with improved outcomes, they have also been associated with high attrition rates [23], and most do not typically extend beyond a 3-month period to focus on long-term recovery [24-26]. To address these limitations, Alvarez-Jimenez et al [1,27] pioneered a model of multicomponent digital interventions entitled moderated online social therapy (MOST). The MOST model integrates the following: (1) interactive psychosocial interventions, (2) social networking, (3) expert clinical moderation, and (4) peer support. To address attrition rates, MOST also aims to enhance long-term engagement by offering a shared, secure, and private social network for young people with similar mental health experiences.

    The social networking and therapeutic elements of the MOST model were first applied in Horyzons, a world-first digital intervention aimed at maintaining long-term treatment effects and engagement and to improve social functioning in young people recovering from FEP after receiving 2 years of SEI treatment [1,12]. Strengths and mindfulness-based approaches to therapy were adopted, with the aim of increasing self-efficacy and positive emotions, which have been linked to improved social functioning in psychosis [28,29]. The principles of self-determination theory (SDT) were also used with the aim of improving social functioning through increased intrinsic motivation [30].

    A 4-week pilot study investigating the acceptability, safety, and clinical benefits of Horyzons indicated that the intervention was feasible, safe, and engaging and may enhance social connectedness in young people recovering from FEP [1]. An 18-month RCT of Horyzons has recently been completed, which found that Horyzons was effective in improving vocational outcomes and reducing presentations to hospital emergency services and hospital admissions compared with a control group receiving treatment as usual (TAU) [20]. Conversely, there were no differences between groups in social functioning over time. However, as there is limited evidence regarding the effectiveness of multicomponent digital interventions based on system use, it is difficult to determine the core therapeutic components of Horyzons, what outcomes they are associated with, and whether a specific pattern of use leads to improved social functioning in this population.

    In line with SDT, multicomponent digital interventions based on the MOST model offer young people a high degree of choice over how and when they engage with the system, which increases flexibility in use. This increased flexibility increases the possibility of variation in use patterns or user trajectories. Distinct user profiles may exist in such multicomponent digital interventions, with users who may differ in use and engagement levels over time. The introduction of additional components, such as a therapeutic social network, is needed to address high attrition rates, increase engagement and tailor interventions to cater to the clinical needs and preferences of young people. However, new methods are needed to understand the complexities associated with determining which aspects and patterns of use lead to improved outcomes. Statistical modeling techniques such as growth mixture modeling can be used to identify different groups of users with similar trajectories over time. These techniques have previously been used for detecting similar symptom trajectories in mental health interventions [31]. K-means clustering techniques have also been used to identify and characterize participants based on unidimensional and multidimensional trajectories [32-34].

    Objectives

    Horyzons provides a unique opportunity to examine the relationship between multidimensional patterns of use and treatment outcomes by categorizing the use of multiple intervention components, such as therapeutic and social networking components. By gaining a better understanding of system use and user trajectories, and how they relate to treatment outcomes, multicomponent digital interventions could be further optimized to improve long-term recovery. Therefore, this study aims to examine the association between user profiles and treatment outcomes on Horyzons by (1) identifying user profiles based on 2D patterns of system use on both therapeutic and social components of the intervention, (2) characterizing the user profiles based on baseline demographic and clinical characteristics, and (3) examining the predictive value of the user profiles for treatment outcomes.


    Study Design

    Horyzons was a single-blind 18-month RCT, where participants with remitted FEP were randomly allocated to either TAU following 2 years of specialized care or TAU along with access to a moderated web-based social therapy intervention (Horyzons) [12]. Horyzons was based on the MOST model, which integrates (1) web-based therapy (Pathways and Steps), (2) peer-to-peer web-based social networking (the Café), (3) peer moderation, and (4) expert support by mental health clinicians and vocational workers. This RCT was registered on the Australian New Zealand Clinical Trials Registry (ACTRN12614000009617).

    Ethics Approval

    Ethical approval for the Horyzons RCT was granted by the Melbourne Health Research Ethics Committee (2013.146).

    Participants

    Participants comprised 86 young people allocated to the Horyzons intervention and 84 young people allocated to TAU, recruited from the Early Psychosis Prevention and Intervention Centre (EPPIC) at Orygen Youth Health, Melbourne, between October 2013 and January 2017. EPPIC is a specialist FEP program that provides 1 ½ to 2 years of specialized care to young people aged 15 to 24 years with FEP [35,36].

    A total of 4 intervention participants who did not use the Horyzons platform independently and subsequently had no valid system use data were excluded from analyses. The remaining 82 intervention participants were aged between 16 and 27 years at randomization (mean 21, SD 2.88 years), and the 84 TAU participants were also aged between 16 and 27 years at randomization (mean 21, SD 2.83 years). Participants met clinical diagnosis for a first-episode psychotic disorder or mood disorder with psychotic features according to the Diagnostic and Statistical Manual of Mental Disorders 4th Edition (DSM-IV) [37], had not been treated with antipsychotic medication for >6 months before attending EPPIC, and showed remission of positive symptoms of psychosis for ≥4 weeks at the time of enrollment in the Horyzons study, as measured using the Positive and Negative Syndrome Scale [38].

    Measures

    Demographic and Clinical Characteristics

    Data from baseline and the 6-month follow-up were used for this study’s analysis. Demographic information collected at baseline included sex, age, and vocational status. Baseline clinical characteristics included psychotic symptoms, levels of social functioning, depression, and anxiety, and are described in the Social Functioning, Psychotic Symptoms, and Depression and Anxiety sections below.

    Social Functioning

    Social functioning was measured using the Personal and Social Performance Scale (PSP) [39]. Ratings are based on functioning in the following four domains: (1) socially useful activities, (2) personal and social relationships, (3) self-care, and (4) disturbing and aggressive behaviors. The following four subscales of the First Episode Social Functioning Scale (FESFS) were also included to capture the full construct of social functioning: (1) living skills, (2) friends and activities, (3) intimacy, and (4) interacting with people [40]. These subscales were chosen based on their strong psychometric properties, independence from psychotic symptoms, and sensitivity to treatment effects [12]. The FESFS was designed specifically for young people with FEP.

    Psychotic Symptoms

    The Positive and Negative Syndrome Scale was used to assess psychotic symptoms, which included three subscales measuring (1) positive symptoms, (2) negative symptoms, and (3) general psychopathology [38]. The total score comprised all items from the three subscales, which indicated overall psychiatric symptom severity.

    Depression and Anxiety

    Depression was assessed using the Calgary Depression Scale for Schizophrenia [41]. Anxiety was assessed using the Depression Anxiety and Stress Scale [42].

    System Use Metrics

    System use metrics were extracted from the Horyzons web-based platform for each user for each day of their trial involvement. Textbox 1 shows an overview of metrics representing aspects of use of the intervention’s therapeutic and social components.

    System use variables extracted from the Horyzons platform.

    Therapy-related variables

    • Number of steps started (steps refer to the intervention modules)
    • Number of actions done (actions refer to the activities that comprise a step)
    • Visited suggested content (suggested content refers to the therapeutic content recommended by clinical moderators)
    • Visited therapy (visiting therapy refers to visiting the homepage of the therapy component of the intervention)

    Social networking–related variables

    • Number of newsfeed posts (newsfeed refers to the social network)
    • Number of newsfeed comments
    • Number of Talk it Out posts (Talk it Out refers to a problem-solving forum run by peer moderators)
    • Number of likes made
    • Number of reactions made (reactions refer to short support messages in response to a post, eg, “thinking of you”)
    • Visited messages (messages refer to a private message section where moderators could contact participants directly)
    • Visited notifications
    • Visited newsfeed
    • Visited Talk it Out
    Textbox 1. System use variables extracted from the Horyzons platform.

    Therapeutic Pathways were divided into themes including understanding psychosis, identifying early warning signs to prevent relapse, identifying and exercising personal strengths, promoting social connections and positive emotions, and managing stress, anxiety, and depression. To increase usability, Pathways were further divided into short interactive Steps, for example, illustrating how to respond empathically to others (to foster positive connections). See Multimedia Appendix 1 for an example of a Step on Horyzons. Each Step was accompanied by Actions or Do its, aiming to translate learning into behavior change, for example, suggestions on how to exercise empathy in specific contexts. Expert clinical moderators could also recommend Pathways, Steps, Actions, and Talk it Outs they felt would be relevant to different users via a private message, which would appear as a notification on the user’s dashboard. Furthermore, users could visit the therapeutic component of Horyzons without completing any therapeutic content, for example, viewing what Pathway and Step was currently allocated to them.

    The social network or the Café was led and moderated by peer-workers, who were trained young people who had a lived experience of mental illness. Participants were encouraged to communicate with one another to foster social support. Participants could post comments on the Newsfeed or like, respond, or react to comments that were already posted. Predeveloped reactions were designed to facilitate social support, for example, “I get you” and “thinking of you.” Furthermore, participants could use the Talk It Out function to nominate relevant issues to discuss in a moderated forum, informed by an evidence-based problem-solving framework [43]. Participants received notifications when other users communicated on the social network. Participants also received private messages when a moderator contacted them directly via the platform. See Multimedia Appendix 2 for an example of a newsfeed post with likes and reactions on the Horyzons social network.

    Daily Activity Categories

    For this study, daily activity for both the therapeutic and social networking components was categorized. Daily activity was defined as the hierarchical level of system use per user per day based on the system use variables outlined in Textbox 1. A user was inactive when they did not display any activity on either the therapeutic or social component of Horyzons. Use was deemed passive when a user visited pages but did not actively engage with any content on either intervention component. A user was engaged with therapy when they started 1 step or completed 1 action. A user was highly engaged with therapy when they started >1 step, completed >1 action, or started at least one step and completed at least one action. Social use was deemed moderate when a user did not actively contribute to the social network but liked or reacted to at least one item. A user was active on the social network when they actively contributed via a post or comment. Figure 1 shows the hierarchical categorization of daily activity into 2 dimensions.

    ‎
    Figure 1. Hierarchical 2D daily activity categories.
    View this figure

    Statistical Analyses

    Identifying User Profiles

    K-means clustering for joint trajectories was implemented using the R package kml3d (R Studio) to identify data-driven user profiles using Euclidean distance [44-46]. This is an unsupervised nonparametric technique that simultaneously partitions user trajectories from both the social and therapy dimensions into distinct cluster groups. This technique uses a hill-climbing expectation-maximization algorithm, alternating through various initialization methods until convergence is reached [45,47].

    To run this analysis, participant trajectories were required to be of the same length. As such, we focused on the maximum number of days that all participants used Horyzons, which was 154 days. Each user’s first day on Horyzons consisted of an induction to the platform, so it could not be viewed as an independent system use. Therefore, daily activity (as per the hierarchical categories on both the social and therapeutic components of the intervention) from days 2 to 155 was used.

    In terms of adherence, it was expected that participants would use Horyzons fortnightly to benefit from the intervention. On this basis, daily activity was transformed into 22 meaningful weekly scores, and these 22 weekly use scores were used as input features for the K-means clustering. Weekly scores comprised the maximum level of use per week, for example, if a user used Horyzons twice during week 1, and this consisted of passive use of the therapy dimension for 1 day (level of activity=2) and engaging with therapy on the second day (level of activity=3); they would obtain a score of 3 for week 1 on the therapy dimension (ie, the highest level of activity in that week).

    No a priori hypothesis existed to substantiate the optimal number of clusters for analysis. Therefore, 2- to 4-cluster solutions were examined to account for complex patterns of system use found outside of a dichotomous high versus low use range. Furthermore, the sample was relatively small (N=82), suggesting that cluster solutions exceeding 4 would comprise too few participants per cluster. Cluster solutions with <15 participants in any cluster were excluded. K-means was rerun 100 times, each with different initial configurations, to ensure a global maximum was reached. A number of nonparametric fit indexes were used to compare cluster solutions, including the criteria developed by Calinski and Harabasz [48], Ray and Turi [49], and Davies and Bouldin [50]. A higher Calinski and Harabasz score indicates better fit, whereas lower Davies and Bouldin and Ray and Turi scores indicate better fit. In addition, cluster solutions were internally validated by calculating a Rand index, with scores closer to 1 indicating a higher likelihood of being assigned to the same cluster upon running 100 resamples [51]. Theoretical justifications and interpretability were also considered to select the optimal cluster solution.

    Characterizing User Profiles

    Differences between user profiles on demographic and baseline clinical characteristics were investigated using the 1-way analyses of variance and chi-square (χ2) tests for categorical variables.

    Examining the Predictive Value of User Profiles for Treatment Outcomes

    General linear mixed models were used to assess the associations between user profiles and treatment outcomes using the R package lme4 [46,52]. Cluster group (user profile), time (baseline, 6-month follow-up), and group-by-time interaction were added as predictors. The predictive value of user profiles was assessed for social functioning, psychotic symptoms, depression, and anxiety. Sex, age, and days of untreated psychosis were added to the models as a priori determined covariates, as shorter days of untreated psychosis has been associated with improved outcomes and remission in FEP [53-58], male sex has been associated with poorer social and functioning outcomes in FEP [53], and age and sex may influence the use of the system. The models also controlled for baseline differences in the outcomes of interest. The effects of interest included (1) the main effect of group, (2) the main effect of time, and (3) the interaction between group and time. User IDs were added to the models as a random intercept effect, as they resolve the nonindependence associated with having multiple responses per user. As a secondary analysis, general linear mixed models were used to assess the associations between each individual user profile and TAU.


    Clusters Based on Joint Trajectories of System Use

    The fit indexes for 2-, 3-, and 4-cluster solutions are reported in Multimedia Appendix 3. The 2-cluster solution was optimal based on all criteria, except for the Bayesian Information Criterion, where the 3-cluster solution showed the best solution. Cluster A (high-decreasing use) and B (low-decreasing use) trajectories remained consistent in the 2-, 3-, and 4-cluster solutions. The 2-cluster solution represented high-decreasing versus low-decreasing use, whereas the 3- and 4-cluster solutions represented more complex intermittent use, which existed based on visual inspections of the data (individual plots available upon request). Cluster C represented more intermittent and consistent use and added valuable information beyond high versus low use in terms of alternative user trajectories. On the basis of these observations, the 3-cluster solution was selected as it was superior to the 4-cluster solution on all fit indexes, and the fit indexes were still relatively high compared with the 2-cluster solution. The 3-cluster solution also showed good internal validity based on the Rand index.

    The trajectories of the user profiles based on the 3-cluster solution are shown in Figure 2. User profile A showed a rapid decrease in use on both the social and therapy dimensions after baseline and remained inactive for the following months; hence, this user profile was termed low use. User profile B showed initial high use on both dimensions, which decreased over time, with users remaining more active on the social dimension than on the therapy dimension (where use was mainly passive or inactive); therefore, user profile B was called maintained social. User profile C showed more variable but sustained use over time, remaining active on both dimensions, except during the final few weeks. User profile C also remained more engaged with the system’s therapy components than the other 2 user profiles and hence was called maintained therapy and social. The low use profile comprised 60% (49/82) of the users, the maintained social profile comprised 23% (19/82) of the users, and the maintained therapy and social profile comprised 17% (14/82) of the users.

    ‎
    Figure 2. User profile trajectories identified based on weekly hierarchical daily activity scores.
    View this figure

    Characteristics of User Profiles

    A 1-way between-groups analysis of variance indicated a statistically significant difference between user profiles on negative psychotic symptoms at baseline (F2, 79=6.375; P=.003). Post hoc comparisons using the Tukey honest significant difference test indicated that the maintained therapy and social profile (mean 14.36, SD 4.99) had significantly higher symptoms than the maintained social profile (mean 10.48, SD 3.14) and the low use profile (mean 11.05, SD 3.55). No significant differences were observed between user profiles on any other clinical characteristics or on any demographic variables at baseline. A full overview of the results can be found in Multimedia Appendix 4.

    Associations Between User Profiles and Treatment Outcomes

    Significant group-by-time interaction effects were found for the primary outcome of social functioning as measured using the PSP, overall psychiatric symptom severity, and negative psychotic symptoms, with baseline effects accounted for (Table 1). Post hoc tests revealed that for social functioning, this interaction effect was accounted for by significant improvements for the maintained therapy and social profile from 0 to 6 months (F1,11=8.81; P=.01), compared with the maintained social profile (F1,15=1.35; P=.26) and the low use profile (F1,28=0.17; P=.68; Figure 3). For overall psychiatric symptom severity, post hoc tests revealed that this interaction effect was accounted for by a significant decrease in symptoms for the maintained therapy and social profile (F1,11=5.99; P=.03), compared with the maintained social profile (F1,15=1.71; P=.21) and the low use profile (F1,27=0.004; P=.95; Figure 4). In terms of negative symptoms, post hoc tests revealed that this interaction effect was accounted for by a significant reduction in symptoms for the maintained therapy and social profile (F1,11=10.94; P=.007), compared with the maintained social profile (F1,15=0.66; P=.43) and the low use profile (F1,27=0.98; P=.33; Figure 5).

    Table 1. Changes on outcomes from baseline to 6 months for user profiles.

    		User profilesGroup × time interaction, F test (df)P value

    		Low use (n=49), mean (SD)Maintained social (n=19), mean (SD)Maintained therapy and social (n=14), mean (SD)

    PSPa

    		Baseline67.09 (1.97)70.17 (3.15)65.31 (3.64)N/AbN/A

    		6 months68.64 (2.62)67.11 (3.15)76.31 (3.64)3.58 (2, 51).04
    FESFSc independent living skills

    		Baseline13.66 (0.33)14.07 (0.52)13.11 (0.60)N/AN/A

    		6 months13.59 (0.41)14.04 (0.54)13.03 (0.62)0.003 (2, 45).99
    FESFS interacting with people

    		Baseline12.96 (0.35)12.52 (0.56)11.90 (0.64)N/AN/A

    		6 months13.26 (0.44)12.09 (0.58)12.27 (0.66)0.87 (2, 45).42
    FESFS friends and activities

    		Baseline18.96 (0.47)17.50 (0.75)17.61 (0.86)N/AN/A

    		6 months19.53 (0.62)17.70 (0.78)18.06 (0.89)0.08 (2, 46).92
    FESFS intimacy

    		Baseline15.37 (0.45)15.22 (0.75).52 (0.85)N/AN/A

    		6 months15.24 (0.58)14.77 (0.75)13.52 (0.90)0.43 (2, 40).65
    PANSSd total

    		Baseline45.15 (1.77)43.69 (2.84)48.91 (3.27)N/AN/A

    		6 months44.96 (2.22)46.80 (2.84)41.73 (3.27)3.23 (2, 50).048
    PANSS positive

    		Baseline10.49 (0.54)10.38 (0.86)10.31 (1.00)N/AN/A

    		6 months10.29 (0.70)11.31 (0.86)9.89 (1.00)0.60 (2, 52).55
    PANSS negative

    		Baseline10.73 (0.55)11.24 (0.89)13.62 (1.03)N/AN/A

    		6 months10.02 (0.72)10.55 (0.89)9.28 (1.03)4.45 (2, 51).02
    PANSS general psychopathology

    		Baseline23.93 (1.06)22.07 (1.70)24.99 (1.96)N/AN/A

    		6 months24.65 (1.33)24.92 (1.70)22.56 (1.96)2.34 (2, 50).11
    CDSSe

    		Baseline3.91 (0.67)3.31 (1.08)2.78 (1.25)N/AN/A

    		6 months4.41 (0.84)3.84 (1.08)2.20 (1.25)0.33 (2, 50).72
    DASSf anxiety

    		Baseline12.14 (1.35)6.79 (2.02)10.15 (2.45)N/AN/A

    		6 months12.02 (1.77)10.58 (2.18)9.42 (2.53)1.35 (2, 42).27

    aPSP: Personal and Social Performance Scale.

    bN/A: not applicable.

    cFESFS: First Episode Social Functioning Scale.

    dPANSS: Positive and Negative Syndrome Scale.

    eCDSS: Calgary Depression Scale for Schizophrenia.

    fDASS: Depression, Anxiety, and Stress Scale.

    ‎
    Figure 3. Mean trends in PSP social functioning scores for user profiles (95% CIs). PSP: Personal and Social Performance Scale.
    View this figure
    ‎
    Figure 4. Mean trends in PANSS overall psychiatric symptom severity scores for user profiles (95% CIs). PANSS: Positive and Negative Syndrome Scale.
    View this figure
    ‎
    Figure 5. Mean trends in PANSS negative symptom scores for user profiles (95% CIs). PANSS: Positive and Negative Syndrome Scale.
    View this figure

    No significant group (user profile) by time associations were found for aspects of social functioning as measured by the FESFS, positive psychotic symptoms, general psychopathology, depression, or anxiety (Table 1). Furthermore, no main effects were found for differences between the profiles at each time point, and no main effects were found for changes over time for each profile on the outcomes.

    Associations Between Individual User Profiles and TAU With Treatment Outcomes

    Significant group-by-time interaction effects were found for social functioning as measured using the PSP, overall psychiatric symptom severity, and negative psychotic symptoms for the maintained therapy and social group versus the TAU group (Table 2). For social functioning, post hoc tests revealed that this interaction effect was accounted for by improvements for the maintained therapy and social group (F1,11=8.81; P=.01) compared with the TAU group (F1,56=0.87; P=.35), and a significant difference between groups at 6 month follow-up, with the maintained therapy and social group having higher social functioning scores than the TAU group (F1,57=5.82; P=.02; Figure 6). In terms of overall psychiatric symptom severity, post hoc tests revealed that this interaction effect was accounted for by decreases in symptoms for the maintained therapy and social group (F1,11=5.99; P=.03), compared with the TAU group (F1,57=0.78; P=.38; Figure 7). In terms of negative symptoms, post hoc tests revealed that this interaction effect was accounted for by decreases in symptoms for the maintained therapy and social group (F1,11=10.94; P=.006) compared with the TAU group (F1,54=0.71; P=.40), and a significant difference between groups at baseline, with the maintained therapy and social group having higher symptoms than the TAU group (F1,76=4.35; P=.04; Figure 8).

    No significant group-by-time associations were found for treatment outcomes for the low use group versus the TAU group (Multimedia Appendix 5). Similarly, with the exception of anxiety, no significant group-by-time associations were found for treatment outcomes for the maintained social group versus the TAU group (Multimedia Appendix 6). Post hoc tests revealed that the interaction effect found for anxiety was accounted for by a significant decrease in symptoms for the TAU group (F1,47=6.50; P=.01), a significant increase in symptoms for the maintained social group (F1,12=6.11; P=.03), and a significant difference between groups at baseline, with the maintained social group having lower anxiety scores than the TAU group (F1,72=4.07; P=.047; Figure 9).

    Table 2. Changes on outcomes from baseline to 6 months for the maintained therapy and social and TAUa groups.

    		TAU (n=84), mean (SD)Maintained therapy and social (n=19), mean (SD)Group × time interaction, F test (df)P value
    PSPb

    		Baseline65.19 (1.57)65.79 (3.82)N/AcN/A

    		6 months66.89 (1.80)76.79 (3.82)4.68 (1, 62).03
    FESFSd independent living skills

    		Baseline13.71 (0.21)13.12 (0.50)N/AN/A

    		6 months13.71 (0.24)13.04 (0.51)0.02 (1, 60).88
    FESFS interacting with people

    		Baseline12.78 (0.24)11.94 (0.58)N/AN/A

    		6 months12.62 (0.27)12.31 (0.59)0.99 (1, 57).32
    FESFS friends and activities

    		Baseline18.47 (0.43)17.74 (1.00)N/AN/A

    		6 months18.25 (0.47)18.20 (1.02)0.60 (1, 55).44
    FESFS intimacy

    		Baseline14.82 (0.39)14.56 (0.92)N/AN/A

    		6 months14.77 (0.42)13.47 (0.96)1.63 (1, 52).20
    PANSSe total

    		Baseline44.26 (1.37)48.57 (3.32)N/AN/A

    		6 months45.72 (1.54)41.40 (3.32)5.66 (1, 63).02
    PANSS positive

    		Baseline9.47 (0.43)10.24 (1.05)N/AN/A

    		6 months9.72 (0.50)9.82 (1.05)0.24 (1, 64).62
    PANSS negative

    		Baseline11.00 (0.43)13.42 (1.05)N/AN/A

    		6 months10.64 (0.48)9.09 (1.05)14.61 (1, 62)<.001
    PANSS general psychopathology

    		Baseline23.78 (0.83)24.92 (2.01)N/AN/A

    		6 months25.39 (0.95)22.50 (2.01)2.76 (1, 63).10
    CDSSf

    		Baseline2.72 (0.41)2.86 (0.99)N/AN/A

    		6 months3.46 (0.46)2.29 (0.99)1.56 (1, 62).22
    DASSg anxiety

    		Baseline12.35 (1.16)8.85 (2.80)N/AN/A

    		6 months9.00 (1.32)8.21 (2.89)0.76 (1, 52).39

    aTAU: treatment as usual.

    bPSP: Personal and Social Performance Scale.

    cN/A: not applicable.

    dFESFS: First Episode Social Functioning Scale.

    ePANSS: Positive and Negative Syndrome Scale.

    fCDSS: Calgary Depression Scale for Schizophrenia.

    gDASS: Depression, Anxiety, and Stress Scale.

    ‎
    Figure 6. Mean trends in PSP social functioning scores for the maintained therapy and social and TAU groups (95% CIs). PSP: Personal and Social Performance Scale; TAU: treatment as usual.
    View this figure
    ‎
    Figure 7. Mean trends in PANSS overall psychiatric symptom severity scores for the maintained therapy and social and TAU groups (95% CIs). PANSS: Positive and Negative Syndrome Scale; TAU: treatment as usual.
    View this figure
    ‎
    Figure 8. Mean trends in PANSS negative symptom scores for the maintained therapy and social and TAU groups (95% CIs). PANSS: Positive and Negative Syndrome Scale; TAU: treatment as usual.
    View this figure
    ‎
    Figure 9. Mean trends in DASS anxiety scores for the maintained social and TAU groups (95% CIs). DASS: Depression Anxiety and Stress Scale; TAU: treatment as usual.
    View this figure

    Principal Findings

    This is the first study that characterized participants’ patterns of use of a multicomponent digital intervention (Horyzons), which included interactive therapy content and social networking, to predict treatment outcomes for young people recovering from FEP. Using a clustering procedure for joint trajectories, we identified three distinct user profiles: (1) low use, (2) maintained use of social components (maintained social), and (3) maintained use of both therapy and social components (maintained therapy and social). The maintained therapy and social profile had higher negative symptoms at baseline compared with the maintained social and low use profiles. The maintained therapy and social profile showed statistically significant improvements in social functioning and decreases in negative symptoms and overall psychiatric symptom severity compared with both the low use and maintained social profiles and the TAU group.

    We used K-means clustering for joint trajectories to identify user profiles beyond that of a high versus low use dichotomy. Our approach accounted for the level of activity over time across the intervention components, going beyond categorizing the number of log-ins, which is limited in terms of meaningful engagement. In doing so, we found that the user profiles of individuals who demonstrated more variable but sustained use of both the therapy and social components over time were significantly associated with improved social functioning and clinical outcomes. In contrast, user profiles consisting of individuals with decreasing use on both social and therapy dimensions (ie, low use) demonstrated clinical outcomes comparable with that of TAU. The use of K-means clustering to identify use patterns in digital mental health interventions in the literature is a novel approach. A recent study by Sanatkar et al [59] used it to examine the association between engagement profiles (based on 2-month system use metrics) and depression and anxiety outcomes. They reported overall reductions in depressive and anxiety symptoms, but no differences were observed between the clusters. However, all users were somewhat engaged during this 2-month period, making it difficult to determine the optimal levels of use for improved treatment outcomes and how this was compared with nonuse. These findings differ from other research indicating that dropout rates in mental health apps are very rapid during the first month, with a retention rate of only 3.3% in the general population [60] and 0.5% to 28.6% completion rates or use beyond 6 weeks in interventions targeting depression and anxiety [61]. However, little is known about use patterns beyond a 2-month period, which our study examined.

    Our findings indicated that the maintained use of both the therapy and social components was significantly associated with improvements in social functioning (the primary outcome of the Horyzons trial) compared with the other 2 user profiles. This is consistent with a recent pilot study investigating the effectiveness of a strengths- and mindfulness-based web-based social therapy for young people at ultrahigh risk of psychosis, which found increases in social functioning at 2-month follow-up [16]. It is worth noting that the MOST platform design is informed by SDT [62], which emphasizes meeting three key psychological needs to support motivation and behavioral change: (1) autonomy (feeling a sense of choice about one’s behavior), (2) competence (being able to bring about positive changes in desired outcomes), and (3) relatedness (feeling accepted by one’s social milieu). It may be the case that the combined system use (eg, the maintained therapy and social profile) aligned with both competence (therapy) and relatedness (social network), providing support for the SDT framework as a potentially mediating means in which to improve social functioning outcomes. Young people could engage in therapy on their own terms, which may also have promoted competence and autonomy. For example, choice in treatment (such as the choice young people had to complete therapy they felt was relevant to their needs on Horyzons) has been tied to the notion of individual autonomy [63]. Furthermore, there is evidence to suggest that moderated therapy, such as that offered on Horyzons, can promote self-competence in young people in particular [64].

    Other improved outcomes, in terms of negative symptoms and overall psychiatric symptom severity, were observed for the maintained therapy and social group compared with the other 2 user profiles on Horyzons. These findings are consistent with those of the Horyzons RCT, which reported lower levels of negative symptoms compared with TAU from baseline to 12 months (which corresponded with a period of higher use of the Horyzons platform) [20], and lends support to the notion that Horyzons may improve negative symptoms for those young people with a certain level of engagement with the digital platform.

    To be able to improve outcomes, our study suggests that sustained engagement with both the therapy and social networking components of Horyzons is required. Although these users comprised only 17% (14/82) of our sample, social functioning and negative symptoms are typically treatment resistant in FEP, which highlights the clinical significance of this finding [65,66]. Overall, 40% (33/82) of the users showed sustained use either on the social network alone (19/82, 23%) or on both the therapy and social networking aspects of the intervention (14/82, 17%), whereas 60% (49/82) of the participants were in the low use profile. This is an important observation, as a recent systematic search indicated a 15-day retention rate of 3.9% and a 30-day retention rate of 3.3% for mental health apps [60]; in contrast, 40% (33/82) of the users in our study showed more sustained use over 155 days. Furthermore, it is important to note that the low use profile did not mean nonuse. This cluster had a mean number of 12 log-ins to Horyzons over 6 months, indicating that these young people did engage with Horyzons but to a lesser extent than the maintained social (mean log-ins 142) and maintained therapy and social (mean log-ins 75) profiles. This also indicates that log-ins are not a good indicator of intervention effectiveness, as the maintained social profile had a higher number of log-ins than the maintained therapy and social profile but lower consistent engagement with therapy content. Therefore, rather than designing platforms to maximize log-ins, we need to design platforms to promote sustained engagement with the therapy and social components.

    Exploratory analyses comparing each user profile to TAU further supported our main findings by demonstrating statistically significant improvements in social functioning, negative symptoms, and overall psychiatric symptom severity for the maintained therapy and social group compared with the TAU group. Conversely, increases in anxiety were observed for those in the maintained social group compared with those in the TAU group. An explanation for this may be that worsening of outcomes may lead to increased motivation to engage with social aspects and low motivation or perceived competence to engage with therapeutic content. Use of the social network was mostly moderate, with users mostly liking and reacting to posts (indicating passive use) rather than actively contributing via a post or comment. These findings are consistent with a study that found passive social media use to be associated with increased anxiety among adolescents [67]. Therefore, although the social network may increase engagement, as users engaged with this more consistently on Horyzons, it is important that it is designed to reduce anxiety and is leveraged to increase young people’s motivation to engage with therapeutic content for continued support, to bring about improved social and clinical outcomes. This is especially important given the critical 5-year period for the risk of relapse in FEP [4].

    Our study confirmed that use is complex and that, although the maintained therapy and social profile showed improvements in outcomes, they also had higher negative symptoms at baseline. This raises the question of whether higher negative symptoms led to higher engagement, higher engagement led to improvements in outcomes, or both. For example, higher baseline symptoms may relate to perceived need to engage with therapeutic content. This is in line with previous research, which found that certain users only engaged with therapy until they completed what was relevant for them. Pung et al [68] offered participants self-help management strategies, similar to what was offered on Horyzons, for example, mindfulness steps and opportunities for social connection. Participants discontinued use after a skill was acquired but still had access to the intervention in case symptoms reemerged. These various reasons for use and disengagement may mask associations and contribute to mixed findings on the effectiveness of digital interventions. Although we controlled for baseline differences and our findings were consistent across social and clinical outcomes, we cannot make causal inferences about the change in outcomes in our study. Future research could address this by using multilevel models with an autoregressive lag [69] and examining the relationship between patterns of engagement and outcomes in real time [70].

    Limitations

    It should also be noted that this study had a number of methodological limitations. Although we controlled for baseline differences and key potential confounders, the findings of these analyses need to be interpreted with caution. First, the analyses comparing each user profile with TAU were exploratory and nonrandomized. Second, owing to the small sample size, we could not correct for multiple comparisons. That said, this is the first study to explore patterns of use in a multicomponent digital intervention for FEP and is arguably an informative starting point, as our results have significant clinical implications. Future research could build upon this contribution to the literature by replicating these analyses with larger sample sizes.

    Conclusions

    In conclusion, our findings indicate that sustained engagement with both the therapeutic and social networking components of Horyzons was key in improving social functioning, negative symptoms, and overall psychiatric symptom severity in young people with psychosis. This supports the therapeutic value of Horyzons and points to the need to capture complex patterns of use over time to determine key therapeutic targets and optimal use for improved outcomes. Going forward, this can be done in real time, with ongoing optimization of intervention features and management against key outcomes. This is a development in progress as digital interventions based on the MOST model are currently being implemented into clinical services as part of routine care, and novel methodologies including fast iterative A/B testing and artificial intelligence optimization methods will be used to fast-track innovation and research translation [70]. These findings have real-world implications for the development of multicomponent digital interventions, as well as for the treatment of young people with psychosis through digital platforms. Future research will need to determine how to distill the contribution of specific aspects of the intervention and how components may work together to sustain user engagement and improve clinical outcomes. We are currently investigating this by determining which aspects of Horyzons system use lead to subsequent use by means of multiple convergent cross mapping.

    Acknowledgments

    The authors would like to thank the young people who participated in the Horyzons study and agreed to share their data for research purposes. The study was funded by the Mental Illness Research Fund (Victorian Government, Australia), the Australian National Health and Medical Research Council (NHMRC), and the Telstra Foundation. MAJ was supported by an Investigator Grant (APP1177235) from the NHMRC. LS was supported by an NHMRC Career Development Fellowship (1140764). SB was supported by the Ronald Philip Griffiths Fellowship. BN was supported by an NHMRC Senior Research Fellowship (1137687).

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Horyzons step: how to flourish.

    PNG File , 249 KB
    ‎
    Multimedia Appendix 2

    Horyzons social network.

    PNG File , 269 KB
    ‎
    Multimedia Appendix 3

    Fit indexes for each cluster solution.

    DOC File , 32 KB
    ‎
    Multimedia Appendix 4

    Baseline demographic and clinical characteristics for user profiles.

    DOC File , 57 KB
    ‎
    Multimedia Appendix 5

    Changes in outcomes from baseline to 6-months for the low use and treatment as usual groups.

    DOC File , 41 KB
    ‎
    Multimedia Appendix 6

    Changes in outcomes from baseline to 6-months for the maintained social and treatment as usual groups.

    DOC File , 41 KB
    1. Alvarez-Jimenez M, Bendall S, Lederman R, Wadley G, Chinnery G, Vargas S, et al. On the HORYZON: moderated online social therapy for long-term recovery in first episode psychosis. Schizophr Res 2013 Jan;143(1):143-149. [CrossRef] [Medline]
    2. Nordentoft M, Rasmussen JO, Melau M, Hjorthøj CR, Thorup AA. How successful are first episode programs? A review of the evidence for specialized assertive early intervention. Curr Opin Psychiatry 2014 May;27(3):167-172. [CrossRef] [Medline]
    3. Addington D, Addington MD, Patten S. Relapse rates in an early psychosis treatment service. Acta Psychiatr Scand 2007 Feb;115(2):126-131. [CrossRef] [Medline]
    4. Alvarez-Jimenez M, Priede A, Hetrick S, Bendall S, Killackey E, Parker A, et al. Risk factors for relapse following treatment for first episode psychosis: a systematic review and meta-analysis of longitudinal studies. Schizophr Res 2012 Aug;139(1-3):116-128. [CrossRef] [Medline]
    5. Bertelsen M, Jeppesen P, Petersen L, Thorup A, Øhlenschlaeger J, Le Quach P, et al. Course of illness in a sample of 265 patients with first-episode psychosis--five-year follow-up of the Danish OPUS trial. Schizophr Res 2009 Feb;107(2-3):173-178. [CrossRef] [Medline]
    6. Gafoor R, Nitsch D, McCrone P, Craig TK, Garety PA, Power P, et al. Effect of early intervention on 5-year outcome in non-affective psychosis. Br J Psychiatry 2010 May 02;196(5):372-376. [CrossRef] [Medline]
    7. Malla A, Joober R, Iyer S, Norman R, Schmitz N, Brown T, et al. Comparing three-year extension of early intervention service to regular care following two years of early intervention service in first-episode psychosis: a randomized single blind clinical trial. World Psychiatry 2017 Oct 21;16(3):278-286 [FREE Full text] [CrossRef] [Medline]
    8. Albert N, Melau M, Jensen H, Emborg C, Jepsen JR, Fagerlund B, et al. Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II). Br Med J 2017 Feb 24;356:j1015 [FREE Full text] [CrossRef] [Medline]
    9. Alvarez-Jimenez M, O'Donoghue B, Thompson A, Gleeson JF, Bendall S, Gonzalez-Blanch C, et al. Beyond clinical remission in first episode psychosis: thoughts on antipsychotic maintenance vs. guided discontinuation in the functional recovery era. CNS Drugs 2016 May 22;30(5):357-368. [CrossRef] [Medline]
    10. Fowler D, Hodgekins J, French P, Marshall M, Freemantle N, McCrone P, et al. Social recovery therapy in combination with early intervention services for enhancement of social recovery in patients with first-episode psychosis (SUPEREDEN3): a single-blind, randomised controlled trial. Lancet Psychiatry 2018 Jan;5(1):41-50. [CrossRef]
    11. Stouten LH, Veling W, Laan W, van der Helm M, van der Gaag M. Psychosocial functioning in first-episode psychosis and associations with neurocognition, social cognition, psychotic and affective symptoms. Early Interv Psychiatry 2017 Feb 13;11(1):23-36. [CrossRef] [Medline]
    12. Alvarez-Jimenez M, Bendall S, Koval P, Rice S, Cagliarini D, Valentine L, et al. HORYZONS trial: protocol for a randomised controlled trial of a moderated online social therapy to maintain treatment effects from first-episode psychosis services. BMJ Open 2019 Feb 19;9(2):e024104 [FREE Full text] [CrossRef] [Medline]
    13. Chang WC, Kwong VW, Lau ES, So HC, Wong CS, Chan GH, et al. Sustainability of treatment effect of a 3-year early intervention programme for first-episode psychosis. Br J Psychiatry 2017 Jul 02;211(1):37-44. [CrossRef] [Medline]
    14. Alvarez-Jimenez M, Alcazar-Corcoles M, González-Blanch C, Bendall S, McGorry P, Gleeson J. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res 2014 Jun;156(1):96-106. [CrossRef] [Medline]
    15. Torous J, Woodyatt J, Keshavan M, Tully LM. A new hope for early psychosis care: the evolving landscape of digital care tools. Br J Psychiatry 2019 May 11;214(5):269-272 [FREE Full text] [CrossRef] [Medline]
    16. Alvarez-Jimenez M, Gleeson J, Bendall S, Penn D, Yung A, Ryan R, et al. Enhancing social functioning in young people at Ultra High Risk (UHR) for psychosis: a pilot study of a novel strengths and mindfulness-based online social therapy. Schizophr Res 2018 Dec;202:369-377. [CrossRef] [Medline]
    17. Rotondi AJ, Anderson CM, Haas GL, Eack SM, Spring MB, Ganguli R, et al. Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr Serv 2010 Nov;61(11):1099-1105. [CrossRef] [Medline]
    18. Bucci S, Barrowclough C, Ainsworth J, Machin M, Morris R, Berry K, et al. Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophr Bull 2018 Aug 20;44(5):1070-1080 [FREE Full text] [CrossRef] [Medline]
    19. Gottlieb JD, Romeo KH, Penn DL, Mueser KT, Chiko BP. Web-based cognitive-behavioral therapy for auditory hallucinations in persons with psychosis: a pilot study. Schizophr Res 2013 Apr;145(1-3):82-87. [CrossRef] [Medline]
    20. Alvarez-Jimenez M, Koval P, Schmaal L, Bendall S, O'Sullivan S, Cagliarini D, et al. The Horyzons project: a randomized controlled trial of a novel online social therapy to maintain treatment effects from specialist first-episode psychosis services. World Psychiatry 2021 Jun 18;20(2):233-243 [FREE Full text] [CrossRef] [Medline]
    21. Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, Holden JL. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull 2012 May 10;38(3):414-425 [FREE Full text] [CrossRef] [Medline]
    22. Glynn SM, Randolph ET, Garrick T, Lui A. A proof of concept trial of an online psychoeducational program for relatives of both veterans and civilians living with schizophrenia. Psychiatr Rehabil J 2010;33(4):278-287. [CrossRef] [Medline]
    23. de Beurs D, van Bruinessen I, Noordman J, Friele R, van Dulmen S. Active involvement of end users when developing web-based mental health interventions. Front Psychiatry 2017 May 03;8:72 [FREE Full text] [CrossRef] [Medline]
    24. Berry N, Lobban F, Emsley R, Bucci S. Acceptability of interventions delivered online and through mobile phones for people who experience severe mental health problems: a systematic review. J Med Internet Res 2016 May 31;18(5):e121 [FREE Full text] [CrossRef] [Medline]
    25. Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22 [FREE Full text] [CrossRef] [Medline]
    26. Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res 2019 Jul 22;21(7):e12869 [FREE Full text] [CrossRef] [Medline]
    27. Alvarez-Jimenez M, Rice S, D'Alfonso S, Leicester S, Bendall S, Pryor I, et al. A novel multimodal digital service (moderated online social therapy+) for help-seeking young people experiencing mental ill-health: pilot evaluation within a national youth e-mental health service. J Med Internet Res 2020 Aug 13;22(8):e17155 [FREE Full text] [CrossRef] [Medline]
    28. Garland EL, Fredrickson B, Kring AM, Johnson DP, Meyer PS, Penn DL. Upward spirals of positive emotions counter downward spirals of negativity: insights from the broaden-and-build theory and affective neuroscience on the treatment of emotion dysfunctions and deficits in psychopathology. Clin Psychol Rev 2010 Nov;30(7):849-864. [CrossRef]
    29. Vaskinn A, Ventura J, Andreassen OA, Melle I, Sundet K. A social path to functioning in schizophrenia: from social self-efficacy through negative symptoms to social functional capacity. Psychiatry Res 2015 Aug 30;228(3):803-807. [CrossRef] [Medline]
    30. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000;55(1):68-78. [CrossRef]
    31. Ellis RE, Seal ML, Simmons JG, Whittle S, Schwartz OS, Byrne ML, et al. Longitudinal trajectories of depression symptoms in adolescence: psychosocial risk factors and outcomes. Child Psychiatry Hum Dev 2017 Aug 12;48(4):554-571. [CrossRef] [Medline]
    32. Mas S, Gassó P, Rodríguez N, Cabrera B, Mezquida G, Lobo A, PEPs group. Personalized medicine begins with the phenotype: identifying antipsychotic response phenotypes in a first-episode psychosis cohort. Acta Psychiatr Scand 2020 Jun 05;141(6):541-552. [CrossRef] [Medline]
    33. Hall M, Holton KM, Öngür D, Montrose D, Keshavan MS. Longitudinal trajectory of early functional recovery in patients with first episode psychosis. Schizophr Res 2019 Jul;209:234-244 [FREE Full text] [CrossRef] [Medline]
    34. Tang A, Van Lieshout RJ, Lahat A, Duku E, Boyle MH, Saigal S, et al. Shyness trajectories across the first four decades predict mental health outcomes. J Abnorm Child Psychol 2017 Nov 25;45(8):1621-1633. [CrossRef] [Medline]
    35. Cotton SM, Filia KM, Ratheesh A, Pennell K, Goldstone S, McGorry PD. Early psychosis research at Orygen, The National Centre of Excellence in Youth Mental Health. Soc Psychiatry Psychiatr Epidemiol 2016 Jan 23;51(1):1-13. [CrossRef] [Medline]
    36. McGorry PD, Edwards J, Mihalopoulos C, Harrigan SM, Jackson HJ. EPPIC: an evolving system of early detection and optimal management. Schizophr Bull 1996 Jan 01;22(2):305-326. [CrossRef] [Medline]
    37. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (4th Edition). Washington, DC: American Psychiatric Association; 1994.
    38. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull 1987 Jan 01;13(2):261-276. [CrossRef] [Medline]
    39. Morosini PL, Magliano L, Brambilla L, Ugolini S, Pioli R. Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale (SOFAS) to assess routine social functioning. Acta Psychiatr Scand 2000 Apr;101(4):323-329. [Medline]
    40. Lecomte T, Corbière M, Ehmann T, Addington J, Abdel-Baki A, Macewan B. Development and preliminary validation of the First Episode Social Functioning Scale for early psychosis. Psychiatry Res 2014 May 30;216(3):412-417. [CrossRef] [Medline]
    41. Addington D, Addington J, Maticka-tyndale E. Assessing sepression in schizophrenia: the Calgary depression scale. Br J Psychiatry 2018 Aug 06;163(S22):39-44. [CrossRef]
    42. Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. Sydney, N.S.W: Psychology Foundation of Australia; 1996.
    43. D'Zurilla TJ, Goldfried MR. Problem solving and behavior modification. Journal of Abnormal Psychology 1971 Aug;78(1):107-126. [CrossRef]
    44. Genolini C, Pingault J, Driss T, Côté S, Tremblay R, Vitaro F, et al. KmL3D: a non-parametric algorithm for clustering joint trajectories. Comput Methods Programs Biomed 2013 Jan;109(1):104-111. [CrossRef] [Medline]
    45. Genolini C, Alacoque X, Sentenac M, Arnaud C. kml and kml3d: R Packages to cluster longitudinal data. J Stat Soft 2015;65(4):1-34. [CrossRef]
    46. RStudio Team. RStudio: Integrated Development Environment for R. 2021.   URL: http://www.rstudio.com/ [accessed 2021-03-22]
    47. Kahkoska AR, Nguyen CT, Adair LA, Aiello AE, Burger KS, Buse JB, et al. Longitudinal phenotypes of type 1 diabetes in youth based on weight and glycemia and their association with complications. J Clin Endocrinol Metabol 2019 Jul 10;104(12):6003-6016. [CrossRef] [Medline]
    48. Calinski T, Harabasz J. A dendrite method for cluster analysis. Commun Stat Theory Methods 1974;3(1):1-27. [CrossRef]
    49. Ray S, Turi R. Determination of number of clusters in k-means clustering and application in colour image segmentation. In: Proceedings of the 4th International Conference on Advances in Pattern Recognition and Digital Techniques. 1999 Presented at: 4th International Conference on Advances in Pattern Recognition and Digital Techniques; 1999; Calcutta, India   URL: https:/​/www.​semanticscholar.org/​paper/​Determination-of-Number-of-Clusters-in-K-Means-and-Ray-Turi/​0ec132fce9971d1e0e670e650b58176dc7bf36da
    50. Davies DL, Bouldin DW. A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 1979 Apr;PAMI-1(2):224-227. [CrossRef]
    51. Rand WM. Objective criteria for the evaluation of clustering methods. J Am Stat Assoc 1971 Dec;66(336):846-850. [CrossRef]
    52. Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Soft 2015;67(1):1-48. [CrossRef]
    53. Chan SK, Hui CL, Chang WC, Lee EH, Chen EY. Ten-year follow up of patients with first-episode schizophrenia spectrum disorder from an early intervention service: predictors of clinical remission and functional recovery. Schizophr Res 2019 Feb;204:65-71. [CrossRef] [Medline]
    54. Marshall M, Lewis S, Lockwood A, Drake R, Jones P, Croudace T. Association Between Duration of Untreated Psychosis and Outcome in Cohorts of First-Episode Patients. Arch Gen Psychiatry 2005 Sep 01;62(9):975. [CrossRef]
    55. Penttilä M, Jääskeläinen E, Hirvonen N, Isohanni M, Miettunen J. Duration of untreated psychosis as predictor of long-term outcome in schizophrenia: systematic review and meta-analysis. Br J Psychiatry 2014 Aug 02;205(2):88-94. [CrossRef] [Medline]
    56. Emsley R, Rabinowitz J, Medori R, Early Psychosis Global Working Group. Remission in early psychosis: rates, predictors, and clinical and functional outcome correlates. Schizophr Res 2007 Jan;89(1-3):129-139. [CrossRef] [Medline]
    57. Álvarez-Jiménez M, Gleeson JF, Henry LP, Harrigan SM, Harris MG, Killackey E, et al. Road to full recovery: longitudinal relationship between symptomatic remission and psychosocial recovery in first-episode psychosis over 7.5 years. Psychol Med 2011 Aug 19;42(3):595-606. [CrossRef]
    58. Alvarez-Jimenez M, Gleeson J, Henry L, Harrigan S, Harris M, Amminger G, et al. Prediction of a single psychotic episode: a 7.5-year, prospective study in first-episode psychosis. Schizophr Res 2011 Feb;125(2-3):236-246. [CrossRef] [Medline]
    59. Sanatkar S, Baldwin PA, Huckvale K, Clarke J, Christensen H, Harvey S, et al. Using cluster analysis to explore engagement and e-attainment as emergent behavior in electronic mental health. J Med Internet Res 2019 Nov 28;21(11):e14728 [FREE Full text] [CrossRef] [Medline]
    60. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res 2019 Sep 25;21(9):e14567 [FREE Full text] [CrossRef] [Medline]
    61. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res 2018 Jun 06;20(6):e199 [FREE Full text] [CrossRef] [Medline]
    62. Ryan RM, Deci EL. Self-Determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness. New York: Guilford Publications; 2018.
    63. Thomas EC, Zisman-Ilani Y, Salzer MS. Self-determination and choice in mental health: qualitative insights from a study of self-directed care. Psychiatr Serv 2019 Sep 01;70(9):801-807 [FREE Full text] [CrossRef] [Medline]
    64. Hanley T. The working alliance in online therapy with young people: preliminary findings. Br J Guid Counsel 2009 Aug;37(3):257-269. [CrossRef]
    65. Laws KR, Darlington N, Kondel TK, McKenna PJ, Jauhar S. Cognitive Behavioural Therapy for schizophrenia - outcomes for functioning, distress and quality of life: a meta-analysis. BMC Psychol 2018 Jul 17;6(1):32 [FREE Full text] [CrossRef] [Medline]
    66. Polese D, Fornaro M, Palermo M, De Luca V, de Bartolomeis A. Treatment-resistant to antipsychotics: a resistance to everything? Psychotherapy in treatment-resistant schizophrenia and nonaffective psychosis: a 25-year systematic review and exploratory meta-analysis. Front Psychiatry 2019 Apr 17;10:210 [FREE Full text] [CrossRef] [Medline]
    67. Thorisdottir IE, Sigurvinsdottir R, Asgeirsdottir BB, Allegrante JP, Sigfusdottir ID. Active and passive social media use and symptoms of anxiety and depressed mood among Icelandic adolescents. Cyberpsychol Behav Soc Netw 2019 Aug;22(8):535-542. [CrossRef] [Medline]
    68. Pung A, Fletcher SL, Gunn JM. Mobile app use by primary care patients to manage their depressive symptoms: qualitative study. J Med Internet Res 2018 Sep 27;20(9):e10035 [FREE Full text] [CrossRef] [Medline]
    69. Zilcha-Mano S, Roose SP, Barber JP, Rutherford BR. Therapeutic alliance in antidepressant treatment: cause or effect of symptomatic levels? Psychother Psychosom 2015 Mar 28;84(3):177-182 [FREE Full text] [CrossRef] [Medline]
    70. D'Alfonso S, Santesteban-Echarri O, Rice S, Wadley G, Lederman R, Miles C, et al. Artificial intelligence-assisted online social therapy for youth mental health. Front Psychol 2017 Jun 02;8:796 [FREE Full text] [CrossRef] [Medline]

    ‎
    EPPIC: Early Psychosis Prevention and Intervention Centre
    FEP: first-episode psychosis
    FESFS: First Episode Social Functioning Scale
    MOST: Moderated Online Social Therapy
    PSP: Personal and Social Performance Scale
    RCT: randomized controlled trial
    SDT: self-determination theory
    SEI: specialist early intervention
    TAU: treatment as usual

    Edited by J Torous; submitted 31.03.21; peer-reviewed by L Balcombe, R Lundin; comments to author 01.06.21; revised version received 18.06.21; accepted 10.01.22; published 07.04.22

    Copyright

    ©Shaunagh O'Sullivan, Lianne Schmaal, Simon D'Alfonso, Yara Jo Toenders, Lee Valentine, Carla McEnery, Sarah Bendall, Barnaby Nelson, John F Gleeson, Mario Alvarez-Jimenez. Originally published in JMIR Mental Health (https://mental.jmir.org), 07.04.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.