Background: Digital mental health interventions offer unique advantages, and research indicates that these interventions are effective for a range of mental health concerns. Although these interventions are less established for individuals with serious mental illnesses, they demonstrate significant promise. A central consideration in traditional face-to-face therapies is the therapeutic alliance, whereas the nature of a digital therapeutic alliance and its relationship with outcomes requires further attention, particularly for individuals with serious mental illnesses.
Objective: This narrative review aims to encourage further consideration and critical evaluation of the therapeutic alliance in digital mental health, specifically for individuals with serious mental illnesses.
Methods: A narrative review was conducted by combining 3 main areas of the literature: the first examining the evidence for digital mental health interventions for serious mental illnesses, the second illuminating the nature and role of the therapeutic alliance in digital interventions, and the third surrounding practical considerations to enhance a digital therapeutic alliance.
Results: Results indicated that a therapeutic alliance can be cultivated in digital interventions for those with serious mental illnesses, but that it may have unique, yet-to-be-confirmed characteristics in digital contexts. In addition, a therapeutic alliance appears to be less directly associated with outcomes in digital interventions than with those in face-to-face therapies. One possibility is that the digital therapeutic alliance is associated with increased engagement and adherence to digital interventions, through which it appears to influence outcomes. A number of design and implementation considerations may enhance the digital therapeutic alliance, including human support and technological features.
Conclusions: More research is required to further understand the nature and specific role of a therapeutic alliance in digital interventions for serious mental illnesses, particularly in informing their design. This review revealed several key research priorities to advance the therapeutic alliance in digital interventions.
A therapeutic alliance is considered fundamental to the success of face-to-face psychological therapies but is an underrecognized consideration in digital mental health . Given that digital mental health research and implementation is rapidly expanding, further exploration of the nature and role of a therapeutic alliance in this context is essential. The driving aim of this narrative review was to encourage further consideration and critical evaluation of a therapeutic alliance in digital mental health, particularly for serious mental illnesses. This review begins with a discussion of the current status of the literature regarding digital interventions and, specifically, those for serious mental illnesses. The following section highlights the relevance of a digital therapeutic alliance and the differences in the traditional conceptualization of therapeutic alliance. Next, the review focuses on conceptual issues surrounding a digital therapeutic alliance in the presence and absence of human support. Finally, practical considerations for enhancing digital therapeutic alliance are explored.
To achieve the aims of this review, three areas of literature were integrated. The first was literature examining the potential of digital mental health interventions for serious mental illnesses, the second was literature illuminating the role of the therapeutic alliance in digital interventions, and the third providing practical considerations with the potential to enhance the digital therapeutic alliance. Digital mental health interventions can be defined as “interventions that provide information, support and therapy (emotional, decisional, behavioural and neurocognitive) for physical and/or mental health problems via a technological or digital platform” . This paper will refer to mental health interventions with (supported) or without (unsupported) human support, accessed via mobile health or electronic health platforms. Given that the focus of this review was not contexts where a human delivers the majority of the intervention, including using digital means to deliver traditional face-to-face therapy sessions via the web or combinations of face-to-face and digital interventions (blended therapy), these were excluded where identified. The focus of this review was intervention studies with participants with a psychotic spectrum or bipolar disorder (BD) or any author-defined serious mental illness; however, where minimal data about serious mental illnesses are available, the broader mental health literature will be drawn upon.
Potential Benefits of Digital Interventions
There are several obvious benefits of digital interventions to enhance mental health, such as overcoming logistical barriers and reducing therapeutic costs [- ]. Indeed, some studies have suggested that these offer benefits not attainable through traditional therapy, such as the extension of the therapeutic hour, accessibility when and where users desire, and the provision of anonymity [ , ]. For these reasons, research into digital interventions is a burgeoning area, with many programs developed and evaluated in recent years.
Specific Benefits of Digital Interventions for Serious Mental Illnesses
Perceived stigma, lack of insight, and lack of trust in available face-to-face treatments have been cited as key reasons for individuals with serious mental illnesses not seeking support for mental health concerns [, ]. In addition, the symptoms and sequelae of BD and psychotic disorders, such as positive and negative symptoms, cognitive difficulties, general psychopathology, and poor social adjustment, are key predictors of poor adherence and engagement in face-to-face therapy [ - ]. Digital interventions offer the potential to address some of these specific barriers, for example, by increasing accessibility and a sense of autonomy and enabling self-pacing and reviewing of therapeutic content. Additional advantages include the provision of real-time and longitudinal patient data, potentially leading to a more accurate diagnosis and treatment decisions, moment-to-moment monitoring, and timely interventions [ ]. Furthermore, digital interventions may be uniquely positioned to target the multilevel risk factors associated with poor physical health and mortality in serious mental illnesses [ ]. Indeed, personal preference may mean that self-guided interventions are ideal for some [ ].
Use of Digital Interventions
An additional layer of stigma that individuals with serious mental illnesses may experience is the assumption that they cannot or do not use digital devices for their mental health. A digital divide exists in mental health, wherein individuals experiencing serious mental illnesses are excluded from receiving support through digital means because of a lack of access, skills, or confidence . However, evidence suggests that this exclusion is declining [ ], particularly within psychotic disorders and BD [ , ]. However, developing digital interventions for individuals experiencing serious mental illnesses can introduce additional challenges because of illness-related factors, such as cognitive impairments or mistrust, thus requiring specific design considerations [ , ].
Effectiveness of Digital Interventions
Overall, digital interventions for improving mental health have demonstrated significant promise. For example, 3 meta-analyses demonstrated that supported digital interventions for a range of mental health concerns were comparable in efficacy (moderate effect sizes) with face-to-face psychological interventions [- ]. Currently, the greatest support exists for the effectiveness of digital interventions for depression and anxiety disorders, whereas data for other clinical disorders are fragmented at this stage. For example, a recent narrative review of digital interventions reported large effect sizes for participants with major depressive disorder, anxiety disorders, obsessive-compulsive disorder, and posttraumatic stress disorder and small effect sizes for eating disorders or problematic alcohol use [ ].
Effectiveness of Digital Interventions for Serious Mental Illnesses
The findings of 6 recent systematic reviews of digital interventions for serious mental illnesses demonstrate that these are feasible and acceptable, with preliminary indications that these may be effective for symptoms and cognitive and social outcomes [- ]. However, in each review, insufficient data were available to draw firm conclusions. Specifically, in a systematic review including individuals with psychosis, results provided preliminary evidence that these may have benefits for positive psychotic symptoms and depression, hospital admissions, medication adherence, socialization, and social connectedness [ ]. For example, the recent Actissist proof-of-concept trial, in a small sample of individuals with early psychosis, demonstrated feasibility and acceptability through high use and adherence as well as symptom improvements in favor of the intervention group [ ]. The handful of digital interventions for BD evaluated to date have produced inconsistent results. A systematic review of these studies showed that while these appear to be feasible and acceptable intervention options, inconsistent evidence was found for their effectiveness for symptoms or recovery outcomes such as quality of life [ ]. The current Canadian Network for Mood and Anxiety Treatments and International Society for Bipolar Disorders treatment guidelines conclude that, to date, there is insufficient evidence to support the use of digital interventions as adjunctive therapies for BD [ ]. However, multiple trials are underway, which will advance this literature [ - ].
The original conceptualization of the therapeutic alliance by Bordin  included 3 components: (1) the bond between the client and therapist, (2) agreement on the tasks directed toward improvement, and (3) agreement on therapeutic goals. This pantheoretical conceptualization has been widely adopted; however, there are various understandings of the therapeutic alliance. For example, Rogers [ ] described the ideal qualities of a therapeutic relationship as acceptance, empathic understanding, and congruence, arguing that it is via these facilitative conditions that growth is achieved in therapy. An additional conceptualization, which forms the basis of the widely used Agnew Relationship Measure (ARM), comprises bond, partnership, confidence, openness, and client initiative [ ].
Therapeutic Alliance and Outcomes in Face-to-Face Therapy
A number of meta-analyses point to a modest but reliable relationship between the quality of therapeutic alliance and outcomes of face-to-face therapy, with effect sizes typically in the moderate range [- ]. Some research also supports the notion of a causal relationship between therapeutic alliance and outcomes in serious mental illnesses, for example, increasing the number of sessions attended was only beneficial in the presence of a strong therapeutic alliance for individuals experiencing psychosis [ ]. Two systematic reviews focusing on psychosis found some evidence that the therapeutic alliance was related to reductions in psychotic symptoms, hospitalizations, and self-esteem outcomes [ ] as well as functioning and treatment adherence [ ]. Similarly, in BD, therapeutic alliance have been linked with decreased stigma, more positive attitudes toward medication, and, less conclusively, a reduction in symptoms [ ].
Conceptualizing the Therapeutic Alliance in Digital Interventions
Given that the therapeutic relationship is a necessary (and some argue, sufficient) component of change in traditional face-to-face psychotherapies, a significant reservation toward digital interventions is the loss or, at a minimum, the modification of the therapeutic relationship. Some commenters have started to consider whether and, if so, how the therapeutic alliance plays a role in digital interventions . Indeed, the James Lind Alliance identification of key priorities in advancing digital interventions included concerns relating to the therapeutic alliance in the top 10 issues raised [ ]. The definition by Bordin has been transplanted to web-based environments, most commonly, digital interventions with human support components. However, it cannot be presumed that traditional conceptualizations of therapeutic alliance transfer to digital environments nor that different digital environments foster therapeutic alliance in equivalent ways.
The dimensions of the therapeutic alliance may differ in digital environments. A systematic review identified additional themes of availability, indicating how freely and conveniently accessible the digital intervention is, and interactivity, indicating the degree to which personalization and feedback based on user input is provided and to which the user feels in control . These themes support the possibility of a perceived bidirectional relationship between a user and a digital system, in which automated aspects of the experience could emulate a reciprocal, trusted relationship. A qualitative analysis mirrored the importance of these domains within the therapeutic alliance for participants. For example, one participant identified that automated personalization helped them understood and conversely, another user would have perceived a relationship if the site responded intelligently [ ]. Furthermore, following an examination of a supported digital intervention for carers, users perceived automated feedback, designed to emulate human communication, as supportive and helpful [ ]. More work is needed to explore therapeutic alliance features and domains within digital contexts. Nonetheless, these investigations shed some light on the nature of the alliance in digital contexts, providing a provisional characterization of factors relevant to the alliance in digital contexts.
In parallel, the measurement of the therapeutic alliance requires specific consideration for digital environments. Simply replacing therapist with program or app in existing measures may fail to account for the complexity of the therapeutic alliance in digital interventions and parcel out the relative contributions of human-human and human-technology relationships. Researchers are beginning to acknowledge this; for example, WAI-Tech, a digital adaptation of the frequently used alliance instrument, the Working Alliance Inventory (WAI), includes reworded items in the bond subscale, omitting the human element . Another adaptation applies to supported contexts, anchoring the task and goals subscales to the intervention and the bond subscale to the therapist [ ]. Similarly, Berry et al [ ] adapted the ARM for digital contexts by consulting end users and mental health professionals. In addition, the recently developed Enlight measure was specifically designed to assess the features of digital interventions, including those related to therapeutic alliance [ ]. These measures require further validation but represent a starting point for capturing the unique qualities of the digital therapeutic alliance.
Therapeutic Alliance Ratings in Digital Interventions
Several recent reviews have demonstrated that client ratings of the therapeutic alliance in various digital interventions are of similar magnitude to those found in face-to-face therapies [, - ], including a small number of samples with serious mental illnesses [ ]. Cavanagh et al [ ] suggest that common factors cultivated within the face-to-face therapeutic alliance, such as hope, empowerment, credibility, expectancy, perspective, and emotional processing, might be achieved within digital interventions through a combination of human support components and the intervention itself, a triadic alliance.
Role of Human Supporters in Digital Interventions
Digital interventions vary with regard to their level of professional support, including no support (entirely self-managed interventions); minimal administrative or technical support; and tailored, regular support from a professional . The frequency and type of support also ranges, but most frequently involve regular asynchronous message contact, typically with a professional, aimed at supporting participants’ engagement and progress through the program’s components [ ]. Similarly, the content and model of support vary, and this may be an additional factor that impacts effectiveness and engagement [ ].
A key issue when conceptualizing the therapeutic alliance in the digital context is whether this is dependent upon a relationship with a human supporter, and therefore whether a therapeutic alliance can be cultivated within wholly unsupported digital environments. In general, ratings of the therapeutic alliance are lower in unsupported interventions. For example, as shown in[ , - ], 2 recent digital interventions for depression demonstrated positive ratings of the therapeutic alliance, which were lower than those in face-to-face versions [ , ]. On the other hand, qualitative interview participants largely rejected the idea of a relationship with the digital intervention, despite indications in their responses that they were experiencing alliance-like processes [ ].
Digital interventions appear to be more effective when support is offered. The findings of multiple meta-analytic and systematic reviews suggest that digital interventions with support tend to have greater effect sizes than those without human support [, - ] for outcomes, including depression and anxiety symptoms, and general well-being. Several meta-analyses have shown that human support moderates the effectiveness of digital interventions [ , ]. Following a systematic review of factors influencing the successful implementation of digital interventions for those with serious mental illnesses, a key recommendation was the inclusion of human support elements [ ].
In addition, supported digital interventions have comparably smaller attrition rates and greater engagement, as demonstrated in systematic reviews [, ]. For example, a study evaluating predictors of adherence to digital interventions for individuals experiencing psychosis reported that support was associated with greater adherence [ ]. Similarly, a recent study compared a digital intervention with and without support for individuals with a history of psychosis, finding that those receiving support engaged significantly more with the site, across a range of engagement parameters [ ]. Furthermore, the inclusion of peer coaching within a digital intervention for BD significantly enhanced engagement and adherence [ ].
However, it should be noted that some studies involving direct comparisons of supported and unsupported interventions have failed to demonstrate significant differences in effectiveness and/or adherence [- ], with authors questioning whether a degree of human support is inherent under randomized controlled trial (RCT) conditions, while cautioning against interpreting these findings as evidence that support is not necessary. For example, Berger et al [ ] found no differences in the effectiveness of a digital intervention for social anxiety across a range of clinical outcomes between unsupported, minimally supported, and flexibly (on-demand) supported programs; however, participants had access to peer support in this program, potentially providing relational benefits. More direct comparisons are required to draw definitive conclusions.
What Kind of Support and How Often?
Although it is evident that support is beneficial within digital interventions, little research has been conducted on the optimal form of support. A small group of studies have attempted to identify whether comparable benefits are attained with more versus less intensive (and, therefore, cost- and resource-effective) support. Klein et al  compared low (once weekly) and high (thrice weekly) frequency email support within a CBT digital intervention for panic disorder. Comparable symptom improvement, perceptions of a therapeutic alliance, and attrition were reported between conditions, suggesting that increased frequency of contact was neither a moderator of these outcomes nor essential to engagement. A recent review of digital interventions for psychosis similarly found that there was a little difference in adherence between interventions with low-, high-, or very high–frequency contact with supports [ ].
Conversely, Palmqvist et al  reported a positive correlation between the amount of therapist contact in minutes and the effect sizes reported in 15 (supported and unsupported) digital interventions. A more recent systematic review of physical and mental health interventions with and without support reported that a higher frequency of support predicted better adherence [ ]. One explanation for these contrasting findings is that the latter studies included unsupported interventions in their analyses, potentially skewing results, that is, these differences may reflect differences between no support versus support, rather than frequency. More research is required to unravel these contradictory findings and identify optimal levels of support and to examine these associations within serious mental illnesses.
Other potential influences on the impact of human support are the type of support or supporter. However, data to date do not support this conclusion. Lindner et al  found no differences in effectiveness between telephone and email support in a small trial. Baumeister et al [ ] found no evidence that the qualifications of supporters influenced the outcomes in their systematic review. Similarly, the level of training did not impact the effectiveness of a digital intervention for depression [ ]. Furthermore, a meta-analysis by Gellatly et al [ ] failed to demonstrate any impact of the number of sessions, level of training, the content of the guidance, or mode of contact on therapeutic outcomes.
Support within digital interventions may be important for several reasons. For example, individuals with BD identified that managing procrastination and motivation issues were likely to be the primary benefits of support within a digital intervention . Participants in an unsupported CBT digital intervention for depression echoed this, expressing the desire for support to assist with discipline and motivation [ ]. Similarly, a meta-analysis of digital interventions for depression and anxiety disorders found that clients reporting lower motivation were less likely to benefit from interventions without support [ ]. Furthermore, 2 qualitative analyses following digital interventions (for generalized anxiety disorder and depression) revealed that personalized support led to improved motivation to engage while fulfilling the needs for relatedness [ , ]. Therefore, a vital function of support within digital interventions may enhance motivation and a sense of relatedness.
Association of Therapeutic Alliance to Outcomes in Digital Interventions, With and Without Support
Although it has been established that the therapeutic alliance can be experienced in digital interventions, links to outcomes are less clear than in face-to-face therapies.
Supported Digital Interventions
A handful of studies on supported digital interventions have reported that ratings of therapeutic alliance are related to treatment outcomes. The first was a small secondary analysis from an RCT targeting anxiety disorders . Ratings on a modified version of the WAI correlated with outcomes, specifically the degree of improvement in well-being and symptoms. In addition, Herbst et al [ ] found that WAI ratings were associated with symptom reduction in a supported digital intervention for obsessive compulsive disorder (OCD). Following another OCD digital intervention in a larger sample, a therapeutic alliance was the best predictor of response to the intervention [ ]. A systematic review aiming to examine whether a therapeutic alliance is associated with mental health outcomes found an association only within these 3 studies [ ]. Two further studies demonstrated that, following small trials of a supported digital intervention for posttraumatic stress disorder (PTSD), ratings on the WAI were associated with reductions in PTSD symptoms [ , ].
Conversely, multiple studies have failed to find associations between the therapeutic alliance and treatment outcomes in supported digital interventions. For example, Andersson et al  found that high ratings on the WAI in 3 supported digital interventions did not correlate with change scores across outcome measures for depression, generalized anxiety disorder, and social anxiety disorder. Furthermore, Hadjistavropoulos et al [ ] reported that ratings were unrelated to outcomes of supported digital interventions for either depression or generalized anxiety disorder. Similarly, Preschl et al [ ] found that high WAI ratings, equivalent to the face-to-face comparison group, were not associated with treatment outcomes following a supported digital intervention for depression. Some quality issues should be noted here, specifically that these analyses were mostly secondary analyses with small samples and no controls, and, therefore, these findings require validation. In addition, studies have measured the alliance at different times (midtreatment [ , ], postintervention [ , ], or both [ , , ]), whereas prior face-to-face research has indicated that the timing of alliance measurement influences its relationship with outcomes [ ]. These inconsistencies may partly explain the discrepancies in the findings.
Unsupported Digital Interventions
In a small, open trial of an unsupported digital intervention, modified ARM ratings were lower than in face-to-face CBT and showed no associations with depression outcomes . In another study, ratings on the WAI-Tech for a digital intervention were similar to those on the WAI in a face-to-face (treatment as usual) group but were not associated with the outcome: cocaine abstinence [ ]. A further study found that ratings on the ARM were not correlated with outcomes for those with depression and anxiety, following an unsupported digital intervention [ ]. As mentioned above, these are small studies, including an open trial [ ] and secondary analyses [ , ], and the alliance was heterogeneously assessed, raising questions about reliability.
Taken together, there appears to be conflicting evidence that the therapeutic alliance links with outcomes in supported digital interventions, and no evidence of such an association is found in unsupported digital interventions. It appears that although the therapeutic relationship is robust to the reduced contact and distance in digital interventions, it may be less intimately tied to outcomes than in traditional therapies. Importantly, for this review, minimal data were available regarding these relationships for serious mental illnesses.
Association of Therapeutic Alliance With Engagement and Adherence in Digital Interventions, With and Without Support
Although evidence relating to a direct relationship between ratings of therapeutic alliance and treatment outcomes is inconclusive for digital interventions, one possibility is that the therapeutic alliance is associated with increased engagement and adherence, in turn, influencing outcomes . It should be noted that engagement and adherence are often used interchangeably or poorly defined, limiting conclusions. In this review, engagement will refer to measures of the amount, frequency, or depth of program usage, whereas adherence will refer to whether participants met an a priori metric of intended usage or dose; however, we are limited to the information presented. For example, a real-world analysis of a range of apps and sites available to the public, with varying support, demonstrated that ratings of the therapeutic alliance (measured using the Enlight measure) were associated with increased user engagement [ ].
Alliances and Engagement in Supported Interventions
Considering adherence, a study by Hargreaves et al  demonstrated that WAI ratings were the most significant predictor of adherence to a supported digital intervention for individuals experiencing psychosis. In another supported intervention, those that completed the intervention provided higher alliance ratings (in a sample with PTSD) [ ]. A further study found a relationship between alliance ratings and engagement with intervention features in adolescents [ ].
Alliances and Engagement in Unsupported Interventions
Clarke et al  did not find a direct relationship between therapeutic alliance (assessed with the ARM) and outcomes, while a significant association was discovered between the alliance and level of engagement with the unsupported digital intervention. In a further supported intervention, the WAI-Tech goal and bond subscales were positively correlated with the number of modules completed [ ].
As can be seen, across the few studies that considered the possibility of a relationship between the alliance and engagement in digital interventions, all have reported an association. However, conclusions are complicated by the small number of studies and the range of conceptualizations of engagement and/or adherence therein.
Association of Engagement and Adherence to Outcomes in Digital Interventions
The above findings are especially compelling given that adherence and engagement are themselves associated with improved outcomes in digital interventions. For example, a meta-analysis demonstrated that adherence to both the intervention and the study was associated with a range of behavioral and health outcomes in digital interventions . Furthermore, a systematic review showed that a broad range of adherence measures (many of which align with our definition of engagement) was associated with physical and psychological outcomes in digital interventions [ ]. Associations with adherence have been demonstrated for a range of outcomes, including depression [ , ] and anxiety [ , ].
Engagement with digital interventions has also been linked to outcomes. Across outcomes such as depression and anxiety [- ], greater changes in symptoms have been associated with better engagement with the site. Similarly, in a program for preventing eating disorders, the duration and extent of program use predicted some outcomes [ ].
These findings suggest that a therapeutic alliance is associated with increased engagement with and adherence to digital interventions, in turn, influencing outcomes. However, this explanation represents one possible interpretation of these limited data and requires direct examination. Given to the inconsistencies within this literature , more work is needed to identify the engagement and adherence metrics with the most importance. In addition, no identified studies investigating links between outcomes, adherence, and engagement included samples with serious mental illnesses.
Conceptualizing the Therapeutic Alliance in Unsupported Digital Interventions
Although it appears that human support is beneficial for therapeutic alliance, in many cases, human support is not practical or desirable. Some users, including those with serious mental illnesses, report a preference for unsupported interventions, with benefits such as the reduced potential for judgment and increased honesty [, ]. How the definition by Bordin [ ] on therapeutic alliance can apply to unsupported interventions is not yet clear, but it has been suggested that therapeutic goals and therapeutic task components (or analogs thereof) may be cultivated within interactions between users and the intervention, while the bond element may be either entirely absent or modified [ ]. For example, while equivalent ratings on the remaining alliance subscales were reported, the bond subscale of the WAI was significantly lower in a digital intervention for cocaine dependence than in a face-to-face group [ ].
In the absence of human support, it may be more important to ensure that unsupported digital interventions incorporate automated features that resemble a bidirectional therapeutic relationship, such as the communication of empathy, responsiveness, and supporting the user’s motivation. Barazzone et al  examined the content of 3 widely used CBT programs to investigate the extent to which they incorporated key features for the establishment, development, and maintenance of a therapeutic alliance between the user and the program. They concluded that the programs exhibited substantial evidence of therapeutic alliance features, such as empathy and acceptance, and the negotiation of goals, providing feedback and building confidence in the program’s effectiveness, and rupture prevention and repair by encouraging users to return to the program after a break. Similarly, Holter et al [ ] describe their attempts to cultivate a therapeutic alliance in a smoking cessation digital intervention, simulating a therapeutic alliance by allowing users to negotiate goals and using a conversational agent to implement human strategies such as empathy, interactivity (via remembering previous communications and tailoring), and humor. These authors did not assess the therapeutic alliance, its relationship with program features, or outcomes. However, Bickmore et al [ ] evaluated the impact of introducing a relational agent that simulated alliance-promoting behaviors such as social dialogue, empathetic feedback, process comments, humor, and nonverbal communication, alongside a fitness digital intervention. The bond subscale was rated significantly higher within the group that had contact with the agent, whereas the overall WAI scores did not differ significantly.
Furthermore, additional provisional characteristics of the digital therapeutic alliance, availability, and interactivity, are not grounded in human support  and speak to the unique strengths of digital interventions. Technology is ideally positioned to be accessible when and wherever users require it, and a number of technological features can be optimized to cultivate interactivity, that is, a bidirectional relationship might be effectively emulated between a user and a digital intervention, with automated features. In addition, people may form attachments to smartphones, similar to an alliance [ , ], although users appear hesitant to call this a relationship [ ]. Although human presence may be an important predictor of the therapeutic alliance in digital interventions, the experience of human presence may not require direct human contact. For example, the object-relations theory explains how devices may act as proxies for relationships with caregivers (the humans behind the intervention [ ]).
Examining which technological features are best positioned to foster the therapeutic alliance in both supported and unsupported digital interventions is therefore an important next step, and we propose that exploring the additional provisional characteristics of availability and interactivity may represent a significant advancement.
Persuasive Systems Design: Potential for Digital Therapeutic Alliance
Particular consideration of the features of the intervention and their ability to promote behavior change  may provide direction for enhancing the therapeutic alliance. Persuasive systems design (PSD) is a broad term used to describe the technological features of a digital intervention specifically intended to motivate behavior or attitudinal change [ , ]. The study findings indicate that incorporating PSD features within interventions can be effective in motivating individuals toward specific goals across physical health [ , ], mental health [ , ], exercise [ , ], and educational [ ] domains. The implementation of PSD with the specific intention of promoting the digital therapeutic alliance has not been examined to date; however, a number of PSD features [ ] are likely to promote the development of a therapeutic alliance between users and digital platforms. This section provides an overview of the PSD factors that are likely to be relevant to the fostering of a therapeutic alliance in the digital space and comments on the potential role of human supporters, where applicable. presents an example of how aspects of the digital intervention itself and interaction with human supporters may promote specific, provisional components of the digital therapeutic alliance.
Tailoring and Personalization as Primary Task Supports
Tailoring systems and personalizing messages to users are important components of system design . An example of tailoring is to suggest specific content based on what is known about users’ preferences or background, whereas personalization might reflect the person’s name or information about their medications, functioning, or previous responses [ ].
Empirical research findings have indicated that when intervention content is matched with users’ psychosocial and behavioral characteristics, it is perceived as personally relevant, which enhances engagement  in line with the elaboration likelihood model [ ]. Conversely, a 2016 review [ ] indicated that the most commonly cited reason for low adherence within the digital intervention was a perception of impersonal or irrelevant content.
Therefore, tailoring and personalization may help foster a digital therapeutic alliance via congruency between the users’ personal needs and goals and interactions with the intervention. In addition, tailoring and personalization speak directly to the provisional digital therapeutic alliance’s characteristic of interactivity. For example, tailoring the intervention to a person’s stage of recovery in face-to-face therapy and not assuming motivation to change is predictive of a strong therapeutic alliance and outcomes . In this vein, it has been argued that individually tailored content or responses within digital interventions enhance the sense of relationship with the platform, whether supported or unsupported [ ]. Research findings provide some support for this claim, with qualitative accounts indicating that when users perceived an app as not tailored to their individual needs (eg, generic), the development of a relationship was hindered [ ]. This is an important consideration when developing digital interventions for individuals with serious mental illnesses, where concerted efforts to build trust that the treatment will address the clients’ own unique goals may provide a buffer against feelings of mistrust or coercion [ ].
Dialogue support is a PSD strategy that supports user-digital interactions deemed by users to be interpersonal or social interactions . Specific examples of dialogue support in digital interventions include offering positive feedback (praise), reflecting users’ goals and tasks (reminders), linking users with peers via forums (social role), or providing recommendations for appropriate content (suggestions) [ , ]. Sustained engagement with a digital intervention is likely when the intervention offers ongoing interaction that is relevant, motivating, and tailored to users’ needs [ ]. Review findings have demonstrated that adherence to digital interventions, in general, is contingent on the extent to which dialogue support elements are utilized [ ]. Similarly, a recent review showed that a greater number of dialogue support elements was associated with beneficial outcomes [ ].
Research findings also demonstrate that the quality of the therapeutic alliance is contingent on the perceived quality of the interaction and, therefore, implementing responsive dialogue support features within digital interventions should be a consideration in fostering a digital therapeutic alliance . Improved dialogue between users and digital interventions could not only assist users to achieve their goals and foster a sense of interactivity but also enhance self-efficacy [ ]. This is an important consideration for individuals with serious mental illnesses because research findings indicate that individuals who perceive greater control over their lives (ie, internal locus of control and greater self-efficacy) are more positive about the therapeutic alliance [ ]. Furthermore, although it has not been studied in the context of digital interventions, an internal locus of control is associated with increased treatment motivation, compliance and treatment adherence, and better treatment outcomes in individuals with serious mental illnesses [ ].
System credibility or credibility support refers to how digital interventions incorporate technological features that convince the user that the system is credible [, ]. Examples of how credibility support can be used in digital interventions include providing evidence-based content, expert moderation (eg, clinician support), and explicit, credible third-party endorsements [ ]. Credibility support is associated with perceived authority, expertise, and trustworthiness among users of digital interventions, and findings show that its deliberate implementation is also associated with engagement [ ].
Implementing credibility support features to increase users’ perceptions of expertise and trustworthiness is likely to help strengthen therapeutic alliance. For example, the bond between therapists and clients in face-to-face settings is contingent on the client’s confidence in the therapist’s competence . Scant literature has examined this phenomenon within a digital context; however, Mackie et al [ ] examined the effectiveness of a mobile-based intervention to treat harmful substance use in veterans presenting with self-harm and found that trust was associated with the quality of the digital therapeutic alliance, whereas damaged trust can lead to disengagement. This is a particularly important clinical consideration for individuals with serious mental illnesses who are often difficult to engage in ongoing treatment and have high dropout rates [ ].
Persuasive design strategies in the social support category refer to technological strategies that motivate users by leveraging social influence (eg, cooperation, social facilitation, and social comparison) . Theoretically, individuals will be more motivated to perform a target behavior if they can use a system to observe (eg, social facilitation) or compare themselves (eg, social comparison) with others performing the behavior and if they are provided with a means to cooperate or connect [ , ]. An example of how digital interventions can include technological features to facilitate social support include incorporating newsfeeds and/or social forums. There is some support to suggest that the inclusion of social networking facilities in digital interventions for individuals with serious mental illnesses can assist in fostering a sense of social connectedness [ , , ], a sense of cooperation (shared goals), and increased accountability toward treatment [ ].
Developing digital interventions that utilize social support technological features may also help foster a digital therapeutic alliance. For example, study findings relating to face-to-face therapy have shown an association between clients’ levels of perceived social support and their ratings of the therapeutic alliance . Research findings demonstrate that individuals with serious mental illnesses report benefits from interacting with peers on the web, including greater feelings of group belonging and social connectedness [ ]. In addition, as discussed previously, individuals with serious mental illnesses who may be uncomfortable with or avoidant of conventional social contact may perceive digital interactions as less threatening [ ]. Therefore, the facilitation of social support should be a design feature of digital interventions for individuals with serious mental illnesses.
There are a number of implications of these findings for the development and evaluation of digital interventions, specifically for designing interventions for individuals with serious mental illnesses. Specifically, there have been very few investigations into the role of specific design features in cultivating digital alliances. For example, if interactivity is confirmed as an important feature of the digital alliance, some specific design principles (such as tailoring and personalization strategies) are ideally positioned to promote users’ perceptions of interactivity. Furthermore, the unique nature of the alliance in digital contexts is likely to interact specifically with serious mental illnesses. For example, individuals who may be uncomfortable with or encounter barriers to interacting within traditional therapeutic encounters may be differentially receptive to forming relationships in digital contexts (even those without human support), and as noted, some individuals prefer accessing digital interventions over traditional forms of support.
The main limitations of this review were that no systematic search methods or formal bias and quality assessment methods were used, as the aim was a broad integration of relevant literature to identify key gaps for future research. Accordingly, there is a significant risk of selection bias. Therefore, the included studies cannot be presumed to represent the available literature, and firm conclusions are not warranted on these bases. In addition, in many included studies, the analyses that were included were secondary analyses (), and many were underpowered because of sample size, inflating type I and type II error risk. There may also be some conceptual issues impacting the results. For example, issues unique to the experiences of individuals with serious mental illnesses and their potential interactions with the relationships of interest should be considered. For example, social withdrawal, mistrust, or paranoia could interact with engagement and independently with an alliance.
More research is required to further understand the specific role of the therapeutic alliance in digital interventions for serious mental illnesses, particularly in informing their design. Specifically, based on the findings of this review, systematic reviews could examine the role of the therapeutic alliance in both engagement and outcomes of digital interventions. Furthermore, a great deal of research has provided data about the role of human support, largely indicating that it is beneficial (but not essential). However, further inquiry is needed to identify the optimal form, intensity, and role that should be taken and to examine nuances such as interactions with different presentations or populations, including those with serious mental illnesses. Separate to this, ways of maximizing therapeutic alliance that are independent of human support require further investigation, and new technologies present multiple promising avenues. The potential for PSD features to enhance the digital therapeutic alliance requires further investigation. Digital phenotyping and machine learning approaches, integrating physiological and behavioral signals to tailor interventions with the aim of enhancing outcomes, demonstrate promise, and for serious mental illnesses, the research in this area is growing [, - ]. However, the specific role of such technologies in cultivating digital therapeutic alliance remains to be unknown. Virtual reality (VR) is another area of growth in mental health; however, limitations to date inhibit the potential for exploring its impact on the therapeutic alliance, including that most VR settings require face-to-face contact with a human, and these technologies currently offer the minimal ability for interpersonal interactions within virtual environments [ ]. Research on the role of a therapeutic alliance in VR is limited, with preliminary studies showing that therapeutic alliance can be cultivated with these technologies [ ]. An additional line of research requiring further attention is concerned with the way digital interventions might be integrated with or enhance care models, such as stepped care, blended models, and virtual clinics, each with specific implications upon a therapeutic alliance [ ].
Digital mental health interventions continue to garner significant research attention, demonstrating effectiveness for a range of mental health outcomes and holding significant promise for those with serious mental illnesses. Although a long tradition of research has demonstrated that the therapeutic alliance is central to outcomes in face-to-face therapy, less emphasis has been placed on the therapeutic alliance in digital contexts. Overall, it appears that a therapeutic alliance can be cultivated in digital interventions, but it may have unique, yet to be confirmed, features in these contexts. With further investigation, it may emerge that the nature of the alliance is so divergent as to reflect a unique construct; however, at this juncture, we believe it is reasonable to consider relationships cultivated in digital context analogs of traditional, bidirectional therapeutic alliance, and hence, this term is used throughout this work. In addition, it appears likely that the therapeutic alliance is less directly linked with outcomes in digital interventions than in face-to-face therapies. Rather, it may be that the therapeutic alliance is associated with increased engagement with and adherence to digital interventions, which, in turn, leads to improved outcomes. Human support is an effective method of enhancing both engagement and therapeutic alliance; however, this is not always feasible or desirable. Alternatively, alongside or in lieu of human support, technological features may be capable of cultivating therapeutic alliance, especially when these are automated to emulate relational characteristics, but more research is needed to explore which features are most closely aligned with the improved therapeutic alliance and, ultimately, improved outcomes. Accordingly, several research priorities have been identified to advance this understanding.
Conflicts of Interest
Intervention characteristics and reported links between alliance, engagement, and outcomes.PDF File (Adobe PDF File), 210 KB
- Henson P, Wisniewski H, Hollis C, Keshavan M, Torous J. Digital mental health apps and the therapeutic alliance: initial review. BJPsych Open 2019 Jan;5(1):e15 [FREE Full text] [CrossRef] [Medline]
- Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual research review: digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry 2017 Apr;58(4):474-503. [CrossRef] [Medline]
- Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res 2012 Dec;12(6):745-764. [CrossRef] [Medline]
- Fenichel M, Suler J, Barak A, Zelvin E, Jones G, Munro K, et al. Myths and realities of online clinical work. Cyberpsychol Behav 2002 Oct;5(5):481-497. [CrossRef] [Medline]
- Proudfoot J, Klein B, Barak A, Carlbring P, Cuijpers P, Lange A, et al. Establishing guidelines for executing and reporting internet intervention research. Cogn Behav Ther 2011;40(2):82-97. [CrossRef] [Medline]
- Richards P, Simpson S. Beyond the therapeutic hour: an exploratory pilot study of using technology to enhance alliance and engagement within face-to-face psychotherapy. Brit J Guid Couns 2014 Aug 28;43(1):57-93. [CrossRef]
- Kessler RC, Berglund PA, Bruce ML, Koch JR, Laska EM, Leaf PJ, et al. The prevalence and correlates of untreated serious mental illness. Health Serv Res 2001 Dec;36(6 Pt 1):987-1007 [FREE Full text] [Medline]
- Smith TE, Easter A, Pollock M, Pope LG, Wisdom JP. Disengagement from care: perspectives of individuals with serious mental illness and of service providers. Psychiatr Serv 2013 Aug 1;64(8):770-775. [CrossRef] [Medline]
- Macbeth A, Gumley A, Schwannauer M, Fisher R. Service engagement in first episode psychosis: clinical and premorbid correlates. J Nerv Ment Dis 2013 May;201(5):359-364. [CrossRef] [Medline]
- Leclerc E, Noto C, Bressan RA, Brietzke E. Determinants of adherence to treatment in first-episode psychosis: a comprehensive review. Braz J Psychiatry 2015;37(2):168-176 [FREE Full text] [CrossRef] [Medline]
- Leclerc E, Mansur RB, Brietzke E. Determinants of adherence to treatment in bipolar disorder: a comprehensive review. J Affect Disord 2013 Jul;149(1-3):247-252. [CrossRef] [Medline]
- Aref-Adib G, McCloud T, Ross J, O'Hanlon P, Appleton V, Rowe S, et al. Factors affecting implementation of digital health interventions for people with psychosis or bipolar disorder, and their family and friends: a systematic review. Lancet Psychiatry 2019 Mar;6(3):257-266. [CrossRef]
- Martinez-Martin N, Insel TR, Dagum P, Greely HT, Cho MK. Data mining for health: staking out the ethical territory of digital phenotyping. NPJ Digit Med 2018;1:- [FREE Full text] [CrossRef] [Medline]
- Naslund JA, Aschbrenner KA. Digital technology for health promotion: opportunities to address excess mortality in persons living with severe mental disorders. Evid Based Ment Health 2019 Feb;22(1):17-22 [FREE Full text] [CrossRef] [Medline]
- Berry N, Lobban F, Bucci S. A qualitative exploration of service user views about using digital health interventions for self-management in severe mental health problems. BMC Psychiatry 2019 Jan 21;19(1):35 [FREE Full text] [CrossRef] [Medline]
- Bernard R, Sabariego C, Cieza A. Barriers and facilitation measures related to people with mental disorders when using the web: a systematic review. J Med Internet Res 2016 Jun 9;18(6):e157 [FREE Full text] [CrossRef] [Medline]
- Robotham D, Satkunanathan S, Doughty L, Wykes T. Do we still have a digital divide in mental health? A five-year survey follow-up. J Med Internet Res 2016 Nov 22;18(11):e309 [FREE Full text] [CrossRef] [Medline]
- Gay K, Torous J, Joseph A, Pandya A, Duckworth K. Digital technology use among individuals with schizophrenia: results of an online survey. JMIR Ment Health 2016 May 4;3(2):e15 [FREE Full text] [CrossRef] [Medline]
- Bauer R, Conell J, Glenn T, Alda M, Ardau R, Baune BT, et al. Internet use by patients with bipolar disorder: results from an international multisite survey. Psychiatry Res 2016 Aug 30;242:388-394. [CrossRef] [Medline]
- Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 2010 Dec;40(12):1943-1957. [CrossRef] [Medline]
- Saddichha S, Al-Desouki M, Lamia A, Linden IA, Krausz M. Online interventions for depression and anxiety - a systematic review. Health Psychol Behav Med 2014 Jan 1;2(1):841-881 [FREE Full text] [CrossRef] [Medline]
- Barak A, Hen L, Boniel-Nissim M, Shapira N. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. Inf Technol J 2008 Jul 3;26(2-4):109-160. [CrossRef]
- Ebert DD, van Daele T, Nordgreen T, Karekla M, Compare A, Zarbo C, et al. Internet- and mobile-based psychological interventions: applications, efficacy, and potential for improving mental health. Eur Psychol 2018 May;23(2):167-187. [CrossRef]
- Batra S, Baker RA, Wang T, Forma F, DiBiasi F, Peters-Strickland T. Digital health technology for use in patients with serious mental illness: a systematic review of the literature. Med Devices (Auckl) 2017;10:237-251 [FREE Full text] [CrossRef] [Medline]
- Berry N, Lobban F, Emsley R, Bucci S. Acceptability of interventions delivered online and through mobile phones for people who experience severe mental health problems: a systematic review. J Med Internet Res 2016 May 31;18(5):e121 [FREE Full text] [CrossRef] [Medline]
- Lawes-Wickwar S, McBain H, Mulligan K. Application and effectiveness of telehealth to support severe mental illness management: systematic review. JMIR Ment Health 2018 Nov 21;5(4):e62 [FREE Full text] [CrossRef] [Medline]
- Naslund JA, Aschbrenner KA, Bartels SJ. How people with serious mental illness use smartphones, mobile apps, and social media. Psychiatr Rehabil J 2016 Dec;39(4):364-367 [FREE Full text] [CrossRef] [Medline]
- Clarke S, Hanna D, Mulholland C, Shannon C, Urquhart C. A systematic review and meta-analysis of digital health technologies effects on psychotic symptoms in adults with psychosis. Psychosis 2019 Aug 12;11(4):362-373. [CrossRef]
- Rus-Calafell M, Schneider S. Are we there yet?!- a literature review of recent digital technology advances for the treatment of early psychosis. Mhealth 2020;6:3 [FREE Full text] [CrossRef] [Medline]
- Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res 2014 Jun;156(1):96-106. [CrossRef] [Medline]
- Bucci S, Barrowclough C, Ainsworth J, Machin M, Morris R, Berry K, et al. Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophr Bull 2018 Aug 20;44(5):1070-1080 [FREE Full text] [CrossRef] [Medline]
- Hidalgo-Mazzei D, Mateu A, Reinares M, Matic A, Vieta E, Colom F. Internet-based psychological interventions for bipolar disorder: review of the present and insights into the future. J Affect Disord 2015 Dec 1;188:1-13. [CrossRef] [Medline]
- Yatham LN, Kennedy SH, Parikh SV, Schaffer A, Bond DJ, Frey BN, et al. Canadian network for mood and anxiety treatments (CANMAT) and international society for bipolar disorders (ISBD) 2018 guidelines for the management of patients with bipolar disorder. Bipolar Disord 2018 Mar;20(2):97-170 [FREE Full text] [CrossRef] [Medline]
- Fletcher K, Foley F, Thomas N, Michalak E, Berk L, Berk M, et al. Web-based intervention to improve quality of life in late stage bipolar disorder (ORBIT): randomised controlled trial protocol. BMC Psychiatry 2018 Jul 13;18(1):221 [FREE Full text] [CrossRef] [Medline]
- Alvarez-Jimenez M, Bendall S, Koval P, Rice S, Cagliarini D, Valentine L, et al. HORYZONS trial: protocol for a randomised controlled trial of a moderated online social therapy to maintain treatment effects from first-episode psychosis services. BMJ Open 2019 Feb 19;9(2):e024104 [FREE Full text] [CrossRef] [Medline]
- Thomas N, Farhall J, Foley F, Rossell SL, Castle D, Ladd E, et al. Randomised controlled trial of a digitally assisted low intensity intervention to promote personal recovery in persisting psychosis: SMART-therapy study protocol. BMC Psychiatry 2016 Sep 7;16(1):312 [FREE Full text] [CrossRef] [Medline]
- Lim MH, Gleeson JF, Rodebaugh TL, Eres R, Long KM, Casey K, et al. A pilot digital intervention targeting loneliness in young people with psychosis. Soc Psychiatry Psychiatr Epidemiol 2020 Jul;55(7):877-889. [CrossRef] [Medline]
- McEnery C, Lim MH, Knowles A, Rice S, Gleeson J, Howell S, et al. Development of a moderated online intervention to treat social anxiety in first-episode psychosis. Front Psychiatry 2019;10:581 [FREE Full text] [CrossRef] [Medline]
- Bordin ES. The generalizability of the psychoanalytic concept of the working alliance. Psychol Psychother 1979;16(3):252-260. [CrossRef]
- Rogers C. Client-Centered Therapy: Its Current Practice, Implications and Theory. Oxford, England: Houghton Mifflin; 1951.
- Agnew-Davies R, Stiles WB, Hardy GE, Barkham M, Shapiro DA. Alliance structure assessed by the agnew relationship measure (ARM). Br J Clin Psychol 1998 May;37(2):155-172. [CrossRef] [Medline]
- Flückiger C, Del Re AC, Wampold BE, Horvath AO. The alliance in adult psychotherapy: a meta-analytic synthesis. Psychotherapy (Chic) 2018 Dec;55(4):316-340. [CrossRef] [Medline]
- Horvath AO, Del Re AC, Flückiger C, Symonds D. Alliance in individual psychotherapy. Psychotherapy (Chic) 2011 Mar;48(1):9-16. [CrossRef] [Medline]
- Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review. J Consult Clin Psychol 2000 Jun;68(3):438-450. [Medline]
- Goldsmith LP, Lewis SW, Dunn G, Bentall RP. Psychological treatments for early psychosis can be beneficial or harmful, depending on the therapeutic alliance: an instrumental variable analysis. Psychol Med 2015 Aug;45(11):2365-2373 [FREE Full text] [CrossRef] [Medline]
- Shattock L, Berry K, Degnan A, Edge D. Therapeutic alliance in psychological therapy for people with schizophrenia and related psychoses: a systematic review. Clin Psychol Psychother 2018 Jan;25(1):e60-e85. [CrossRef] [Medline]
- Browne J, Nagendra A, Kurtz M, Berry K, Penn DL. The relationship between the therapeutic alliance and client variables in individual treatment for schizophrenia spectrum disorders and early psychosis: narrative review. Clin Psychol Rev 2019 Jul;71:51-62. [CrossRef] [Medline]
- Andrade-González N, Hernández-Gómez A, Álvarez-Sesmero S, Gutiérrez-Rojas L, Vieta E, Reinares M, et al. The influence of the working alliance on the treatment and outcomes of patients with bipolar disorder: a systematic review. J Affect Disord 2020 Jan 1;260:263-271. [CrossRef] [Medline]
- Henson P, Peck P, Torous J. Considering the therapeutic alliance in digital mental health interventions. Harv Rev Psychiatry 2019;27(4):268-273. [CrossRef] [Medline]
- Hollis C, Sampson S, Simons L, Davies EB, Churchill R, Betton V, et al. Identifying research priorities for digital technology in mental health care: results of the James Lind alliance priority setting partnership. Lancet Psychiatry 2018 Oct;5(10):845-854. [CrossRef] [Medline]
- Hillier L. Exploring the Nature of the Therapeutic Alliance in Technology-based Interventions for Mental Health Problems. Lancaster EPrints. 2018. URL: https://eprints.lancs.ac.uk/id/eprint/127663/1/2018hilliermphil.pdf [accessed 2020-07-15]
- Lederman R, Gleeson J, Wadley G, D’alfonso S, Rice S, Santesteban-Echarri O, et al. Support for carers of young people with mental illness. ACM Trans Comput Hum Interact 2019 Feb 23;26(1):1-33. [CrossRef]
- Kiluk BD, Serafini K, Frankforter T, Nich C, Carroll KM. Only connect: the working alliance in computer-based cognitive behavioral therapy. Behav Res Ther 2014 Dec;63:139-146 [FREE Full text] [CrossRef] [Medline]
- Penedo JM, Berger T, Grosse Holtforth M, Krieger T, Schröder J, Hohagen F, et al. The working alliance inventory for guided internet interventions (WAI-I). J Clin Psychol 2020 Jun;76(6):973-986. [CrossRef] [Medline]
- Berry K, Salter A, Morris R, James S, Bucci S. Assessing therapeutic alliance in the context of mhealth interventions for mental health problems: development of the mobile agnew relationship measure (MARM) questionnaire. J Med Internet Res 2018 Apr 19;20(4):e90 [FREE Full text] [CrossRef] [Medline]
- Baumel A, Faber K, Mathur N, Kane JM, Muench F. Enlight: a comprehensive quality and therapeutic potential evaluation tool for mobile and web-based ehealth interventions. J Med Internet Res 2017 Mar 21;19(3):e82 [FREE Full text] [CrossRef] [Medline]
- Berger T. The therapeutic alliance in internet interventions: a narrative review and suggestions for future research. Psychother Res 2017 Sep;27(5):511-524. [CrossRef] [Medline]
- Pihlaja S, Stenberg J, Joutsenniemi K, Mehik H, Ritola V, Joffe G. Therapeutic alliance in guided internet therapy programs for depression and anxiety disorders - a systematic review. Internet Interv 2018 Mar;11:1-10 [FREE Full text] [CrossRef] [Medline]
- Sucala M, Schnur JB, Constantino MJ, Miller SJ, Brackman EH, Montgomery GH. The therapeutic relationship in e-therapy for mental health: a systematic review. J Med Internet Res 2012 Aug 2;14(4):e110 [FREE Full text] [CrossRef] [Medline]
- Cavanagh K, Millings A. (Inter)personal computing: the role of the therapeutic relationship in e-mental health. J Contemp Psychother 2013 Jul 17;43(4):197-206. [CrossRef]
- Newman MG, Szkodny LE, Llera SJ, Przeworski A. A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy? Clin Psychol Rev 2011 Feb;31(1):89-103. [CrossRef] [Medline]
- Schueller SM, Tomasino KN, Mohr DC. Integrating human support into behavioral intervention technologies: the efficiency model of support. Clin Psychol Sci Pract 2016 Nov 17;24(1):27-45. [CrossRef]
- Clarke J, Proudfoot J, Whitton A, Birch M, Boyd M, Parker G, et al. Therapeutic alliance with a fully automated mobile phone and web-based intervention: secondary analysis of a randomized controlled trial. JMIR Ment Health 2016 Feb 25;3(1):e10 [FREE Full text] [CrossRef] [Medline]
- Gega L, Smith J, Reynolds S. Cognitive behaviour therapy (CBT) for depression by computer vs therapist: patient experiences and therapeutic processes. Psychother Res 2013;23(2):218-231. [CrossRef] [Medline]
- Ormrod JA, Kennedy L, Scott J, Cavanagh K. Computerised cognitive behavioural therapy in an adult mental health service: a pilot study of outcomes and alliance. Cogn Behav Ther 2010;39(3):188-192. [CrossRef] [Medline]
- Bickmore T, Gruber A, Picard R. Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ Couns 2005 Oct;59(1):21-30. [CrossRef] [Medline]
- Nordgren LB, Carlbring P, Linna E, Andersson G. Role of the working alliance on treatment outcome in tailored internet-based cognitive behavioural therapy for anxiety disorders: randomized controlled pilot trial. JMIR Res Protoc 2013 Jan 18;2(1):e4 [FREE Full text] [CrossRef] [Medline]
- Herbst N, Franzen G, Voderholzer U, Thiel N, Knaevelsrud C, Hertenstein E, et al. Working alliance in internet-based cognitive-behavioral therapy for obsessive-compulsive disorder. Psychother Psychosom 2016;85(2):117-118. [CrossRef] [Medline]
- Andersson E, Ljótsson B, Hedman E, Enander J, Kaldo V, Andersson G, et al. Predictors and moderators of internet-based cognitive behavior therapy for obsessive–compulsive disorder: results from a randomized trial. J Obsess-Compuls Rel 2015 Jan;4:1-7. [CrossRef]
- Andersson G, Paxling B, Wiwe M, Vernmark K, Felix CB, Lundborg L, et al. Therapeutic alliance in guided internet-delivered cognitive behavioural treatment of depression, generalized anxiety disorder and social anxiety disorder. Behav Res Ther 2012 Sep;50(9):544-550. [CrossRef] [Medline]
- Hadjistavropoulos HD, Pugh NE, Hesser H, Andersson G. Therapeutic alliance in internet-delivered cognitive behaviour therapy for depression or generalized anxiety. Clin Psychol Psychother 2017 Mar;24(2):451-461. [CrossRef] [Medline]
- Knaevelsrud C, Maercker A. Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: a randomized controlled clinical trial. BMC Psychiatry 2007 Apr 19;7:13 [FREE Full text] [CrossRef] [Medline]
- Preschl B, Maercker A, Wagner B. The working alliance in a randomized controlled trial comparing online with face-to-face cognitive-behavioral therapy for depression. BMC Psychiatry 2011 Dec 6;11:189 [FREE Full text] [CrossRef] [Medline]
- Wagner B, Brand J, Schulz W, Knaevelsrud C. Online working alliance predicts treatment outcome for posttraumatic stress symptoms in Arab war-traumatized patients. Depress Anxiety 2012 Jul;29(7):646-651. [CrossRef] [Medline]
- Baumel A, Yom-Tov E. Predicting user adherence to behavioral eHealth interventions in the real world: examining which aspects of intervention design matter most. Transl Behav Med 2018 Sep 8;8(5):793-798. [CrossRef] [Medline]
- Baumel A, Kane JM. Examining predictors of real-world user engagement with self-guided ehealth interventions: analysis of mobile apps and websites using a novel dataset. J Med Internet Res 2018 Dec 14;20(12):e11491 [FREE Full text] [CrossRef] [Medline]
- Hargreaves A, Dillon R, Castorina M, Furey E, Walsh J, Fitzmaurice B, et al. Predictors of adherence to low support, computerised, cognitive remediation training in psychosis. Psychosis 2018 Oct 4;10(4):298-306. [CrossRef]
- Anderson RE, Spence SH, Donovan CL, March S, Prosser S, Kenardy J. Working alliance in online cognitive behavior therapy for anxiety disorders in youth: comparison with clinic delivery and its role in predicting outcome. J Med Internet Res 2012 Jun 28;14(3):e88 [FREE Full text] [CrossRef] [Medline]
- Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
- Cowpertwait L, Clarke D. Effectiveness of web-based psychological interventions for depression: a meta-analysis. Int J Ment Health Addiction 2013 Jan 18;11(2):247-268. [CrossRef]
- Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions — a systematic review. Internet Interv 2014 Oct;1(4):205-215. [CrossRef]
- Palmqvist B, Carlbring P, Andersson G. Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost? Expert Rev Pharmacoecon Outcomes Res 2007 Jun;7(3):291-297. [CrossRef] [Medline]
- Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
- Gellatly J, Bower P, Hennessy S, Richards D, Gilbody S, Lovell K. What makes self-help interventions effective in the management of depressive symptoms? Meta-analysis and meta-regression. Psychol Med 2007 Sep;37(9):1217-1228. [CrossRef] [Medline]
- Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med 2016 Dec;23(6):776-794. [CrossRef] [Medline]
- Killikelly C, He Z, Reeder C, Wykes T. Improving adherence to web-based and mobile technologies for people with psychosis: systematic review of new potential predictors of adherence. JMIR Mhealth Uhealth 2017 Jul 20;5(7):e94 [FREE Full text] [CrossRef] [Medline]
- Arnold C, Villagonzalo K, Meyer D, Farhall J, Foley F, Kyrios M, et al. Predicting engagement with an online psychosocial intervention for psychosis: exploring individual- and intervention-level predictors. Internet Interv 2019 Dec;18:100266 [FREE Full text] [CrossRef] [Medline]
- Simon GE, Ludman EJ, Goodale LC, Dykstra DM, Stone E, Cutsogeorge D, et al. An online recovery plan program: can peer coaching increase participation? Psychiatr Serv 2011 Jun;62(6):666-669 [FREE Full text] [CrossRef] [Medline]
- Titov N, Dear BF, Staples LG, Terides MD, Karin E, Sheehan J, et al. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: a randomized controlled trial. J Anxiety Disord 2015 Oct;35:88-102 [FREE Full text] [CrossRef] [Medline]
- Josephine K, Josefine L, Philipp D, David E, Harald B. Internet- and mobile-based depression interventions for people with diagnosed depression: a systematic review and meta-analysis. J Affect Disord 2017 Dec 1;223:28-40. [CrossRef] [Medline]
- Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported Internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev 2016 Mar 12;3:CD011565 [FREE Full text] [CrossRef] [Medline]
- Berger T, Caspar F, Richardson R, Kneubühler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther 2011 Mar;49(3):158-169. [CrossRef] [Medline]
- Klein B, Austin D, Pier C, Kiropoulos L, Shandley K, Mitchell J, et al. Internet-based treatment for panic disorder: does frequency of therapist contact make a difference? Cogn Behav Ther 2009;38(2):100-113. [CrossRef] [Medline]
- Kelders SM, Kok RN, Ossebaard HC, van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012 Nov 14;14(6):e152 [FREE Full text] [CrossRef] [Medline]
- Lindner P, Olsson EL, Johnsson A, Dahlin M, Andersson G, Carlbring P. The impact of telephone versus e-mail therapist guidance on treatment outcomes, therapeutic alliance and treatment engagement in Internet-delivered CBT for depression: a randomised pilot trial. Internet Inte 2014 Oct;1(4):182-187. [CrossRef]
- Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomized controlled trial comparing clinician vs technician assistance. PLoS One 2010 Jun 8;5(6):e10939 [FREE Full text] [CrossRef] [Medline]
- Todd NJ, Jones SH, Lobban FA. What do service users with bipolar disorder want from a web-based self-management intervention? A qualitative focus group study. Clin Psychol Psychother 2013;20(6):531-543. [CrossRef] [Medline]
- Gerhards SA, Abma TA, Arntz A, de Graaf LE, Evers SM, Huibers MJ, et al. Improving adherence and effectiveness of computerised cognitive behavioural therapy without support for depression: a qualitative study on patient experiences. J Affect Disord 2011 Mar;129(1-3):117-125. [CrossRef] [Medline]
- Walsh A, Richards D. Experiences and engagement with the design features and strategies of an internet-delivered treatment programme for generalised anxiety disorder: a service-based evaluation. British Journal of Guidance & Counselling 2016 Feb 28;45(1):16-31. [CrossRef]
- Wilhelmsen M, Lillevoll K, Risør MB, Høifødt R, Johansen M, Waterloo K, et al. Motivation to persist with internet-based cognitive behavioural treatment using blended care: a qualitative study. BMC Psychiatry 2013 Nov 7;13:296 [FREE Full text] [CrossRef] [Medline]
- Flückiger C, Del Re AC, Wampold BE, Symonds D, Horvath AO. How central is the alliance in psychotherapy? A multilevel longitudinal meta-analysis. J Couns Psychol 2012 Jan;59(1):10-17. [CrossRef] [Medline]
- Brown A, Mountford VA, Waller G. Is the therapeutic alliance overvalued in the treatment of eating disorders? Int J Eat Disord 2013 Dec;46(8):779-782. [CrossRef] [Medline]
- Cugelman B, Thelwall M, Dawes P. Online interventions for social marketing health behavior change campaigns: a meta-analysis of psychological architectures and adherence factors. J Med Internet Res 2011 Feb 14;13(1):e17 [FREE Full text] [CrossRef] [Medline]
- Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011 Aug 5;13(3):e52 [FREE Full text] [CrossRef] [Medline]
- Enrique A, Palacios JE, Ryan H, Richards D. Exploring the relationship between usage and outcomes of an internet-based intervention for individuals with depressive symptoms: secondary analysis of data from a randomized controlled trial. J Med Internet Res 2019 Aug 1;21(8):e12775 [FREE Full text] [CrossRef] [Medline]
- El Alaoui S, Ljótsson B, Hedman E, Svanborg C, Kaldo V, Lindefors N. Predicting outcome in internet-based cognitive behaviour therapy for major depression: a large cohort study of adult patients in routine psychiatric care. PLoS One 2016;11(9):e0161191 [FREE Full text] [CrossRef] [Medline]
- Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther 2012 Aug;50(7-8):463-468. [CrossRef] [Medline]
- Hedman E, Andersson E, Ljótsson B, Andersson G, Andersson E, Schalling M, et al. Clinical and genetic outcome determinants of internet- and group-based cognitive behavior therapy for social anxiety disorder. Acta Psychiatr Scand 2012 Aug;126(2):126-136. [CrossRef] [Medline]
- Christensen H, Griffiths KM, Korten A. Web-based cognitive behavior therapy: analysis of site usage and changes in depression and anxiety scores. J Med Internet Res 2002;4(1):e3 [FREE Full text] [CrossRef] [Medline]
- Fuhr K, Schröder J, Berger T, Moritz S, Meyer B, Lutz W, et al. The association between adherence and outcome in an internet intervention for depression. J Affect Disord 2018 Mar 15;229:443-449. [CrossRef] [Medline]
- Donkin L, Hickie IB, Christensen H, Naismith SL, Neal B, Cockayne NL, et al. Rethinking the dose-response relationship between usage and outcome in an online intervention for depression: randomized controlled trial. J Med Internet Res 2013 Oct 17;15(10):e231 [FREE Full text] [CrossRef] [Medline]
- Manwaring JL, Bryson SW, Goldschmidt AB, Winzelberg AJ, Luce KH, Cunning D, et al. Do adherence variables predict outcome in an online program for the prevention of eating disorders? J Consult Clin Psychol 2008 Apr;76(2):341-346. [CrossRef] [Medline]
- O'Brien HL, Toms EG. What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci 2008 Apr;59(6):938-955. [CrossRef]
- Beintner I, Görlich D, Berger T, Ebert DD, Zeiler M, Camarano RH, ICare Consortium. Interrelations between participant and intervention characteristics, process variables and outcomes in online interventions: a protocol for overarching analyses within and across seven clinical trials in iCare. Internet Interv 2019 Apr;16:86-97 [FREE Full text] [CrossRef] [Medline]
- Barazzone N, Cavanagh K, Richards DA. Computerized cognitive behavioural therapy and the therapeutic alliance: a qualitative enquiry. Br J Clin Psychol 2012 Nov;51(4):396-417. [CrossRef] [Medline]
- Holter MT, Johansen A, Brendryen H. How a fully automated ehealth program simulates three therapeutic processes: a case study. J Med Internet Res 2016 Jun 28;18(6):e176 [FREE Full text] [CrossRef] [Medline]
- Bucci S, Schwannauer M, Berry N. The digital revolution and its impact on mental health care. Psychol Psychother 2019 Jun;92(2):277-297. [CrossRef] [Medline]
- Fullwood C, Quinn S, Kaye LK, Redding C. My virtual friend: a qualitative analysis of the attitudes and experiences of smartphone users: implications for smartphone attachment. Comput Hum Behav 2017 Oct;75:347-355. [CrossRef]
- Cohen J, Torous J. The potential of object-relations theory for improving engagement with health apps. J Am Med Assoc 2019 Nov 14:- epub ahead of print. [CrossRef] [Medline]
- Fogg B, Euchner J. Designing for behavior change—new models and moral issues. Res-Technol Manag 2019 Sep 5;62(5):14-19. [CrossRef]
- Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst 2009;24:485-500. [CrossRef]
- Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011 Jan 6;13(1):e2 [FREE Full text] [CrossRef] [Medline]
- Lehto T, Oinas-Kukkonen H. Persuasive features in web-based alcohol and smoking interventions: a systematic review of the literature. J Med Internet Res 2011 Jul 22;13(3):e46 [FREE Full text] [CrossRef] [Medline]
- Wozney L, Huguet A, Bennett K, Radomski AD, Hartling L, Dyson M, et al. How do ehealth programs for adolescents with depression work? A realist review of persuasive system design components in internet-based psychological therapies. J Med Internet Res 2017 Aug 9;19(8):e266 [FREE Full text] [CrossRef] [Medline]
- Wildeboer G, Kelders SM, van Gemert-Pijnen JE. The relationship between persuasive technology principles, adherence and effect of web-based interventions for mental health: a meta-analysis. Int J Med Inform 2016 Dec;96:71-85. [CrossRef] [Medline]
- Purpura S, Schwanda V, Williams K, Stubler W, Sengers P. Fit4life: the Design of a Persuasive Technology Promoting Healthy Behavior and Ideal Weight. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2011 Presented at: CHI'11; March 15-17, 2011; Vancouver, BC, Canada. [CrossRef]
- Toscos T, Faber A, An S, Gandhi M. Chick Clique: Persuasive Technology to Motivate Teenage Girls to Exercise. In: CHI '06 Extended Abstracts on Human Factors in Computing Systems. 2006 Presented at: CHI EA'06; April 22-27, 2006; Montréal, Québec, Canada. [CrossRef]
- Mintz J, Aagaard M. The application of persuasive technology to educational settings. Educ Techol Res Dev 2012 Jan 26;60(3):483-499. [CrossRef]
- Bellur S, Sundar SS. Talking health with a machine: how does message interactivity affect attitudes and cognitions? Hum Commun Res 2016 Sep 15;43(1):25-53. [CrossRef]
- Petty R, Cacioppo J. The elaboration likelihood model of persuasion. Adv Exp Soc Psychol 1986;19:123-205. [CrossRef]
- Macneil CA, Hasty MK, Evans M, Redlich C, Berk M. The therapeutic alliance: is it necessary or sufficient to engender positive outcomes? Acta Neuropsychiatr 2014 Jun 24;21(2):95-98. [CrossRef]
- Dixon LB, Holoshitz Y, Nossel I. Treatment engagement of individuals experiencing mental illness: review and update. World Psychiatry 2016 Feb;15(1):13-20 [FREE Full text] [CrossRef] [Medline]
- Ardito RB, Rabellino D. Therapeutic alliance and outcome of psychotherapy: historical excursus, measurements, and prospects for research. Front Psychol 2011;2:270 [FREE Full text] [CrossRef] [Medline]
- Ruchlewska A, Kamperman AM, van der Gaag M, Wierdsma AI, Mulder NC. Working alliance in patients with severe mental illness who need a crisis intervention plan. Community Ment Health J 2016 Jan;52(1):102-108 [FREE Full text] [CrossRef] [Medline]
- Harrow M, Hansford BG, Astrachan-Fletcher EB. Locus of control: relation to schizophrenia, to recovery, and to depression and psychosis -- a 15-year longitudinal study. Psychiatry Res 2009 Aug 15;168(3):186-192 [FREE Full text] [CrossRef] [Medline]
- Parada F, Martínez V, Espinosa HD, Bauer S, Moessner M. Using persuasive systems design model to evaluate 'cuida tu ánimo': an internet-based pilot program for prevention and early intervention of adolescent depression. Telemed J E Health 2020 Feb;26(2):251-254 [FREE Full text] [CrossRef] [Medline]
- Radomski AD, Wozney L, McGrath P, Huguet A, Hartling L, Dyson MP, et al. Design and delivery features that may improve the use of internet-based cognitive behavioral therapy for children and adolescents with anxiety: a realist literature synthesis with a persuasive systems design perspective. J Med Internet Res 2019 Feb 5;21(2):e11128 [FREE Full text] [CrossRef] [Medline]
- Bachelor A. Clients' and therapists' views of the therapeutic alliance: similarities, differences and relationship to therapy outcome. Clin Psychol Psychother 2013;20(2):118-135. [CrossRef] [Medline]
- Mackie C, Dunn N, MacLean S, Testa V, Heisel M, Hatcher S. A qualitative study of a blended therapy using problem solving therapy with a customised smartphone app in men who present to hospital with intentional self-harm. Evid Based Ment Health 2017 Nov;20(4):118-122. [CrossRef] [Medline]
- Valentine L, McEnery C, D’Alfonso S, Phillips J, Bailey E, Alvarez-Jimenez M. Harnessing the potential of social media to develop the next generation of digital health treatments in youth mental health. Curr Treat Options Psych 2019 Sep 11;6(4):325-336. [CrossRef]
- Alvarez-Jimenez M, Bendall S, Lederman R, Wadley G, Chinnery G, Vargas S, et al. On the HORYZON: moderated online social therapy for long-term recovery in first episode psychosis. Schizophr Res 2013 Jan;143(1):143-149. [CrossRef] [Medline]
- Lakey B, Cohen JL, Neely LC. Perceived support and relational effects in psychotherapy process constructs. J Couns Psychol 2008;55(2):209-220. [CrossRef]
- Naslund JA, Aschbrenner KA, Marsch LA, Bartels SJ. The future of mental health care: peer-to-peer support and social media. Epidemiol Psychiatr Sci 2016 Apr;25(2):113-122 [FREE Full text] [CrossRef] [Medline]
- Chronister J, Chou C, Kwan KK, Lawton M, Silver K. The meaning of social support for persons with serious mental illness. Rehabil Psychol 2015 Aug;60(3):232-245 [FREE Full text] [CrossRef] [Medline]
- Osipov M, Behzadi Y, Kane JM, Petrides G, Clifford GD. Objective identification and analysis of physiological and behavioral signs of schizophrenia. J Ment Health 2015;24(5):276-282 [FREE Full text] [CrossRef] [Medline]
- Barnett I, Torous J, Staples P, Sandoval L, Keshavan M, Onnela J. Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology 2018 Jul;43(8):1660-1666 [FREE Full text] [CrossRef] [Medline]
- Faurholt-Jepsen M, Busk J, Þórarinsdóttir H, Frost M, Bardram JE, Vinberg M, et al. Objective smartphone data as a potential diagnostic marker of bipolar disorder. Aust N Z J Psychiatry 2019 Feb;53(2):119-128. [CrossRef] [Medline]
- Freeman D, Reeve S, Robinson A, Ehlers A, Clark D, Spanlang B, et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med 2017 Oct;47(14):2393-2400 [FREE Full text] [CrossRef] [Medline]
- Miragall M, Baños RM, Cebolla A, Botella C. Working alliance inventory applied to virtual and augmented reality (WAI-VAR): psychometrics and therapeutic outcomes. Front Psychol 2015;6:1531 [FREE Full text] [CrossRef] [Medline]
- Torous J, Hsin H. Empowering the digital therapeutic relationship: virtual clinics for digital health interventions. NPJ Digit Med 2018;1:16 [FREE Full text] [CrossRef] [Medline]
|ARM: Agnew Relationship Measure|
|BD: bipolar disorder|
|OCD: obsessive compulsive disorder|
|PSD: persuasive systems design|
|PTSD: posttraumatic stress disorder|
|RCT: randomized controlled trial|
|VR: virtual reality|
|WAI: Working Alliance Inventory|
Edited by S D'Alfonso, R Lederman; submitted 29.11.19; peer-reviewed by K Berry, E Caballeria, P Navarro; comments to author 13.01.20; revised version received 07.06.20; accepted 07.06.20; published 07.08.20Copyright
©Hailey Tremain, Carla McEnery, Kathryn Fletcher, Greg Murray. Originally published in JMIR Mental Health (http://mental.jmir.org), 07.08.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.