The myCompass RCT examined the effectiveness of a fully automated and self-guided psychological treatment that is delivered via the Internet to mobile and stationary technology devices for improving mental health symptom and functional outcomes. Participants in the RCT were 720 community volunteers with self-reported mild-to-moderate depression, anxiety, and/or stress symptoms who were recruited nationally in Australia between October 2011 and March 2012 via Internet, radio, and print media advertising. Following baseline assessment, participants were randomly allocated to the myCompass intervention, an attention control program, or a waitlist control condition, for 7 weeks. Subsequent assessments were conducted online on completion of the intervention phase (8 weeks) and at 20 weeks. The design and recruitment procedures for the myCompass RCT have previously been reported in greater detail [28].

Quantitative data for this secondary analysis was derived from participants randomly allocated to receive the myCompass intervention. This was supplemented by qualitative data provided by a purposively selected sample that completed the post-intervention questionnaire and agreed, via email, to participate in a semi-structured telephone interview with one of the authors (MAB). A sampling to saturation recruitment method was used in which data collection continued until no new themes emerged from the interviews [29].

The RCT was approved by the Human Research Ethics Committee of the University of New South Wales, Sydney, Australia (HREC 10019) and registered with the Australian and New Zealand Clinical Trials Registry (ACTRN 12610000625077). The CONSORT-EHEALTH checklist is provided as Multimedia Appendix 1.

myCompass is a fully automated public health CBT-based intervention for the treatment of mild-to-moderate depression, anxiety, and stress [30]. The program is completely self-guided with no therapist input, and is accessible from any Internet-enabled mobile phone, tablet, or computer.

The myCompass program assesses users’ self-reported symptoms on registration and then provides 24/7 access to a personalized intervention that includes real-time, self-monitoring of moods and behaviors (via mobile phone, tablet, or computer), and interactive, evidence-based psychotherapeutic modules (via tablet and computer). In addition, users can schedule SMS text message (short message service) or email reminders to facilitate self-monitoring, receive and print graphical feedback about their self-monitoring alongside contextual information on their phone or computer (to monitor change and assist identification of triggers), and elect to receive helpful facts, mental health care tips or motivational statements by text message or email. Registering to use the program is free, and users are not billed for the text messages they receive. A detailed description of the myCompass intervention is provided in Proudfoot et al [28].

Quantitative analyses were conducted using SPSS version 21, and used data derived from participants who returned a post-intervention questionnaire. Internal consistency of the modified ARM subscales was tested using Cronbach’s alpha, and subscale means were examined to determine alliance strength. In line with previous studies of Internet-delivered therapies [22,23], residual gain scores were calculated to represent post-intervention treatment gains using the formula described by Steketee and Chambless [37] (the standardized subscale score at post-intervention minus the standardized subscale score at baseline, multiplied by the correlation between these scores). Residual gain scores thus represented treatment gains at post-intervention scores adjusted for baseline scores, and bivariate correlation analyses examined relations between these and scores on the ARM subscales. Since ARM data was collected at post-intervention only, it was not possible to perform the analyses on an intention-to-treat basis [24].

Interview transcripts were audio recorded, transcribed, and analyzed independently by 2 of the authors (MRB and MAB) using a word processing package (MS Word). Thematic analysis of responses was used to identify the main themes and subcategories, while at the same time a thorough search was conducted for divergent views to enable a richer description of therapeutic alliance features [38]. A 95% level of agreement was reached for themes and subcategories, with differences resolved after discussion. A total of 8 principal themes were identified and agreed upon, and these have been used as organizing themes for the qualitative data.

Of the 231 RCT participants allocated to the myCompass intervention, 126 (54.5%, 126/231) returned post-intervention questionnaires (Figure 1). On closer inspection, we found that 11 (8.7%, 11/126) had not completed the ARM subscales, and a further 25 (10.8%, 25/231) had not registered with and did not use myCompass. We excluded data from both of these groups, leaving a final sample for this secondary analysis of 90 participants that were predominantly female (72%, 65/90), and employed (78%, 70/90), with a mean age of 38 years (SD 10 years) (Figure 1).

Interviews were conducted with 16 participants, 3 of whom were male with a mean age 40.1 years (SD 8.4 years). Thematic analysis elicited 8 themes that described participants’ relationship with the myCompass intervention (Textbox 1).

This secondary analysis of data from a large RCT explored the extent and nature of the therapeutic alliance in the fully automated, mobile phone and Web-based intervention, myCompass. Consistent with Ormrod et al [26] and other studies of Internet-delivered therapies [21,22], results of the quantitative analysis showed that participants reported development of a positive alliance with the myCompass intervention. A point of differentiation from most previous studies, however, is that these findings were obtained in the absence of any therapist involvement in delivery of the intervention. Insights gained from the qualitative interviews provided further evidence that non-specific alliance features were experienced in participants’ interactions with the intervention, including empathy, collaboration, reassurance, and reciprocity. Whereas classic definitions of the therapeutic alliance require that a client and therapist are involved in the relationship [39], and therapists have previously questioned or down-played the existence of relationship process variables in computer-based therapies [19], our findings show that a significant and positive alliance can exist in the Internet environment in the absence of human support.

We also examined whether therapeutic alliance features were associated with treatment gains and program engagement. Consistent with previous studies of Internet-delivered interventions across a range of disorders [21,22,26], we found no support for a link between therapeutic alliance factors and symptom, functional, and positive well-being outcomes. These findings contrast with those for face-to-face psychotherapies [40], and lend further support for the idea that the quality of the alliance in the Internet environment may be less important than other factors for understanding treatment gains [21,26].

On the other hand, components of the therapeutic alliance did show significant and positive associations with participants’ level of engagement with the myCompass intervention. Most notably, ratings of perceived emotional connection with the program correlated positively with program logins, frequency of self-monitoring, and the number of psychoeducational modules completed, suggesting that users may engage with an Internet intervention to the extent that they experience a collaborative partnership that is working well, just as in face-to-face therapy [11]. Engagement with program content is generally linked with increased treatment gains, yet rates of adherence with Internet-delivered therapies are characteristically low [34], and attrition rates are high [41]. A contribution of our study, therefore, is to suggest that incorporating program content and functions that target alliance processes directly may improve client adherence and retention in the Internet context. Indeed, it has recently been proposed that the real value of a strong alliance may lie in its capacity to promote therapeutic engagement as opposed to contributing to clinical improvement [11].

Because the ARM and other popular measures of the therapeutic alliance (eg, the Working Alliance Inventory [42]) were originally developed for the face-to-face context, they are unlikely to tap clients’ perceptions of common factors or relationship features that may be distinct to Internet psychotherapies. For example, flexibility in the nature of the therapeutic encounter (in terms of time, location, and duration of access) is generally considered a particular advantage of Internet treatment [1], as is the option of interacting via different communication pathways [43]. However, these relationship variables are not assessed in existing alliance measures, and they remain largely unexplored as contributors to psychotherapy outcomes in the Internet context [1]. Further work is needed to conceptualize, from a client’s perspective, the common or non-specific processes that characterize relationships in the Internet environment, and to examine the implications of these for treatment outcomes and program engagement using appropriately developed and validated alliance measures.

Some limitations must be considered in interpreting these findings. As we have discussed previously [28], dropout attrition in the intervention arm of the RCT was high, especially among employed participants. Given positive links between alliance strength and treatment engagement [11], and a typical pattern of high alliance ratings among trial participants [23], we cannot discount the possibility that dropout from the intervention group reflected a lack of perceived alliance with the myCompass intervention. Furthermore, the sample was predominantly female which is potentially problematic as gender differences appear to influence the therapeutic alliance [44]. The generality of our findings, therefore, is somewhat limited.

From a measurement point of view, we remain uncertain as to whether the psychometric adequacy of the ARM is affected when scale items are adapted along the lines reported here (ie, replacing the word “therapist” with “program”). Until reliability and validity of the modified ARM is demonstrated and/or alternative measures of the human-computer alliance are developed, our conclusions must be considered tentative. In addition, therapeutic alliance data were collected at only one time point (ie, at post-intervention). Previous researchers have discussed the importance of measuring therapeutic alliance variables at various stages of the therapeutic process [25], and there is evidence that therapeutic success in the face-to-face context is more likely for patients whose alliance increases in the early stages of treatment [45,46]. At the same time, relations between therapeutic alliance features and program engagement are potentially reciprocal, such that a strong alliance predicts increased program interactions and vice versa [11]. Repeated administration of the ARM over the course of the intervention would have allowed a stronger test of the contribution made by therapeutic alliance features to treatment outcomes, program engagement, and study dropout.

This study is among the first to provide quantitative and qualitative support for the existence of a positive therapeutic alliance with a fully automated, mobile phone and Web-based psychotherapy intervention involving no therapist assistance. Although it appears that a strong alliance contributes less proximally to therapy outcomes in a fully automated, Internet-delivered intervention compared with face-to-face psychotherapy, our data suggest that the ability to connect meaningfully and work collaboratively may be similarly important for client engagement across both contexts. Client engagement is vital for therapeutic success, yet effective translation of non-specific alliance components of face-to-face therapies into the Internet environment is not easily achieved [10]. For the sake of expediency, and in the interest of optimizing therapeutic gains, future studies should isolate the critical relational components of encounters in the Internet environment that relate to treatment outcomes and client engagement. These can then be honed in the development of new and refinement of existing Internet-delivered interventions.