Mental health disorders are the single largest cause of health-related economic burden worldwide [1], and, globally, there is mounting pressure on health care providers to ensure rapid access to effective, evidence-based, organizationally efficient, and cost-effective psychological interventions. Digital technology bridges the demand-supply gap by offering easily accessible, flexible, and personalized support [2], thereby creating the possibility to affect large-scale and low-cost change in public mental health [3]. The World Health Authority has therefore championed development and evaluation of electronic mental health care [4]. Electronic mental health services are defined by the provision of digital interventions via mobile apps/tablets and online, web-based programs [5], with interventions either being delivered as standalone technologies or being integrated into face-to-face therapy [6]. The technology allows symptom monitoring, provides psychoeducation, and promotes ongoing self-management strategies [7]. The technology also collects in situ assessments of mental health symptoms, which are an ecologically valid source of naturalistic research data [8]. Online, web-based programs have generated a large evidence base in support of their efficacy and effectiveness [9], while the evidence in support of mental health apps is still under development [10].

Evident enthusiasm for electronic mental health innovation is somewhat tempered by evidence that certain mental health conditions (eg, depression, paranoia, or psychosis) create problems with engagement and may weaken users’ trust in the technology itself [11]. “Technology push” also occurs when the commercial concerns of digital health care companies trump the wants/needs of patients and also challenges the values of clinicians [2]. The lack of sufficient depth of clinical and academic collaboration regarding electronic mental health innovation has been highlighted as a key feature of “technology push” [12]. The speed at which electronic mental health can be developed also threatens to ignore (or be ignorant of) robust methods for treatment development and associated evaluation [13]. The content of some electronic mental health apps has also been criticized for not being grounded in sufficient theory [14] and for the fact that the outcomes achieved during clinical trials are rarely replicated in routine service delivery settings [15]. Although a plethora of electronic mental health apps are readily available for the treatment of a variety of disorders and in a variety of contexts, questions can linger concerning a lack of evidence regarding feasibility, safety, clinical effectiveness, and efficacy [12]. The potential for technological innovations to outstrip the co-development of a sufficiently robust evidence base therefore risks a loss of confidence/trust from both clinicians and patients [16].

The electronic mental health field is currently also dominated by apps reflecting the changing methods of cognitive behavioral therapy (CBT), in part because of the ease with which those changing methods can be translated into app content [12]. For example, a recent review of CBT-based apps for the management of depression found that 31 apps were available [3]. However, the acceptability of the CBT approach is not universal and so uptake and dropout rates can be very variable [17]. There are no known examples of apps that support the work of integrative and psychodynamic psychotherapies. A widely practiced integrative psychotherapy is cognitive analytic therapy (CAT) [18]. CAT is a time-limited and relationally-driven psychotherapy [18,19], which has a generally high quality evidence base [20]. A recent meta-analysis showed that CAT had moderate-to-large effects on global functioning, interpersonal difficulties, and depression in practice-based studies, while during clinical trials the pooled effect size showed a small but significant treatment effect in favor of CAT over controls [21].

CAT is an integration of personal construct and object relations theory [22]. Internalized early object relations are termed “reciprocal roles” which influence/limit relational repertoires [23]. “Target problems” are the diagnosis/presenting problem reframed in relational themes, and “target problem procedures” describe the present day “traps” (ie, vicious circles), “snags” (ie, self-sabotage), and “dilemmas” (ie, “either/or” dilemmas) maintaining relational problems [18]. The procedural sequence object relations model anchors target problem procedures in reciprocal role activation [19]. CAT has been recently summarized into a competency framework [24]. CAT uses a 3-phase approach: (1) an initial “reformulation” stage during which target problems and target problem procedures are summarized in narrative and diagrammatic reformulations, (2) a middle “recognition” stage facilitating self and relational awareness, and (3) a final “revision” stage that is focused on change. Therapeutic change during CAT is founded on effective relational awareness [25].

The CAT-App had the aim of digitizing the in-session and between-session recognition and revision tasks that constitute the middle and final phases of the therapy [26]. Previously, within each session, the CAT therapist and patient reviewed and rated recognition and revision of target problems and target problem procedures on paper-based rating sheets. Patients were also previously provided with a variety of paper-based target problem procedure recognition tools for use between-sessions. These between-session tools have been criticized for potentially creating stigma or embarrassment due to the difficulty of completing the tools surreptitiously [27]. The CAT-App was a proposed improvement on between-session methods, as the technology provides an unobtrusive method of collecting ideographic data in real time, contains dynamic feedback on recognition and revision trajectories, contains narrative and diagrammatic reformulations, and also captures users’ current moods. The potential in-session time saving would be that CAT therapists would be able to open the app during sessions and review ongoing recognition and revision rather than making single ratings within the session. The overall aims of this study were to evaluate the relationship between app usage and clinical outcome and also to attain feedback from patients and therapists on their experiences of the technology.

The results are presented in two parts to meet the objectives of the study. The first section presents the quantitative results in terms of describing the sample, dropout rate, target problem and target problem procedure recognition and revision ratings, the clinical outcomes, and relationship between app usage and both clinical outcome and mood change over treatment time. The second section concerns the qualitative results of therapists’ and patients’ experience of CAT-App usage.

Ten patients initially consented to the study and 3 (30%) dropped out early during treatment (<3 sessions). Therefore, the case series (ie, the computer sample) consisted of 7 patients, 4 male and 3 female. The mean age of the patient sample was 34.71 years (SD 7.18). The presenting problems were mixed anxiety and depression (n=5), borderline personality disorder (n=1), and narcissistic personality disorder (n=1). Treatment duration was 16 sessions for the mixed anxiety and depression cases and 24 sessions for the personality disorder cases and is consistent with session allocation within the CAT model [18]. Six of the 7 completers were “cases” at assessment on the CORE-OM. Table 1 describes the 21 target problems rated in the app. In terms of the associated target problem procedures, 12 were traps, 6 were dilemmas, and 2 were snags. These were described in their original individual long form on the CAT-App.

For example, for the patient with narcissistic personality disorder, the target problem was “how I feel about myself keeps flipping between bigging myself up or feeling like a loser” with the associated dilemma “Either I feel superior to other people and am a bit contemptuous of them or I feel they are looking down on me and I think I'm rubbish”. Please note that the exact wording of these examples have been altered to protect patient anonymity, but without altering the clinical meaning

In terms of usage data, the completer sample used the technology on average 119.86 times (SD 97.98) ranging from 11-239 occurrences. The dropout sample used the technology on average 5.00 times (SD 2.71) ranging from 3-9 occurrences. There was a significant difference in usage between completers and dropouts (t=2.29, P=.04). Figure 1 describes a CORE-OM Jacobson plot of the clinical outcomes for the 7 completers. There was a significant reduction in psychological distress between start (M=13.02, SD 5.41) and completion (M=4.68, SD 3.19) of treatment (z=4.50, P=.01), with a large effect size (d+=1.51). One of the patients remained a “case” on the CORE-OM at termination; 4 had a reliable reduction in distress (3 of whom also were in the community norm on the CORE-OM by end of treatment). The “recovery rate” was therefore 3/7 (42.85%).

Table 2 compares the percentage of ineffective and effective recognition and revision ratings for the clinical outcome groups. The completer sample is split between those who had no reliable change on the CORE-OM (n=3; 246 app observations, range 11-235) versus those who had a reliable pre-post reduction on the CORE-OM (n=4; 572 app observations, range 51-239). Usage (number of observations) did not significantly differ between patients who experienced reliable change (M=143, SD 82.82) and those that did not (M=89; SD 126.54; t(5)=0.689, P=.521). Table 3 presents the mean mood scores for the completer sample and then those patients with reliable change and those that did not change.

Mood scores did not significantly differ according to clinical outcome. Figure 2 plots the observation quartile mood scores throughout treatment for the clinical outcome groups. Levene’s tests and associated t-tests for each phase failed to identify any significant differences in mood means and variances (SDs) between the clinical outcome groups (ie, P>.05 in all 4 comparisons of means and variances on each mood scale).

Of the 7 patients, 6 were interviewed (ie, a single patient was uncontactable to interview) and all 3 therapists were interviewed. Two main themes emerged from therapists: (1) incorporating the technology into clinical practice and (2) the perceived barriers and benefits to the technology use. Table 4 provides the themes, subthemes, and example quotes from the therapists and patients.

A benefit of the technology identified by therapists was the app was easy and unobtrusive for patients to use while prompting them to engage in relational awareness tasks. The therapists made a range of design recommendations about future iterations of the app including ability to add more diagrammatic reformulations, ability to add more than 3 target problems, modification of the recognition scale to also include “not occurred” to avoid ambiguity, nudges/incentives for change, rewards to reinforce relational change, additional journaling functions, addition of CORE-OM outcome scores, needing to be available offline, and also design improvements to color and style.

All 6 of the patient participants reported that they were frequent users of “apps” relating to social media, health and wellness, games, banking, and information (news, weather). All of the patients stated that they found the CAT-App easy to access and use, with no setup challenges reported. With respect to data protection and storage, 3 patients commented on the fact that they had no issues with agreeing to data sharing and storage, as this is such a common requirement for app usage. No other patient mentioned data storage or protection.

One patient reported that they would like to add some security to accessing the CAT-App on their phone, in addition to the personal identification number, such as adding date or birth or a fingerprint access (ID5). The limited functionality of the sequential diagrammatic reformulations hindered the use of the app for some patients. However, the easy, 24-hour access availability of the app did support some patients in practicing relational awareness unobtrusively on their phones and in the moment. A difference emerged between patients who preferred free-text writing and journaling and those who preferred tracking and quantifying change, with the former disengaging earlier from using the app. With respect to personalization, the ability to customize scales and tracking within the app was a feature that all patients mentioned they would find appealing. In terms of future development of the technology, the need for motivation to use the app was an issue, with reminders and messages being thought to be of potential use.

This project concerned a pragmatic, real-world clinical case series evaluation of a new electronic mental health app to support the delivery of an integrative form of psychotherapy. The use of apps to support integrative psychotherapies is an innovation in the field. The app was designed to enable relational awareness and change, making it distinct in terms of aims and content to the plethora of CBT-based apps available [3]. The CAT-App particularly sought to improve and innovate on extant in and out of session paper-based tools. The recognition phase of CAT was particularly well suited to electronic mental health adaptation, due to the emphasis on building relational awareness through regularly reflection on those roles and patterns maintaining contemporary distress. The content and processes of the app were therefore theoretically mapped onto the middle and final phases of CAT. This was because some electronic mental health has been criticized for not considering the manner in which the technology can be well integrated into routine care pathways [26]. However, the dropout rate in this study (30%) was higher than that reported in traditional delivery of the therapy (15%) [21]. The feasibility of using the app in clinical practice was supported by the brief (ie, 2 hour) training that therapists undertook.

Dynamic and real-time comparisons of recognition, revision, and mood ratings were not possible when using the extant paper-based recognition methods [18], and so patients received enhanced feedback to support their relational recognition efforts when using the app. The technology improved the old, paper-based approaches through being unobtrusive, containing more feedback features, capturing mood on validated scales, and storing the personal reformulations securely in one place. This would therefore be an example of technology being able to improve and innovate on previously widely used/accepted clinical methods. If the aim of recognition is to enable “a reflective and observing self” [19] that can then direct effective change, then the technology would appear a promising clinical tool in the support of this endeavor.

The aim of the CAT-App was not to replace the therapist and the therapy, but rather enhance the relational awareness work of the patient, and this aim was echoed in the interviews conducted with therapists. Therefore, therapy was still necessary alongside patient’s engagement with the app, with the two working symbiotically. The CAT-App is therefore an example of “blended digital treatment” [37] rather than “standalone” electronic mental health, as the technology was embedded into the CAT clinical care pathway [38]. The promise of blended approaches is that technology increases treatment efficiency by completing some tasks normally completed by therapists [39]. The app offered a time saving ability to therapists by having the ratings precollected and analyzed, thus circumventing the need to produce ratings in sessions. Additionally, as the app contained highly idiographic information (ie, target problems, target problem procedures, narrative reformulation, sequential diagrammatic reformulations, and goodbye letters), this would be an example of personalized medicine [40].

The question of how, when, and where electronic mental health is best used during psychological interventions is particularly relevant with regards to the generation of “digital phenotypes” that can predict impending relapse, and so effectively step in and support early intervention [41]. The promise of electronic mental health is particularly highlighted in terms of its potential to increase access to mental health services, but if the technology is purely standalone, then there is a risk that patients feel that they are being offered a “down-graded version” of face-to-face psychological work. Electronic mental health app users often abandon the technology after a brief intense period of usage, due to data inputting burden, loss of interest, and the lack of inclusive features [42]. This may have been the case here because those patients that dropped out of therapy did not use the app to any significant degree and also dropped out in the early stages of therapy.

In the context of a relational therapy such as CAT, it has been previously noted that electronic mental health users do use relational concepts when describing the technology, such as being open with and forming a bond with the technology [27]. There is a need for the “therapeutic relationship” between user and technology to be effective [43], and there have been calls for more intensive and dynamic measurement of the “therapeutic relationship” during ehealth interventions [44]. It has been argued that developers should pay far greater attention to developing and maintaining the relationship the patient has with the technology and that apps should involve an initial “relationship building” aspect [27]. The current study has highlighted that encouragement and prompts to continue with app usage would be helpful. On the other hand, promoting overly close or enmeshed relationships with technology can have potential unintended negative effects. For example, it is apparent that technology use has expanded and proliferated in most people’s everyday life [45], running the risk of interfering and intruding into relationships (“technoference”). The unintentional shadow cast by electronic mental health is that accessing the technology may actually disrupt or interfere with close relationships.

In terms of study limitations, while the size of the sample in the case series was appropriate for an early practice-based study, it still represents a study limitation in terms of being a small sample size. For example, the large effect size (>1.5) needs to be interpreted with due caution, due to the unreliability of Cohen d in small samples [46]. All the inferential statistics reported therefore need to be treated with prudence due to the small sample size, particularly with respect to the dropout rate reported. There was an acknowledged convenience selection bias in the recruitment of patients that could have been corrected through random sampling of referrals. Similarly, potential participants feeling less confident with the offer of an app may have excluded themselves from the study [47]. Other sources of selection bias were the need to own a smartphone and willingness to use the app, all the patients being white adults and the study being conducted in private psychotherapy practice. These issues of context and selection bias mean that the results may not generalize to other populations including specific ethnic groups, children or the elderly, or public health settings. There was no control group using the traditional paper-based recognition tools of CAT to compare app outcomes against. Clinical outcome measurement was limited to single outcome measure and could have been broader. The app is currently only available in English and as CAT is now practiced internationally [21], sister versions in other languages need to be developed quickly [48]. The promise of standalone and blended electronic mental health is pronounced in developing countries where funding for mental health provision may be piecemeal [49].

The study did not contain a follow-up period and therefore the degree to which the app was used beyond the end of CAT was not recorded. Capturing app usage during follow-up would be a goal in future studies, particularly as structured follow-up support is integral to the CAT clinical model [19]. When services are commissioned only to deliver acute-phase treatment, then no follow-up support is made available in the continuation phase [50], and so the potential role of technology supporting patients during follow-up appears particularly important to consider. The competence with which therapy was delivered could have been assessed using the competence in CAT measure [51]. Using the app during therapy does not change the key competencies of the CAT model [24], and this needs to be evidenced in future studies. The lack of interviewing of patients that dropped out did not enable their particular difficulties with the app to be understood.

In terms of future directions, the CAT-App needs to go through another design and content iteration assessment in order to act on the feedback received. The study was conducted in private psychotherapy practice and so evaluation in public mental health services is now indicated. Further research is needed to understand how specific and different diagnoses affect app uptake, adherence, and outcomes. For example, patients in the midst of a major depressive episode may find it difficult to use technological support due to the well-evidenced problems with attention and concentration that can interfere with people’s ability to use or interact with technology [52]. CAT principles have been applied to a 6-session guided self-help intervention for delivery at step 2 of Improving Access to Psychological Therapies services in the UK [53], and finding ways in which the app could integrate with this psychoeducational approach would be particularly useful. Similarly, the utility of the app with patients with personality disorders could be evaluated as the recognition phase of CAT is prolonged with this patient group [54]. CAT is also delivered as a one session “personal reformulation” to health professionals to help develop better professional role repertoires, and a new version of the app needs to be developed to support personal reformulations. The gamification of electronic mental health offers promise in the design of innovative delivery platforms to children and young people [55].

In conclusion, this project aimed to use a mixed methods case series design to evaluate an app designed to map onto the theoretical stages and content of a widely practiced and integrative form of psychotherapy [21]. We have learnt from this study that the recognition and revision phases of CAT can be supported using mobile technology to support patients practicing relational awareness. Such awareness has previously been illustrated to be the plinth upon which change occurs [25]. Any efforts to make relational awareness tools more accessible and less obtrusive to patients between sessions and also more time efficient for therapists within sessions are at a premium. As the app provided detailed ongoing feedback absent from the previous paper-based methods, this represents a technology-enabled advance in clinical practice. Further larger and more controlled studies are now indicated, and it may be possible to progress onto a patient preference clinical trial in which app-assisted CAT is compared against routine CAT [56]. Although the app appears to hold clinical promise, future development and associated outcome studies are also clearly indicated.