A total of 520 patients were recruited via the United States national online registry ResearchMatch between September 2023 and January 2024. Eligibility criteria included being 18 years or older, currently undergoing individual psychotherapy at a frequency of at least 2 sessions per month, and possessing sufficient proficiency in the English language. Exclusion criteria were confined to patients under legal guardianship. Potential participants were provided with a detailed online consent form about the study, with those interested giving their electronic consent. Participants received no financial incentives for their involvement in this trial. Although therapists were not directly informed of their patients’ participation by the research team, patients were encouraged to discuss their involvement in the study with their therapists before and during the trial.

Patients providing electronic informed consent underwent an online baseline assessment before being randomized into either the intervention or control group. After randomization, participants in the intervention group were provided with a link to complete a brief postsession battery following each therapy session over the course of the 10-week trial. The link for the postsession battery remained unchanged throughout the study. Each survey included a question about the date of the next session. Additionally, participants were asked if they wanted to receive a reminder email on the day of their next therapy session. If they opted in, a reminder was sent on the day of their session. Assessments for participants in both the intervention and control groups were conducted at baseline, mid-trial (5 weeks post-baseline), and end of trial (10 weeks post-baseline). Participants received 2 reminders to complete each of the follow-up assessments. Data collection was facilitated through Qualtrics.

Ethical approval for the study was granted by the Institutional Review Board of the University of North Carolina at Chapel Hill (reference number 23-1067). The trial was duly registered at ClinicalTrials.gov (identifier: NCT06038747), with the trial protocol published in a separate publication [22].

Participants allocated to the intervention group continued their regular individual psychotherapy sessions. After each session, they were asked to complete a brief postsession battery comprising 2 scales designed to evaluate affective reactions toward their therapist: the in-Session Patient Affective Reactions Questionnaire (SPARQ) [23,24] and Rift In-Session Questionnaire (RISQ) [23]. Upon completing the battery, participants received generalized feedback, emphasizing the importance of discussing their session-related feelings and reflections with their therapist. The generalized feedback was considered an integral part of the intervention and consisted of the following sentence: “Experiencing positive or negative emotions—or even a mix of both—is not uncommon in psychotherapy. It can be hard to tell our therapist about our feelings towards them, especially the ‘negative’ ones (such as feeling shy, ashamed, rejected, scared, attacked, or put down). However, sharing and discussing these feelings with your therapist can help address concerns, build trust, explore underlying issues, and, more generally, foster a supportive and collaborative therapeutic relationship that promotes personal growth and well-being.” The battery’s foremost aim was to promote participants’ self-awareness and reflection on their interactions and the overall dynamics of the therapeutic relationship.

Patients in the control group received only their regular individual psychotherapy (treatment as usual) and did not complete any postsession battery. Additionally, they did not receive any communication encouraging them to discuss their session-related emotions with their therapist.

The postsession battery incorporated 2 distinct self-report scales: the SPARQ [23,24] and RISQ [23].

The SPARQ, consisting of 8 items, delves into the cognitive, emotional, and behavioral responses patients exhibit toward their therapists during therapy sessions. This questionnaire is divided into 2 components: the Positive Affect and Negative Affect scales. Analysis of the initial session data from the trial revealed Cronbach α values of 0.86 for the Positive Affect scale and 0.79 for the Negative Affect scale.

The RISQ, comprising 4 items, aims to assess the likelihood of disruptions occurring within the therapeutic relationship. According to the findings from this study, the RISQ exhibited a Cronbach α coefficient of 0.68.

Immediately after completing the postsession battery, written feedback encouraging participants to discuss their in-session feelings and reflections with their therapist was displayed.

The primary outcome was the level of psychological distress at the end of the 10-week trial (T2), as evaluated using the 34-item Clinical Outcomes in Routine Evaluation–Outcome Measure (CORE-OM) [25], a reliable and valid self-report measure of change in psychotherapy [25,26]. It comprises 4 domains: subjective well-being (Cronbach α=0.78), symptoms (α=0.90), function (α=0.86), and risk (α=0.84). The Cronbach α based on data at T0 was 0.95 for the total scale. A higher score on the CORE-OM indicates more severe difficulties.

The level of the real relationship between the patient and therapist from the patient's point of view was evaluated with the 8-item version of the Real Relationship Inventory-Client-short form (RRI-C-SF) [27]. The real relationship is defined as “the personal relationship between therapist and patient marked by the extent to which each is genuine with the other and perceives/experiences the other in ways that befit the other” [28]. The RRI-C-SF comprises 2 subscales: genuineness (α=0.84) and realism (α=0.82), with realism referring to the accurate perception of the other. The overall scale reliability was α=0.89. Higher scores reflect a stronger real relationship.

The quality of the working alliance between patient and therapist from the patient’s perspective was assessed with the 12-item version of the Working Alliance Inventory-Short Revised (WAI-SR) [29]. The WAI-SR consists of 3 subscales: agreement on therapy goals (α=0.91), tasks (α=0.90), and the development of an affective bond (α=0.90). The total scale showed a Cronbach α of 0.95. Higher scores indicate a more robust alliance.

Baseline depressive symptoms were assessed with the Patient Health Questionnaire-9 (PHQ-9) [30], while the baseline level of anxiety was measured with the Generalized Anxiety Disorder-7 (GAD-7) [31]. In this study, these scales demonstrated Cronbach α coefficients of 0.86 and 0.88, respectively.

The power calculation, assuming a power of 0.80, a 2-tailed α of 0.05, an effect size (d) of 0.50, and a 1:1 allocation ratio, determined that a sample size of 128 patients (64 per group) was necessary to detect this effect size. However, to enhance statistical power and precision while accounting for a potential 50% attrition rate among participants in the intervention group, who must complete a postsession assessment after each therapy session during the 10-week trial, we aimed to enroll 520 participants (260 per group).

Participants were randomly assigned to either the intervention or control group using a computer-generated list following the completion of baseline assessments. Patients were not blind to the randomization.

Analyses adhered to per-protocol and intention-to-treat principles. To identify baseline differences between responders and nonresponders, unpaired t tests and logistic regression models were utilized. Analyses of covariance (ANCOVA) evaluated group outcomes at mid-trial and end of trial, adjusting for baseline clinical and demographic characteristics of patients, as well as treatment characteristics. Within-subject changes in outcome measures were examined using paired sample t tests. Findings include means, standard deviations (for ANCOVA and between-group analyses), t test statistics, and effect sizes (Cohen d), with Cohen d derived from ANCOVA-estimated means, adjusted for baseline scores, and corrected pooled standard deviations for group size disparities. Predictors of dropout in both control and intervention groups were explored using logistic regression analysis (completers=1), considering demographic, clinical, and treatment characteristics, as well as treatment assignment. No imputation was performed. All statistical tests were 2-sided, with a significance level set at .05, utilizing R version 4.3.1 for data analysis.

Figure 1 shows the flow of the study participants from recruitment to the end of the intervention (T2). Of the 520 patients recruited, 45 (8.7%) were excluded from the study for not meeting the inclusion criteria (ie, a minimum of 2 therapy sessions per month—13/45, 2.9%) or for not completing the baseline assessment (32/520, 6.2%). Thus, a total of 475 (91.3%) patients were randomized, with 243 allocated to the intervention group and 232 allocated to the control group. The discrepancy (n=11) between the number of individuals randomized to the active treatment group (n=243) and the control group (n=232) resulted from the use of a pre-established computer-generated randomization list based on a sample of 520. This list did not account for the exclusion of 45 patients who either did not meet the inclusion criteria or withdrew during the baseline assessment.

Regarding the control group, 64.2% (149/232) of the participants completed the mid-trial assessment (T1), and 53.9% (125/232) completed the end-of-trial assessment (T2). A total of 48 participants completed T1 but not T2, while 24 participants missed T1 but completed T2. Between T0 and T1, 7 patients terminated their psychotherapy but completed the mid-trial assessment, and between T1 and T2, another 7 patients terminated their psychotherapy but completed the end-of-trial assessment.

In the intervention group, 180 (180/242, 74.4%) patients completed at least 1 postsession battery and read the feedback. At T1, 51.4% (125/243) of the participants completed the assessment, and 26.7% (65/243) completed the assessments at both T1 and T2. Between T0 and T1, 5 patients terminated their psychotherapy but completed the mid-trial assessment, and between T1 and T2, 10 patients terminated their psychotherapy but completed the end-of-trial assessment.

The attrition rate at T1 was 35.8% (83/232) for the control group and 48.6% (118/243) for the intervention group, showing a statistically significant difference (P=.005). At T2, the attrition rate increased to 46.1% (107/232) for the control group and 73.3% (178/243) for the intervention group (P<.001).

Sociodemographic and clinical characteristics of both the intention-to-treat and the per-protocol samples are reported in Table 1. Participants in the intention-to-treat sample were predominantly women (361/475, 76%), were in the mean age range of 30 years to 39 years (142/475, 29.9%), and held bachelor’s degrees or higher. Most participants had a diagnosis of a psychiatric disorder (427/475, 89.9%) and received a psychotropic medication (337/475, 70.9%). Many were in psychotherapy for more than 24 months (225/475, 47.4%), typically at a frequency of 2 to 3 sessions per month (255/475, 53.7%). Participants in the intervention and control groups did not differ statistically significantly for sociodemographic and clinical characteristics at baseline (all P values ≥.057).

Participants who completed the entire trial did not significantly differ from those who dropped out regarding sociodemographic characteristics or scores on primary and secondary outcome measures as measured at baseline (before randomization). However, significant differences were observed in some clinical features and treatment characteristics (see Multimedia Appendix 1). Notably, the nature of these significant differences varied between dropouts in the control group and those in the intervention group. In the intervention group, participants without a diagnosed psychiatric disorder were more likely to drop out (P=.002), as were those with lower baseline levels of real relationship as measured by the RRI-C-SF (P=.002), those with a lower tendency to disclose their emotional reactions to the therapist (P=.01), those attending fewer sessions per month (P=.02), and those who canceled more sessions during the 5 weeks preceding the trial (P=.005). Similarly, among control participants, a lower quality of working alliance as measured by the WAI-SR was almost statistically significantly associated with a higher dropout likelihood (P=.051). In the intervention group, the dropout rate was higher among participants receiving treatment in public health institutions and lower among those in private health institutions (P=.04).

Logistic regression analysis revealed distinct predictors of dropout within the control group and intervention group. Analysis of the whole sample, including treatment arm and interaction terms, revealed several findings. Patients with higher baseline psychological distress (CORE-OM scores) were less likely to achieve the outcome (β=–0.04, P=.02), while those with higher baseline depressive symptoms (PHQ-9 baseline score) were more likely to achieve it (β=0.16, P=.004). Undergoing psychotherapy for more than 24 months (β=1.28, P=.009) and having a male therapist (β=1.07, P=.02) also increased the likelihood of achieving the outcome. Conversely, a history of session cancellations in the 5 weeks before the trial reduced the likelihood of achieving the outcome (β=–0.99, P=.02).

Significant interactions were found between being in the intervention group and being in a university counseling center (β=4.01, P=.02) and between being in the intervention group and having a male therapist (β=–2.07, P=.001). This means that within the intervention group, being treated in a university counseling center significantly decreased the likelihood of dropout, whereas having a male therapist increased the likelihood of dropout.

In the control group, patients who, at baseline, exhibited higher levels of psychological distress (CORE-OM baseline scores) had a reduced likelihood of completing the trial (β=–0.04, P=.02). Conversely, elevated levels of depressive symptoms (PHQ-9 baseline score; β=0.16, P=.004), undergoing psychotherapy for more than 2 years (β=1.28, P=.009), and having a male therapist notably increased the likelihood of completing the trial (β=1.07, P=.02). Additionally, a history of session cancellations in the 5 weeks preceding the trial’s onset was significantly associated with a reduced likelihood of completing the trial (β=–0.99, P=.02).

In the intervention group, patients with male therapists showed lower odds of completing the trial compared with those with female therapists (β=–1.00, P=.04).

The results of the intention-to-treat analyses are provided in the appendixes. More specifically, Multimedia Appendix 2 presents the means and standard deviations for the primary and secondary outcomes at T0, T1, and T2, along with within-group Cohen d values. Multimedia Appendix 3 offers between-group comparisons for the primary and secondary outcomes analyzed at T1 and T2, using analyses of covariance (F) and the corresponding Cohen d values. Finally, Multimedia Appendix 4 details within-group comparisons for the primary and secondary outcomes using t tests and r correlation values for the T0-T1, T1-T2, and T0-T2 periods.

The limited efficacy of psychotherapeutic interventions affects about 30% of patients [32]. To address this issue, our study examined the potential of internet-based, therapist-independent interventions that encourage patient self-assessment and thus self-reflection on a crucial element of the therapeutic relationship: the emotional reactions toward the therapist during individual psychotherapy sessions. We aimed to determine if the use of this web-based routine process monitoring and feedback system over a period of 10 weeks could significantly reduce psychological distress or improve the real relationship and working alliance quality, compared with a treatment-as-usual group receiving only individual therapy.

In our per-protocol analyses, no significant differences were found across groups for the primary outcome of psychological distress, as assessed by the CORE-OM. Despite significant improvements observed over time in both groups, the difference in improvement was more notable in the control group for the primary outcome (d=–31 versus d=–18). However, it is crucial to remember that the control group had higher baseline scores (50.47 vs 47.60), which allowed more room for improvement. By the end of the trial, CORE-OM scores were nearly indistinguishable between the groups. Regarding the secondary outcomes, our findings were contrary to our initial hypothesis. The intervention group, which focused on self-assessment of in-session emotional reactions over the 10-week intervention period, did not demonstrate improvements in working alliance quality or real relationship levels. In fact, there was a decrease in the levels of real relationship observed. Compared with the control group, the intervention group had lower effect sizes concerning the quality of these therapeutic relationship elements, which were also noted with lower baseline scores in the control group.

Of particular interest was the change over time in the levels of real relationship between the groups. The most significant difference was observed in the genuineness dimension, with the intervention group showing an effect size of d=–0.27, contrasted with d=0.01 in the control group at T2. Although this finding is not consistent with our original hypotheses (ie, an increase in both genuineness and realism scores), it provides an important clinical insight, as well as a potentially important theoretical insight. Our post hoc interpretation relies on the fact that the items composing the genuineness subscale of the RRI-C-SF relate to being oneself with the therapist, openness and honesty, the ability to communicate one’s moment-to-moment inner experience, and the expression of feelings in therapy. Thus, it is possible to hypothesize that the intervention—which required patients to reflect on the aforementioned specific aspects of their personal relationship with their therapist—may have helped the patients to gain a greater awareness of certain inner struggles or resistances of him/her/them within the relationship. In this sense, the lower scores observed in the RRI-C-SG genuineness dimension do not signify a worsening of the real relationship with the therapist; instead, they might denote a more realistic and accurate perception of it. The maintenance of the bond dimension of the working alliance (d=0.05), corroborates the nondeterioration of the relationship. Furthermore, at the end of trial, the effect sizes for the task dimension of the working alliance were d=0.22 for the control group and d=–0.03 for the intervention group, allowing us to hypothesize that a focus on the emotional aspects of the therapeutic relationship might reduce the emphasis on or perceived importance of therapy tasks.

Another important finding is the very high attrition rate observed in the intervention group, where 45% of the patients who completed the postsession battery at least once eventually dropped out. This substantial dropout rate suggests that the implementation of this particular form of routine self-monitoring may not be feasible or acceptable for many patients. Notably, in the intervention group (but not in the control group), patients who dropped out had statistically significantly lower levels of real relationship (P=.002), lower quality of the working alliance (P=.051), and less disclosure of their emotional states to the therapist (P=.01) compared with patients who did not drop out. This might imply that patients who perceived the quality of their therapeutic relationship as subpar were less motivated to engage with it. Furthermore, since our intervention included encouraging patients to discuss their in-session emotional reactions with their therapist, it is important to recognize that evidence indicates that disclosure in psychotherapy is positively correlated with the strength of the therapeutic alliance [33] and that common psychotherapy-related deceptions often involve patients feigning agreement with the therapist’s comments [34]—and thus concealment of real emotional reactions toward such comments. Therefore, patients experiencing a lower quality of alliance may have been more likely to withdraw from a study that required them to engage in behaviors, like direct emotional disclosure, that they are unaccustomed to.

Taken together, our findings on retention and outcomes only partially support the notion that a systematic monitoring process and feedback system, involving only the patient, is a viable and effective addition to individual psychotherapy. However, future research should test our hypothesis that the statistically significant lower scores observed in the genuineness dimension of the real relationship might indicate a more realistic and accurate perception by the patients. If this proves true, utilizing our postsession battery over a longer period could lead to a significantly more accurate self-perception of their levels of genuineness with the therapist. This might be clinically significant, given the direct relationship between the quality of the real relationship and session and treatment outcomes [35], as well as its close association with the working alliance [36], which in turn is closely linked to treatment outcomes [37,38]. Additionally, future research should explore the use of this battery as a tool administered within psychotherapy sessions—at the very start, referring to the previous session—so that therapists and patients can discuss it directly. This method of monitoring process or outcome and feedback to the therapist is currently the most widely used [13,16].

Some of the findings were post hoc and unexpected. It is possible that asking about the sessions and affect raised the salience of these issues. The study design did not provide scores nor feedback to the therapist. This may have brought feelings to the surface while not providing a context to process them in the next session. The larger plan for a “feedback system” is intended to include such a mechanism, and the role of “salience without processing” is a testable hypothesis that should be addressed in future work. Possibilities include focus groups or “exit interviews” with participants, as well as development of an integrated monitoring and reporting system.

From a practical perspective, if future research confirms the hypothesis that our online self-monitoring system fosters a more realistic and accurate perception of the therapeutic relationship from the patient’s point of view, it could have significant clinical and economic benefits. In this case, the system should be adapted to include the therapist, who would receive the patient’s self-assessed outcome and use this information to guide the psychotherapeutic intervention. This could strengthen the therapeutic relationship and improve treatment outcomes. Although this monitoring and feedback system would involve the participation of the therapist, it would not increase their assessment workload. Therapists would only need to review the feedback and determine how to incorporate it into the therapeutic process. This system could be applied in various therapeutic settings without imposing additional costs on patients or excessive demands on therapists, making it highly cost efficient. By allowing more frequent and systematic monitoring of patient progress without requiring in-person assessments, such tools could reduce both the duration and overall cost of treatment. In fact, research shows that feedback systems not only improve outcomes but also lower costs associated with prolonged therapy and high attrition rates [18]. These economic advantages are especially relevant in resource-limited health care settings, where optimizing the efficiency of therapeutic interventions is essential to expand access to care [18,39].

Due to the nature of online recruitment and assessment, all data collected in this study were self-reported. Although we recommended that participants discuss the study intervention with their therapists, it is unclear whether they followed this advice or what the therapists’ reactions were (whether encouraging, discouraging, or neutral). Given that some therapists may be skeptical about research, some of them might not have actively encouraged participation (or might even have discouraged it), potentially adversely affecting the study’s attrition rate. Another limitation concerns the types of psychotherapeutic approaches used (eg, psychodynamic, cognitive behavioral). We did not collect information on the specific type of psychotherapy used, which could influence the results. Last, participation in the study did not require specific demographic nor clinical characteristics, such as particular diagnoses or levels of psychological distress. This approach enabled the inclusion of a broad spectrum of individuals undergoing individual psychotherapy. Although this heterogeneity expanded the reach of our study and reflected a realistic clinical setting, it also introduced a level of variability that may affect the generalizability and interpretation of our findings.

Although routine monitoring and feedback systems involving therapists have consistently demonstrated effectiveness in various trials, the benefits of internet-based systems that solely involve the patient remain uncertain. In this study, our new web-based intervention aimed at enhancing self-monitoring and reflection on emotional responses did not yield the expected benefits and appears to be somewhat disadvantageous in terms of the quality of the therapeutic relationship. However, we identified a notable finding: Our postsession battery significantly influences the patients’ perception of the genuineness dimension of the real relationship. This outcome suggests a promising avenue for future research and potential clinical applications, underscoring the need for further investigation.