This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Students in need of mental health care face many barriers including cost, location, availability, and stigma. Studies show that computer-assisted therapy and 1 conversational chatbot delivering cognitive behavioral therapy (CBT) offer a less-intensive and more cost-effective alternative for treating depression and anxiety. Although CBT is one of the most effective treatment methods, applying an integrative approach has been linked to equally effective posttreatment improvement. Integrative psychological artificial intelligence (AI) offers a scalable solution as the demand for affordable, convenient, lasting, and secure support grows.
This study aimed to assess the feasibility and efficacy of using an integrative psychological AI, Tess, to reduce self-identified symptoms of depression and anxiety in college students.
In this randomized controlled trial, 75 participants were recruited from 15 universities across the United States. All participants completed Web-based surveys, including the Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder Scale (GAD-7), and Positive and Negative Affect Scale (PANAS) at baseline and 2 to 4 weeks later (T2). The 2 test groups consisted of 50 participants in total and were randomized to receive unlimited access to Tess for either 2 weeks (n=24) or 4 weeks (n=26). The information-only control group participants (n=24) received an electronic link to the National Institute of Mental Health’s (NIMH) eBook on depression among college students and were only granted access to Tess after completion of the study.
A sample of 74 participants completed this study with 0% attrition from the test group and less than 1% attrition from the control group (1/24). The average age of participants was 22.9 years, with 70% of participants being female (52/74), mostly Asian (37/74, 51%), and white (32/74, 41%). Group 1 received unlimited access to Tess, with daily check-ins for 2 weeks. Group 2 received unlimited access to Tess with biweekly check-ins for 4 weeks. The information-only control group was provided with an electronic link to the NIMH’s eBook. Multivariate analysis of covariance was conducted. We used an alpha level of .05 for all statistical tests. Results revealed a statistically significant difference between the control group and group 1, such that group 1 reported a significant reduction in symptoms of depression as measured by the PHQ-9 (
This study offers evidence that AI can serve as a cost-effective and accessible therapeutic agent. Although not designed to appropriate the role of a trained therapist, integrative psychological AI emerges as a feasible option for delivering support.
International Standard Randomized Controlled Trial Number: ISRCTN61214172; https://doi.org/10.1186/ISRCTN61214172.
Approximately 20 million college students suffer from mental illness in the United States alone [
Mental health care solutions such as computer-assisted therapy (CAT) have been shown to be a less-intensive and more cost-effective method to deliver empirically validated treatments for depression and anxiety [
Moreover, 1 study revealed that nearly 70% of patients expressed interest in using mobile health (mHealth) apps to self-monitor and self-manage their mental health [
The objective of this study was to assess the efficacy of using the integrative psychological AI, Tess, to reduce symptoms of depression and anxiety in an engaging way. Tess was designed to deliver personalized conversations based on the expressed emotions and mental health concerns of participants, not to replace trained therapists. Tess focuses on language as the most explicit form of communication, with the proposition that communication between people reveals individual conceptualizations of specific emotions (unpublished data [
This study compared outcomes from 2 to 4 weeks of using integrative psychological AI (Tess) with an information control group (National Institute of Mental Health’s [NIMH] eBook) in a nonclinical college population. It was hypothesized that engaging in conversations with Tess would lead to greater improvement in symptoms relative to the information control group. In addition, we predicted that the duration of time in which participants interacted with Tess would impact the level of symptom reduction. To assess this, participants in the test group were randomly selected to participate in 1 of the 2 groups, which received either 2 or 4 weeks of unlimited access to Tess.
Participants were recruited using a flyer (
Confirmed participants were randomized via a computer algorithm that automatically generated a number between 0 and 2 (
Participant recruitment flow. AI: artificial intelligence.
This study was retrospectively registered under the trial number of ISRCTN61214172, and involved a nonclinical population of college students. Refer to the
Participants indicated their consent to the study’s terms via a checkbox (e-signature) on a closed form (
The third author of this paper served as an external representative to support data collection and analysis, which was provided by X2AI Inc. All data, including usage, was deidentified so as to protect the privacy of participants and was reported in aggregate only.
Tess is a psychological AI chatbot designed by X2AI Inc to deliver brief conversations in the form of integrative mental health support, psychoeducation, and reminders. Tess serves as a therapeutic tool or resource that can be used as an adjunct to therapy that supports an integrative approach and is not intended to replace the role of a therapist.
Tess was inspired by the first conversational AI, ELIZA, which examined natural language communication between people and machines in the 1960s [
The novelty of Tess is that it is a customizable platform, which allows for content to be tailored aligning with a specific form of treatment or user demographics, for example, 1 large health system in the United States customized Tess to deliver interventions based on motivational interviewing and behavioral activation to help reinforce weight management goals in a group of adolescents suffering from prediabetes symptoms (unpublished data [
Tess is maintained using a combination of technologies, emotion algorithms, and machine learning techniques to support a variety of features. Collaboration with mental health and emotion experts is a critical element of Tess’ capacity for success. All content is developed, screened, and matched to specific user inputs (ie, emotions and topics) by experienced professionals. The partnership required between humans and technology to create and maintain the chatbot is made possible through an administration panel that may be integrated with existing electronic health record systems. Each organization or clinician is provided a log-in with restricted access to manage their clients (Tess users) and content. For this study, the principal investigator and authors were provided with unique log-ins to a restricted view of participant interactions with Tess. Personally identifiable information was removed from all transcripts. Processing and storage are done on secure servers that meet Health Insurance Portability and Accountability Act–compliant regulations. Data are stored within the country of residence for all participants given access.
Tess can be configured to deliver services through existing communication channels such as Facebook messenger, Slack, and short messaging service text messaging, without requiring users to download an app. Users can access Tess using a mobile phone number or through their personal accounts associated with a specific communication channel. Tess is capable of interpreting free text messages; alternatively, users can opt for preselected responses similar to existing chatbots. This enhances Tess’ capacity to deliver more personalized and integrative interventions.
Content was specially selected, expanded upon, and tested by mental health professionals for the purpose of this study. Furthermore, 30 min to 1-hour interviews were conducted with students, professors, and university counselors to support content development. User acceptance testing was conducted with a small group of students, which provided feedback to enhance the quality and reliability of interventions and scripts. A modest amount of emojis were included in the conversations to increase user engagement.
To support the evaluation of duration and frequency of automatic messages from Tess, the test groups were assigned to 1 of the 2 experiments. Group 1 received daily messages from Tess over a 2-week period, introducing new topics or following up on previously discussed concerns. Group 2 received the same content and option for follow-up messages from Tess, but with biweekly messages over a 4-week period. It is of note that Tess was disguised as
Although Tess is capable of connecting users with a counselor in case of crisis, this study limited crisis support to match the methodology of a previous study [
Tess delivers mental health interventions that have repeatedly been shown to reduce symptoms of depression and anxiety, such as CBT [
During the study, Tess delivered interventions rooted in a variety of psychological modalities such as CBT, mindfulness-based therapy, emotionally focused therapy, acceptance and commitment therapy, motivational interviewing, self-compassion therapy, and interpersonal psychotherapy. For example, journaling and relaxation strategies are used across multiple modalities, although the strategy and language used to deliver these interventions vary. Tess is structured to reply with prescripted statements, reviewed by mental health professionals, to replicate an empathic response that is appropriate to the participants’ inputted emotion or concern [
Just as therapists adjust their style to accommodate a client’s therapeutic preference over time, Tess gathers feedback to deliver interventions that best meet a user’s needs. After every intervention, Tess asks a simple question such as “was that helpful?,” to which user replies are coded as either positive, negative, or neutral. For example, if a user responds positively (ie, yes, thank you) to a CBT-based intervention and negatively (ie, no, not really) to self-compassion therapy, Tess will deliver more interventions rooted in CBT. For users who respond negatively or neutrally, Tess will continue to offer alternative interventions until the user responds positively or voluntarily ends the conversation.
Participants in the information-only control group were provided with an electronic link to the NIMH’s eBook on depression among college students [
Participants were invited to take the following assessments via a closed email survey so that only those who were invited could gain access. All assessments were delivered through Google Forms.
The PHQ-9 [
The Generalized Anxiety Disorder 7-item scale (GAD-7) [
The Positive and Negative Affect Schedule (PANAS) [
A user satisfaction survey was created, tested for usability and technical functionality, and delivered to all participants at the end of the study to gather qualitative results. The survey included 9 questions, with 4 scaled questions, such as: “How satisfied were you overall?” and “How satisfied were you with the content?” as well as 2 open-ended questions, such as “What was the best thing about using the chatbot?” The remaining 3 questions were forced choice with response options of yes or no, such as “Did you learn anything new?.” Finally, for test group participants only, we measured engagement based on the number of messages exchanged between Tess and the participants per group and in total.
Analyses were conducted using SPSS. A multivariate analysis of covariance (MANOVA) was used to compare the anxiety (GAD-7), depression (PHQ-9), and PANAS scale means of male and female students for the 3 groups, namely, control, group 1 (Tess for 2 weeks), and group 2 (Tess for 4 weeks). The multivariate analysis showed significance between the control group and group 1,
A post hoc analysis showed a significant difference was found with the Tukey’s Honestly Significant Difference Test (
In the control group, 67% (16/24) of participants were females, 29% (7/24) were males, and 4% (1/24) identified as nonconforming. The average age for the control group was 22.5 years. The majority of the control group participants were white (11/24, 46%). The remainder of the control group participants were Asian (8/24, 33%), other (3/24, 13%), and African American (2/24, 8%).
Group 1 consisted of 17 (17/24, 71%) females and 7 (7/24, 29%) males out of 24 participants. The average age for group 1 was 24.1 years. This group consisted of mostly white (13/24, 54%) and Asian (11/24, 46%) participants.
Group 2 consisted of 19 (19/26, 73%) females and 7 (7/26, 27%) males out of 26 participants. The average age for group 2 was 22.2 years. This group had mostly Asian participants (18/26, 69%), with 31% (8/26) being white.
Demographic and clinical variables of participants at baseline.
Demographic and clinical variables | Information control | Tess group 1 | Tess group 2 | |
Depression (PHQa-9) | 8.17 (4.2) | 6.67 (4.6) | 7.04 (4.9) | |
Anxiety (GADb-7) | 9.46 (3.9) | 6.71 (4.0) | 7.5 (4.9) | |
Positive affect | 22.13 (1.4) | 19.88 (1.4.) | 21.31 (1.3) | |
Negative affect | 15.75 (1.3) | 13.08 (1.3) | 14.38 (1.3) | |
Age in years, mean (SD) | 22.5 (4.0) | 24.1 (5.4) | 22.19 (2.8) | |
Female | 16 (67) | 17 (71) | 19 (73) | |
Male | 7 (29) | 7 (29) | 7 (27) | |
Nonconforming | 1 (4) | 0 (0) | 0 (0) | |
African American | 2 (8) | 0 (0) | 0 (0) | |
Asian | 8 (33) | 11 (46) | 18 (69) | |
White | 11 (46) | 13 (54) | 8 (31) | |
Other | 3 (13) | 0 (0) | 0 (0) |
aPHQ: Patient Health Questionnaire.
bGAD: Generalized Anxiety Disorder Scale.
Statistical power calculations using MANOVA revealed a moderate to large effect size (Cohen
A statistically significant difference was found between the control group and group 1, which had unlimited access to Tess with daily check-ins for 2 weeks (
Change in depression by group (patient health questionnaire-9 score).
A statistically significant difference was found between the control group and both test groups 1 and 2 for symptoms of anxiety as measured by the GAD-7. Group 1 (
A statistically significant difference was found on the GAD-7 between the control group and both groups 1 and 2. For group 1, the
A statistically significant difference was found on the PANAS between the control group and group 1 (
Change in anxiety by group (generalized anxiety disorder-7 score).
Qualitative results to post survey questions.
Post survey questions | Control group (N=24), n (%) | Tess group (N=50), n (%) |
Overall satisfaction | 14 (60) | 43 (86) |
Content satisfaction | 15 (65) | 40 (80) |
Extend emotional awareness | 17 (73) | 43 (86) |
Learned something new | 10 (43) | 40 (80) |
Information relevant to everyday life | 11 (47) | 40 (80) |
More comfortable with therapeutic process | 11 (47) | 32 (64) |
Thematic flow of participants’ most favored features while interacting with Tess.
A total of 48 open comments were received as feedback from 50 participants. Overall, 2 participants appeared to find their interaction with Tess to be particularly meaningful:
Based on our interactions I do somewhat feel like I’m talking to a real person and I do enjoy the tips you’ve given. In that sense, you’re better than my therapist in that she doesn’t necessarily provide specific ways I can better myself and problems.
I’ve been learning new things and I have some ideas on ways I can make small changes that could help me!.
Engagement was measured based on the number of messages exchanged between Tess and the participants. The X2AI Inc administration panel was used to calculate engagement metrics reported in this section. Participants from both test groups exchanged a total of 14,238 messages with Tess. Group 1 exchanged an average of 283 messages with daily pings and unlimited access to Tess for 2 weeks (SD 147.6; median 278; range 72-755). Group 2 exchanged an average of 286 messages with biweekly pings and unlimited access to Tess for 4 weeks (SD 104.6; median 288; range 133-535). Although we were unable to measure engagement of the NIMH eBook as it was not possible to track page views or URL click rates, 15 (15/24, 63%) control group participants reported that they were satisfied with the content supplied.
Thematic flow of participants’ least favored features while interacting with Tess.
The objective of this study was to assess the feasibility and efficacy of using an integrative psychological AI to reduce self-identified symptoms of depression and anxiety in college students. The methodology and results aligned with those from a previous randomized trial, which examined the potential for a chatbot to deliver CBT-based interventions to college students [
Results revealed that both test groups 1 and 2 experienced a significant reduction in symptoms of anxiety with unlimited access to Tess over the course of 2 or 4 weeks. Furthermore, the test group that received daily check-ins from Tess over 2 weeks experienced a significant reduction in symptoms of depression. Participants who interacted with Tess displayed higher levels of engagement and overall satisfaction than those from the control group. Test group participants indicated that the content was more relevant to their everyday life and made them more comfortable with the therapeutic experience.
Although this study included participants across 15 universities across the United States, the generalizability of results is limited, particularly as socioeconomic status was not formally assessed. The recruitment method further limits generalizability as we are unable to evaluate differences between the participants in this study who were recruited via social media and individuals who may use Tess but are not active on social media. The methodology called for 2 test groups and 1 control group, making the number of participants per group more limited. In addition, this study did not collect follow-up data to assess if benefits were sustained over time. Alternative to previous studies, the control group experienced a slight increase in symptoms of anxiety and depression, suggesting that the eBook was not a sufficient form of mental health support. One possible explanation for this outcome is that the eBook may have increased awareness of symptoms without providing ongoing treatment, leading to an increase over time. One study revealed that consumers of self-help books are more sensitive to stress and show higher depressive symptomatology [
Traditional therapeutic methods allow for emotional assessment on many different levels, including facial expressions, body cues, tone of voice, and language. The psychological AI used in this study delivered interventions via conversation, and therefore emotion identification was limited to language. It is unclear how much this limited the psychological AI’s assessment of emotion, as language is the most readily available nonphenomenal access people have to emotions. Assessing emotion through facial expressions [
Finally, the system errors, as expressed during qualitative feedback, are explained by the research team’s limited resources and attempt to keep all content approved by experts intact. During the study, changes to the system were restricted, and so the research team was unable to report errors related to natural language processing or emotion mismatch until the completion of the study.
Aligned with results from a previous study, using Tess was associated with a significant reduction in depression and anxiety as measured by the PHQ-9 and GAD-7, respectively. The effect size (Cohen
With a growing demand for scalable solutions that deliver more cost-effective mental health support, it has been shown that CAT is capable of delivering empirically validated treatments for depression and anxiety [
Previous studies suggest that individuals are more willing to disclose personal information to a psychological AI than to a
I do somewhat feel like I’m talking to a real person.
...you’re better than my therapist [who] doesn’t necessarily provide specific ways I can better myself.”
And that Tess was able to:
coach [the participant] through a difficulty.
These comments reinforce the potential for psychological AI to remove barriers and stigma and operate as an adjunct to traditional therapeutic methods.
This study reveals that AI offers a cost-effective and accessible mental health solution and may serve as a scalable tool to complement traditional treatment methods. Although integrative psychological AI is not designed or intended to replace the role of a trained therapist, Tess emerges as a feasible option for delivering emotional support. The results support and expand on findings from a previous randomized controlled trial [
Recruitment flyer.
Consolidated Standards of Reporting Trials-eHealth checklist.
Informed consent.
Multiple comparisons table.
Multivariate test table.
Tests of between-subjects effects.
Intervention sample GIF.
artificial intelligence
computer-assisted therapy
cognitive-behavioral therapy
Diagnostic and Statistical Manual of Mental Disorders
generalized anxiety disorder
multivariate analysis of covariance
mobile health
National Institute of Mental Health
positive and negative affect scale
Patient Health Questionnaire
The authors would like to thank Russell Fulmer, PhD, for academic advisory contributions; Angela Joerin, MS, and LLP for project management; Bre Gentile, PhD, for statistical analysis; Lysanne Lakerink for research design and implementation; Michiel Rauws, MBA, for review and approval of the manuscript; Romi Sadlik for manuscripts edits; X2AI for providing a budget for the student reimbursements and giving the research team free access to their customization platform and chatbot.
AJ and MR are employees of X2AI Inc that created the intervention (Tess) that was used in this trial and, therefore, have financial interest in that company. X2AI Inc covered the cost of participant incentives.