Published on in Vol 8, No 2 (2021): February

Preprints (earlier versions) of this paper are available at, first published .
Suitability of Text-Based Communications for the Delivery of Psychological Therapeutic Services to Rural and Remote Communities: Scoping Review

Suitability of Text-Based Communications for the Delivery of Psychological Therapeutic Services to Rural and Remote Communities: Scoping Review

Suitability of Text-Based Communications for the Delivery of Psychological Therapeutic Services to Rural and Remote Communities: Scoping Review


1The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Penrith, Australia

2Centre for Work Health and Safety, NSW Government, Gosford, Australia

Corresponding Author:

Mark Antoniou, PhD

The MARCS Institute for Brain, Behaviour and Development

Western Sydney University

Locked Bag 1797

Penrith, 2751


Phone: 61 97726673


Background: People living in rural and remote areas have poorer access to mental health services than those living in cities. They are also less likely to seek help because of self-stigma and entrenched stoic beliefs about help seeking as a sign of weakness. E-mental health services can span great distances to reach those in need and offer a degree of privacy and anonymity exceeding that of traditional face-to-face counseling and open up possibilities for identifying at-risk individuals for targeted intervention.

Objective: This scoping review maps the research that has explored text-based e-mental health counseling services and studies that have used language use patterns to predict mental health status. In doing so, one of the aims was to determine whether text-based counseling services have the potential to circumvent the barriers faced by clients in rural and remote communities using technology and whether text-based communications, in particular, can be used to identify individuals at risk of psychological distress or self-harm.

Methods: We conducted a comprehensive electronic literature search of PsycINFO, PubMed, ERIC, and Web of Science databases for articles published in English through November 2020.

Results: Of the 9134 articles screened, 70 met the eligibility criteria and were included in the review. There is preliminary evidence to suggest that text-based, real-time communication with a qualified therapist is an effective form of e-mental health service delivery, particularly for individuals concerned with stigma and confidentiality. There is also converging evidence that text-based communications that have been analyzed using computational linguistic techniques can be used to accurately predict progress during treatment and identify individuals at risk of serious mental health conditions and suicide.

Conclusions: This review reveals a clear need for intensified research into the extent to which text-based counseling (and predictive models using modern computational linguistics tools) may help deliver mental health treatments to underserved groups such as regional communities, identify at-risk individuals for targeted intervention, and predict progress during treatment. Such approaches have implications for policy development to improve intervention accessibility in at-risk and underserved populations.

JMIR Ment Health 2021;8(2):e19478



Defining E-Mental Health Services

E-mental health is an umbrella term used to describe services delivered via online, mobile, or phone-based platforms to diagnose, treat, or prevent mental health conditions, such as depression, anxiety, and substance abuse. This burgeoning field includes websites offering static information and resources such as Headspace and Beyond Blue [1,2], peer support services such as 7 Cups [3], interactive counseling services via telephone such as Kids Help Line [4], videoconferencing, email, and text-based chat (eg, SMS). With numerous approaches to e-mental health, some services use several modes of delivery (eg, Mental Health Online offers therapy via email, chat, and video). Technology has evolved rapidly, and the number of e-mental health services is growing, but without a rigorous evaluation of service delivery, much remains unknown.

Context and Scope for the Review

E-mental health services can potentially fill an important service gap in the context of improving mental health outcomes for communities outside major cities. In Australia, these account for 28% of the population (over 7 million people) [5]. Living in rural and remote communities is associated with poorer health outcomes, including mental illness. As remoteness increases, the prevalence of mental illness and the incidence of self-harm and suicide also increase [6]. Suicide rates are 40% higher in rural areas than in cities, and double in remote areas, whereas suicide rates in regional and rural areas are increasing faster than in capital cities [7]. Farmers are at an increased risk of suicide compared with other occupations in rural and remote communities [8-10]. People living in rural areas generally have poorer access to health services than those living in cities for several reasons. First, fewer mental health professionals work in rural and remote areas [11], rendering access to mental health services more difficult owing to time requirements, distance, and limited service capacity. Public transport is often not viable in low-density rural areas, which is particularly problematic in Australia [12]. Second, stigma toward mental health issues becomes a help-seeking barrier in smaller communities, because of fears of compromised confidentiality and being the topic of community gossip [13]. This may be more pronounced for males, for whom self-stigma and entrenched stoic beliefs about help seeking as a sign of weakness can delay psychological counseling or prevent engagement with mental health services altogether [13,14]. Per capita needs for mental health services in rural and remote areas are at least as great as in cities; therefore, these barriers to mental health service delivery are a cause for concern. Through this lens of concern, we approach our review.

With the proliferation of smartphones, mental health services can use technology to circumvent barriers and span great distances to reach individuals. E-mental health initiatives can potentially make a valuable contribution to service delivery in rural areas, although more research is needed [15]. Two aspects of text-based counseling services are especially likely to appeal to people living in rural and remote communities. First, the potentially anonymous nature of text-based interactions with therapists will appeal to individuals who are concerned with stigma and are reluctant to engage with face-to-face services. Second, text-based counseling allows clients to interact in real time with a qualified therapist at a time convenient for them, without the need to take time off work. Encouragingly, there is evidence that clients who have accessed e-mental health services show comparable outcomes to those in traditional face-to-face counseling services, as described previously [16].


This scoping review maps the research that has explored text-based e-mental health counseling services and studies that have used language use patterns to predict mental health status. Our search considered publications in both peer-reviewed and gray literature to capture existing and emerging research within the field. In addition to providing an overview of the literature, this review aims to identify priorities for future research. As will become clear, text-based communications lend themselves to predictive analyses more readily and on a larger scale than spoken communication.


We performed a scoping review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) Extension for Scoping Reviews checklist (Multimedia Appendix 1). A scoping review protocol was created to guide the process and is available from the corresponding author upon request.

Eligibility Criteria

It was decided that relatively broad inclusion criteria would be used to consider all publications with reference to the relevant mental health conditions (ie, depression, suicidal ideation, and anxiety), online counseling, and linguistic indicators for psychological conditions. All publication types were available for selection (journal articles, conference proceedings, reports, and unpublished theses). Therefore, studies were included if they (1) involved an intervention that occurred in real time, (2) were text-based, and (3) involved a qualified human clinician.

Publications were also considered for inclusion if they were related to mental health or counseling more generally if they had a text-based component. In addition, studies were included if they examined whether text patterns could be used to predict mental health status. Only articles written in English were included in this study.

Our scope is limited to the literature that examined e-mental health approaches involving text-based, real-time communication with a qualified human therapist and the predictive power of language use patterns. Within the eHealth sector, SMS text messaging and email are increasingly used for mental health care [17], medication adherence [18], treatment compliance [19], aftercare support [20], and reminders and supportive messages between face-to-face visits [21,22]. Encouragingly, virtual interventions incorporating scheduled messages or emails to support client participation have shown reductions in symptoms of depression, anxiety, and stress [21,23-25]. However, text-based technologies or the provision of mental health services by virtual agents or artificial intelligence algorithms cannot capture the dynamic back-and-forth communication exchanges of interest here and are not covered in the review. Therefore, studies that fell into the above categories were excluded.

Information Sources

Articles were identified by searching through the following databases: PsycINFO, PubMed, ERIC, and Web of Science.

Search Strategy

A literature search was conducted in July 2019 and updated in November 2020. The following search terms were used: (“therap*” OR “counsel*” OR “intervention” OR “depress*” OR “suicid*” OR “anxiety”) AND (“internet” OR “text” OR “linguistic” OR “word use” OR “natural language processing”). The search strategy was modified according to the different databases. The databases were searched from inception to November 24, 2020. As a final step, the reference lists contained within selected publications were searched for other relevant publications.

Selection of Sources of Evidence

Screening is a two-step process. AD removed duplicates and screened titles and abstracts to determine relevance to the topics of mental health and text-based counseling. These publications were then read in full to determine their relevance to the research question. CLC and MA checked the excluded studies and confirmed that the exclusion criteria were applied correctly.

Data Charting Process

We created a standardized template to extract relevant data from the included studies, such as publication type, journal name, reference to text-based e-mental health (or its variants) in the article title, reference to psychological conditions in the publication title, country of focus of the research, geographic location of data collection (urban, rural, etc), sample size of individual groups, and the gender composition of the sample. Data were extracted by AD, DE, and CLC and cross-checked by MA to ensure correctness and completeness.

Data Analysis and Synthesis

We synthesized the collated data using descriptive statistics (frequencies and proportions). Microsoft Excel was used to analyze the data.

Study Selection

We identified 9134 publications in our initial search. From these, 1487 duplicates were removed, leaving 7647 records for consideration. Next, titles and abstracts were screened based on our eligibility criteria, and 7244 publications were excluded. A further 333 publications were excluded during the full-text screening stage. The PRISMA flow diagram is presented in Figure 1. The full list of the included studies is provided in Multimedia Appendix 2 [16,26-95].

Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) flow diagram.
View this figure

Characteristics of Sources of Evidence

Of the 70 articles included in this review, 26% (18) reported positive outcomes following client participation in text-based e-mental health services, whereas only 4% (3) reported that text-based services were not effective in meeting the needs of clients. A total of 16% (11/70) of studies offered an insight into why some individuals are more or less likely to engage in text-based communications. Furthermore, 33% (23/70) analyzed text-based communications to identify predictors of mental health status, and 19% (13/70) specifically analyzed text-based communication during the course of therapy in an attempt to predict future mental health status. More studies were published in 2018 and 2019 than in any other years (n=10 and 11, respectively), and more than half of the studies (40/70, 58%) had been published in the last 5 years, with five published in 2020 (5/70, 7%). Forty-five studies (45/70, 64%) reported participants’ sex, and in the majority of those studies (34/45, 76%), a greater proportion of participants were female. The table in Multimedia Appendix 2 gives details of the number of participants, their age range, sex, background information/psychological status, and the location of the study for each source cited.

Text-based communication between clients and mental health professionals may involve SMS text messages, mobile phone apps, or internet chats. This review reveals that (1) few studies of these services have been conducted, (2) existing studies focus largely on children and adolescents in urban and suburban areas, (3) the quality of the studies varies considerably, and (4) fundamental questions remain unanswered. We first reviewed studies that reported a positive effect of text-based chat therapy on mental health outcomes.

Effectiveness of Text-Based E-Mental Health Services

In total, 18 studies reported that text-based e-mental health services were effective in treating mental health conditions. A single session of text-based counseling reduced anxiety to a degree comparable to traditional face-to-face counseling [16]. Similarly, text-based counseling increased the level of hope in a sample of young people (16-25 years) at the 6-week follow-up compared with those who did not pursue online counseling [26].

Fukkink and Hermanns examined children’s well-being (8-18 years) after accessing the Dutch Kindertelefoon phone help line or online text-based chat service. In the first study [27], 902 children (mostly girls) completed a 2-item mental health survey before and after a session (339 chats and 563 phones). Text-based conversations lasted approximately 30 min, whereas telephone conversations lasted only approximately 8 min on average. Both counseling modes increased children’s sense of well-being and decreased their perceived problem burden. Interestingly, the effect was more pronounced for text-based chats than for telephone communication. A follow-up questionnaire completed by 223 children indicated that benefits persisted 1 month after the intervention, at which time the 2 modes did not differ. In the second study [28], which analyzed a sample of 95 conversations (53 chat and 42 telephone), both service modes improved children’s well-being and decreased their perceived problem burden.

An analysis of the Kids Help Line service available to Australian youths [29] revealed that telephone counseling sessions (45-60 min) were superior to text-based sessions (50-80 min), although text-based counseling (chosen by clients experiencing more severe mental health issues) was still rated as helpful.

A series of studies examining the text-based subscription service Talkspace—within which clients interact with licensed therapists via text-based chat using an app—also showed positive mental health outcomes. A feasibility study examined clinical outcomes, client satisfaction, and the quality of the therapeutic relationship among 57 questionnaire respondents who had been using Talkspace for 3-4 months [30]. Comparisons of pre- and posttherapy scores of psychiatric distress indicated clinically significant improvements in 46% (25/54) of the sample. Self-reports of client satisfaction predominantly rated text therapy as the same as or better than face-to-face therapy regarding convenience, access, effectiveness, and progress with problems. The quality of the therapeutic relationship (accounting for 30% of the variance in reported clinical improvements) was rated significantly lower than that of in-person treatments. Another study evaluated outcomes for 51 Talkspace clients (mean age 34 years, SD 9; 67% female) after 14-15 weeks of treatment and found clinically significant posttherapy symptom reduction for depression (31/38, 84%) and anxiety (25/39, 64%) [31]. Data from 267 users indicated that the service was predominantly accessed by females (198/267, 74.2%), aged 21-50 years (243/267, 91.0%; median 34 years), with a college education (212/267, 79.4%), living in (sub)urban locations (232/267, 86.9%). Past barriers to face-to-face treatment reported by 240 participants included cost (130/240, 54.2%), excessive time and inconvenience (101/240, 42.1%), and face-to-face therapy not being helpful (67/240, 27.9%) [32]. These findings indicate that Talkspace is effective in addressing a range of mental health issues, although the results should be interpreted with caution because the authors were affiliated with the service.

Evaluating 318 users’ experiences with BetterHelp, a multimodal digital psychotherapy intervention using text, video, and/or phone communication, Marcelle et al [33] found that depression symptom severity was reduced after intervention, with no differences found for gender, socioeconomic status, or self-reported physical health status. The intervention involved a significant text-based chat component as well as other modes of service delivery.

One of the most rigorous studies concerning the effectiveness of text-based counseling [34] randomly assigned 297 adults with depression (18-75 years) to online cognitive behavioral therapy (up to ten 55-min sessions) in addition to their regular medical care or to a control condition. At the 4-month follow-up (n=210), 38.1% (43/113) patients in the intervention group had recovered from depression, compared with 24% (23/97) in the control group, with therapeutic gains maintained after 8 months, providing strong evidence that e-mental health services (including text-based counseling) offer effective and lasting treatment for serious mental health conditions.

Young people with depressive symptoms (n=263) were randomly assigned to either an intervention condition (up to 5 real-time chat sessions with a trained health care professional) or a waiting list [35]. Text-based chat was more effective than waiting list control in reducing depressive symptoms, with effects increasing from follow-up to posttest, suggesting benefits extended beyond the intervention. An online cognitive behavioral therapy intervention (8 modules and 8 chat sessions over 8 weeks) was more effective than an attention counseling control condition in reducing depressive symptoms in a sample of 70 adolescents (15-19 years) [36], with improvement maintained after 6 months.

Goldin et al [37] evaluated the effectiveness of the Ascend text-based program for treating depression (8 modules over 8 weeks). Clients were primarily female (22/22, 100% in study 1; 76/95, 80% in study 2), young adults (mean age 23.2 years, SD 1.1 in study 1; mean age 32.0 years, SD 9.9 in study 2), and college educated. On average, the therapist spent 20 min chatting with each patient per week. The intervention reduced symptoms of depression, suggesting that text-based chat may be effective in augmenting therapy.

Recent evaluation of Australian crisis support service, Lifeline Text, linked text message counseling to improved mental health [38]. Help seekers with suicidal ideation and poor mental health symptoms, including anxiety, depression, or issues related to domestic and family violence, showed overall reductions in distress, increased social connection, and greater confidence in their ability to cope after the text conversation.

These studies provide compelling evidence that text-based counseling alleviates mental health issues and is effective in treating psychological distress and depression. It may also be effective for treating substance abuse [39,40], reducing high-risk behaviors [41], and improving the subjective experiences of individuals with attention deficit hyperactivity disorder and/or autism [42].

Limitations of Text-Based E-Mental Services

Three studies did not report positive therapeutic outcomes following text-based counseling. Almost half of a sample of African American women reported that text-based interventions for depression and anxiety were not a desirable form of treatment [43]. A randomized controlled trial [44] showed that depression symptoms were reduced for a standard treatment with automated internet-delivered depression management more than with standard care plus optional text-based chat with a therapist. Despite encouragement, few users chose the chat option, making it unclear whether the chat component was ineffective or whether lack of engagement limited its usefulness. Lack of engagement was problematic in a trial involving a chat-based intervention aimed at adolescents, which was abandoned because of insufficient recruitment [45]. Interestingly, two-thirds of those who declined to participate also refused usual care; the remaining third cited preferences for face-to-face counseling.

Two previous systematic reviews [46,47], evaluating evidence from a total of 14 studies (a subset of those reviewed here), reported emerging support for the use of online text-based chat as an alternative to face-to-face and phone-delivered therapy, despite some mixed findings that can be accounted for by variability in intervention design (eg, target of treatment, length, and type of intervention). Andersson et al [48-50] argue that therapist-guided internet-delivered treatments are effective in treating a range of mental health conditions, can be as effective as face-to-face treatments, and lead to sustainable improvements. However, there will always be clients who prefer face-to-face treatment. To understand why, we next examined client experiences of text-based e-mental health services.

Client Experiences of Text-Based Counseling

Eleven studies examined client experiences of text-based counseling and how fit and preference affect outcomes. All suggest that people generally disclose emotions similarly in computer-mediated and face-to-face communications, and there is evidence that computer-mediated communication can encourage the expression of emotion [51] and true-self qualities [52]. To understand why text-based modes of delivery might better suit some clients than others, we review studies that explored why some people are more likely to engage in text-based counseling.

For children accessing the Kindertelefoon e-mental health service [28], chat was viewed positively because children appreciated the distance (anonymity, security, privacy, and control over self-presentation), especially when discussing emotional problems. Those who preferred phone conversations perceived chat as impersonal or distant. These opposing assessments of text-based communication (private vs impersonal) are a recurring theme in studies investigating why clients do or do not engage with text-based counseling.

In an interesting case study [53], a client with severe obsessive-compulsive disorder received 40 psychotherapy sessions via videoconferencing during which the client disclosed a number of low-shame issues. Adding text chat in later sessions promoted the disclosure of clinically important information, particularly regarding feelings of shame, guilt, and embarrassment, which the client had difficulty disclosing in video-only sessions. Switching to text chat eliminated anxiety-provoking eye contact when disclosing more personally confronting topics.

Interviews with 24 patients with depression [54] were used to evaluate an online e-mental health treatment [34]. Some patients experienced difficulty expressing themselves in writing, but for others, it facilitated re-examination of thoughts, feelings, and behaviors, and reflection on therapeutic exchanges. Patients who were not comfortable with online communication disliked the less fluid interactions, reduced content covered during sessions, and waiting for therapist responses. For others, the delays created space to think, reflect, and communicate without interruption. Three studies observed that text-based chat sessions take longer than phone counseling sessions [27,29] and generate fewer words than verbal exchanges [55]. Overall, the findings suggest that when patients with depression are comfortable with online communication and with expressing their feelings in text, the anonymity of an online therapeutic relationship may be attractive, and online therapy is an acceptable and helpful alternative to face-to-face services.

Analyzing Text-Based Communications to Identify Predictors of Mental Health Status

Studies investigating linguistic predictors of mental health status are scarce. Few have linguistically analyzed texts from individuals at risk of mental health problems. In total, 13 studies analyzed text-based therapy transcripts with the aim of predicting future mental health status. We also review 9 studies (from the much larger literature on natural language processing) that mined social media or internet forum posts with the specific aim of identifying linguistic patterns that predict mental health status because these are useful in situating the emerging text-based counseling research within the literature and offer insight into likely future directions. Recent advances in computational linguistics have increased the sophistication and complexity of analyses, enabled by the development of computational linguistic tools such as Linguistic Inquiry and Word Count (LIWC) [56], which produces statistical distributions of words within predefined and psychologically meaningful categories (eg, function words, pronouns, or verbs).

With LIWC (and similar algorithms), researchers can explore language patterns across the mental health conditions of interest. Ten studies have used LIWC to show that people with depression tend to focus on themselves. This was realized linguistically as an increased use of first-person singular pronouns (eg, I, me, my, mine) [57-59]. However, first-person singular pronouns may be a linguistic marker of general proneness to distress or negative emotionality rather than depression [60,61]. Early studies analyzed formal written texts for such patterns. Stirman and Pennebaker [62] compared 300 poems written by 18 poets, half of whom had committed suicide. Suicidal poets made more frequent use of first-person singular pronouns and less use of collective pronouns (eg, we, our). Rude et al [63] analyzed narratives written by college students who were depressed, formerly depressed, or had never been depressed. Depressed students used more first-person singular pronouns, more negative valence words, and fewer positive emotion words than those who had never been depressed. First-person singular pronoun use has been linked to the severity of depression and anxiety symptoms [64,65]. More recently, LIWC was used to examine whether first-person singular pronouns in patient interviews predict future mental health in a clinical sample of 29 inpatients undergoing psychodynamic treatment for depression [66]. First-person singular pronoun use did not predict depressive symptoms at baseline but predicted depressive symptom severity at follow-up 8 months later, especially the use of objective (me) and possessive (my) pronouns.

Eleven studies have investigated linguistic indicators of depression other than frequency of pronouns, in text data from social media platforms such as Twitter and Facebook [67] or online forums such as Reddit [68]. In social media posts, the onset of major depressive disorder can be predicted by reduced user engagement, use of negative valence words, and first-person pronouns [69]. Content words that could reliably predict depression included symptoms (eg, anxiety, withdrawal), disclosure (eg, enjoy, care), relationships (eg, home, friends), religion (eg, Jesus, bible), and treatment (eg, side effects, therapy). Regression models based on Facebook language use have been used to predict the degree [70] and incidence of depression [71].

Aspect-based sentiment analysis of online forum posts has been used to determine how individuals with different mental health conditions express themselves about various topics [72]. Al-Mosaiwi and Johnstone [73] found that internet forum users with depression, anxiety, and suicidal ideation used more absolutist words (ie, words indicating certainty). A content analysis comparing letters posted in online self-help groups for suicide, depression, and anxiety with letters posted in a control group [74] found that use of absolutist words was a better predictor of suicidal ideation than use of negative valence words or first-person pronouns [73,74] and can thus be a specific linguistic marker of depression, anxiety, and especially suicidal ideation.

Emotion words may also be predictive of mental health status. Owen et al [75] evaluated the efficacy of an online self-guided coping skills training and support intervention for women with early stage breast cancer. Greater use of words expressing anxiety, sadness (but not anger), and cognitive processing were associated with improved emotional well-being at follow-up, whereas greater expression of sadness (but not anxiety or anger) was associated with improved quality of life. This demonstrates that in a treatment context, limiting the analysis to emotion word use at a categorical level can potentially overlook subtle but meaningful differences in emotional expression.

Using more sophisticated measures that captured subtle changes in emotional state from negative emotion word instability, Seabrook et al [76] reliably predicted depression severity. They also created an emoji and internet slang supplement to the LIWC dictionary, which increased the accuracy of depression identification.

Low complexity in clients’ language has been shown to predict depression. Individuals undergoing treatment who were likely to remain depressed used fewer complex syntactic constructions such as adverbial clauses, perhaps because these require greater cognitive effort. Morales [77] also found that a number of LIWC content categories (eg, job, sad, sleep) were positively correlated with depression levels.

A recent study considered sex differences in fictional letters. Depressive symptoms were predicted by the ratios of pronouns to nouns and of verbs to nouns for men but by the ratios of finite verbs to number of sentences and of punctuation to number of sentences for women [78]. Thus, sex differences may need to be considered when identifying linguistic predictors.

In summary, there is a range of linguistic predictors at both the lexical and syntactic levels, which have been shown to correlate with mental health status and can be reliably identified with modern computational linguistic tools.

Analysis of Text-Based Communication During the Course of Therapy

Ten studies that examined transcripts of therapeutic conversation transcripts revealed changing narrative processes during the course of treatment, and these were linked to treatment outcomes (eg, [79-81]). For example, clients’ increasing use of reflexive language and decreasing use of external language in therapeutic conversation were associated with better therapeutic outcomes [79]. Exploratory text-based analysis of therapist emails can also provide insight into therapist behaviors that support therapy adherence [82].

One study examined word use in short essays written by outpatients treated for personality disorders 3 times over 2 years of treatment [83]. The use of first-person singular pronouns, negative emotion words, causation words (eg, because, effect), past tense verbs, and future tense verbs declined over treatment, whereas the use of positive emotion and present tense words increased. However, only a reduction in the use of negative emotion words was a reliable predictor of symptom improvement.

Van der Zanden et al [84] examined 7 word categories (first-person pronouns, positive emotions, negative emotions, causation, insight, discrepancy, and social processes) in text-based chat session transcripts of participants who completed 5-6 sessions in an online group course for young adults with depressive symptoms. Only increasing the use of discrepancy words (eg, should, wish) reliably predicted depression improvement.

Dirkse et al [85] analyzed texts from adults with generalized anxiety symptoms participating in a 12-module course of therapist-assisted, internet-delivered therapy. Greater use of negative emotion, anxiety, and sadness words positively correlated with heightened anxiety ratings; greater use of negative emotion, sadness, and anger words positively correlated with heightened depression ratings; and greater use of negative emotion and anger words positively correlated with heightened panic ratings. Patients’ self-rated mental health status correlated with decreasing use of emotion, anxiety, and causation words, and increasing use of past tense words.

Using data from the York I depression study [86], Huston et al [87] examined clients’ language use in face-to-face psychotherapy sessions. They analyzed 6 word categories (positive emotion, negative emotion, causation, past focus, negation, and first-person singular pronouns) in 24 sessions, early (T1) and late (T2) for 12 clients identified as having good (n=6) and poor (n=6) treatment outcomes. No significant change in language use patterns occurred from T1 to T2, but greater use of positive emotion words at T1 was associated with good treatment outcomes and greater use of past focus words at T1 with poor treatment outcomes. Acknowledging the limitations of word count analysis, the authors highlighted the need for research using more frequent sampling to explore language patterns as potential measures of therapeutic progress.

Two studies examined large databases of client-therapist interactions [88,89]. To determine whether discourse features of text message counseling conversations could predict clinical outcomes, Althoff et al [88] examined a large data set (approximately 80,000 conversations of approximately 40 messages each). After the session ended, clients’ answers to How are you feeling now? became their ground-truth labels. Clients with a smaller amount of self-focus tended to have more successful conversations. Tay [89] examined transcripts from 4 client-therapist dyads each lasting 20 sessions (472,009 words) that were taken from the counseling and psychotherapy transcripts, client narratives, and reference works.

A study of an internet-delivered pain management intervention for adolescents with chronic pain [90] used text mining and analytic techniques to messages between coaches and patients and successfully identified messages that raised concerns. Cluster analysis identified subgroups of individuals with common communication and engagement patterns that could be used to tailor interventions. Such techniques could conceivably also be applied to e-mental health services.

Calvo et al [91] offered a comprehensive review of natural language processing in mental health applications. Their discussion of computational linguistic analyses reveals that demographic, linguistic, behavioral, and social data may be combined to construct sophisticated models to identify at-risk individuals. Although no study to date has succeeded in the ultimate goal of accurately predicting suicide risk and guiding mental health professionals to intervene to save lives, progress has been made in recent years. A study by Ruiz et al [92] built 6 machine learning models to predict suicide risk levels from posts on the Reddit forum using a large training data set of 31,553 posts from 496 users. Naive Bayes was the most accurate model for flagging urgent suicide risks.

Quantitative measures of therapist emails can also shed light on the key aspects of program delivery and client adherence to eHealth programs [82]. Content analysis of 490 therapist emails to clients showed that therapist behaviors were represented by 8 behavior codes: deadline flexibility, task reinforcement, alliance bolstering, task prompting, psychoeducation, self-disclosure, self-efficacy shaping, and empathetic utterances. Client adherence to the program was supported by therapist communication that supported deadline flexibility and task reinforcement.

In summary, modern computational linguistic tools and techniques allow analyses that may help predict progress during treatment and may ultimately identify individuals at risk. They also offer an opportunity to quantitatively assess the interrelationship between therapist behaviors, program adherence, and success.

Principal Findings

The aim of this scoping review was to map the research that has explored text-based e-mental health counseling services and studies that have used language use patterns to predict mental health status. One of the aims of this study was to assess whether text-based counseling with a therapist could offer an e-mental health solution appropriate for underserved populations, such as rural and remote communities. Of the 9334 reviewed records, 70 met the eligibility criteria.

Although more research is needed to understand the nuances and complex interactions necessary for personalized care, converging evidence shows that text-based counseling services and interventions are effective in treating a variety of mental health conditions. This review of the relevant literature gives reason to be optimistic regarding the potential for text-based interventions to contribute to addressing the mental health needs of underserved populations. Evidence suggests that text-based counseling produces desirable outcomes for a range of mental health conditions for clients willing to engage with this delivery mode.

However, there is a dearth of research concerning its applicability to underserved communities, including those living in rural and remote regions. Our review has revealed that the characteristics of text-based approaches to service delivery (longer sessions, fewer words transmitted, more anonymous or impersonal interactions) are perceived positively by some cohorts but negatively by others, and this perception is likely to affect engagement with such services. This led Hoermann et al [47] to suggest that if this mode of intervention delivery generates similar effectiveness but longer session times, generating fewer words, and in some groups, lower satisfaction, this draws into question the clinical practicality of this mode of delivery. However, our focus on identifying a desirable delivery mode of mental health services for people living in rural and remote communities leads us to a different conclusion. First, if text-based communications are effective, they can provide much needed mental health services. Second, for certain segments of the population, the supposed limitations of text-based communication (slower communication, facelessness, and lack of voice) may actually be advantageous or desirable. Increased anonymity and privacy may make some individuals more likely to engage and reveal more truthful information. Some therapists also prefer chat sessions because they allow editing outside the client’s view; thus, minimizing possible awkwardness when fumbling for words in face-to-face communication [16]. Thus, text-based counseling has the potential to both increase access to mental health services and improve clinician-client relationships.

It is important to acknowledge that text-based counseling requires a level of technology literacy, and this may present a barrier for some clients to engage with such e-mental health services (and this is likely to correlate with age). Service providers should heed this limitation by designing their services in accessible ways that minimize the burden for clients who lack technical expertise (eg, SMS may be more accessible than internet chat). Relatedly, high-cost services will not reach (and therefore not help) some of the population groups that are both high need and high risk. Many such groups have lower socioeconomic status and incomes and are unlikely to be able to afford or engage in high-cost or subscription-based approaches to treatment.

Finally, the transcripts of text-based therapy-related communications not only permit reflection on the part of the client but also allow researchers to conduct linguistic analyses to predict future mental status. The potential applications of an accurate, scalable approach to mental health are far-reaching, with implications for early screening and targeted interventions.

Future Directions in Analyses of Text Data

Computational linguistic techniques will continue to evolve as more sophisticated tools are developed to overcome the current limitations. For example, LIWC is unable to detect or compute context and cannot account for changes in meaning resulting from irony, sarcasm, or idioms [93], nor does it take into account negations [87] or qualifiers before words. Predefined LIWC dictionaries may not be sufficiently broad to account for the categories of interest [93]. Although LIWC does not permit the creation of new categories, new dictionaries can be added. Thus, some limitations can be overcome by using dictionaries composed of nonambiguous adjectives and expanded by a distributional strategy [94], whereas others may be addressed by supplementing LIWC with computational linguistic techniques such as unsupervised natural language processing models. To date, researchers must work around these limitations to conduct computational linguistic analyses of text data.

As chat transcripts automatically document the therapeutic process, the text may be used to predict future mental health status and identify at-risk individuals. The accuracy of such predictive models depends on the integrity of the training data. A biased data set affects the accuracy of the predictions or classifications the model makes. Another concern is how false negatives are dealt with, particularly in cases where an individual’s safety is at risk. This is an important consideration to bear in mind when such techniques are used to augment the diagnosis or treatment. It is also worth noting that the ethical and privacy risks associated with text-based counseling can be mitigated by using encryption and deidentifying transcripts (eg, names of people and places).

In addition, there is evidence that linguistic patterns in children differ from those in adults [95]. Therefore, it should not be assumed that predictive relationships between linguistic patterns and mental health status will apply equally across the lifespan. More research is needed to understand how developmental differences influence these predictive relationships.

Limitations of the Study

This scoping review covers a growing area of research, and thus fundamental questions need to be answered to advance the field. Are findings in one population or region transferable to another (eg, from adolescents in urban areas to adults in rural areas)? What obstacles might potentially limit such generalizability? In addition, the scope here is limited to the literature documenting text-based e-mental health services and is not an evaluation of the technologies used by those services. Given the increased development of mobile apps for many services, a review of e-mental health apps following the MARS protocol [96] would complement this work.


The review reveals a clear need for intensified research into how evidence-based e-mental health practices can reach and serve clients. This is especially important for communities where mental health services are limited, and individuals must overcome multiple barriers to help seeking (as is the case in rural and remote areas). This need is likely to increase as governments and organizations respond to the COVID-19 pandemic by exploring flexible working arrangements that include remote working and exacerbate social isolation due to social distancing policies.

Further investigation is required to determine whether transcripts of text-based communication between clients and therapists can be used to accurately predict mental health outcomes. Our review of the computational linguistic literature on text-based predictors of mental health suggests that large-scale automatic screening of mental illness and identification of at-risk individuals may soon be possible.


All authors contributed to the authorship of this review. The authors acknowledge funding from the NSW Government Centre for Work Health and Safety tender SWNSW/654567/2019.

Conflicts of Interest

None declared.

Multimedia Appendix 1

PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Reviews) checklist.

DOCX File , 84 KB

Multimedia Appendix 2

Study information.

DOCX File , 41 KB

  1. Headspace. 2020.   URL: [accessed 2020-08-03] [WebCite Cache]
  2. Beyond Blue. 2020.   URL: [accessed 2020-08-03] [WebCite Cache]
  3. 7 Cups.   URL: [accessed 2021-02-06] [WebCite Cache]
  4. Kidshelpline. 2020.   URL: [accessed 2021-02-06] [WebCite Cache]
  5. Regional population growth, Australia, 2007-2017. Australian Bureau of Statistics. 2018.   URL: [accessed 2020-08-03] [WebCite Cache]
  6. Mental health in rural and remote Australia. National Rural Health Alliance. 2017.   URL: https:/​/www.​​sites/​default/​files/​publications/​nrha-mental-health-factsheet-dec-2017.​pdf [accessed 2020-08-03] [WebCite Cache]
  7. Hazell T, Dalton H, Caton T, Perkins D. Rural suicide and its prevention: A CRRMH position paper. Centre for Rural and Remote Mental Health, University of Newcastle, Australia. 2017.   URL: [accessed 2020-08-03] [WebCite Cache]
  8. Andersen K, Hawgood J, Klieve H, Kõlves K, De Leo D. Suicide in selected occupations in Queensland: evidence from the State suicide register. Aust N Z J Psychiatry 2010 Mar;44(3):243-249. [CrossRef] [Medline]
  9. Kennedy AJ, Maple MJ, McKay K, Brumby SA. Suicide and accidental death in Australia's rural farming communities: a review of the literature. Rural Remote Health 2014;14(1):2517 [FREE Full text] [Medline]
  10. Milner A, Spittal MJ, Pirkis J, LaMontagne AD. Suicide by occupation: systematic review and meta-analysis. Br J Psychiatry 2013 Dec 2;203(6):409-416. [CrossRef]
  11. Mental health services in Australia: Mental health workforce. Australian Institute of Health and Welfare. 2020.   URL: https:/​/www.​​reports/​mental-health-services/​mental-health-services-in-australia/​report-contents/​mental-health-workforce [accessed 2020-08-03]
  12. Nutley S. Indicators of transport and accessibility problems in rural Australia. Journal of Transport Geography 2003 Mar;11(1):55-71. [CrossRef]
  13. Perceval M, Ross V, Kõlves K, Reddy P, De Leo D. Social factors and Australian farmer suicide: a qualitative study. BMC Public Health 2018 Dec 12;18(1). [CrossRef]
  14. Judd F, Jackson H, Komiti A, Murray G, Fraser C, Grieve A, et al. Help-seeking by rural residents for mental health problems: the importance of agrarian values. Aust N Z J Psychiatry 2006 Sep;40(9):769-776. [CrossRef] [Medline]
  15. Benavides-Vaello S, Strode A, Sheeran BC. Using technology in the delivery of mental health and substance abuse treatment in rural communities: a review. J Behav Health Serv Res 2013 Jan;40(1):111-120. [CrossRef] [Medline]
  16. Cohen GE, Kerr BA. Computer-Mediated Counseling. Computers in Human Services 1999 Apr 22;15(4):13-26. [CrossRef]
  17. Berrouiguet S, Baca-García E, Brandt S, Walter M, Courtet P. Fundamentals for Future Mobile-Health (mHealth): A Systematic Review of Mobile Phone and Web-Based Text Messaging in Mental Health. J Med Internet Res 2016 Jun 10;18(6):e135 [FREE Full text] [CrossRef] [Medline]
  18. Watson T, Simpson S, Hughes C. Text messaging interventions for individuals with mental health disorders including substance use: A systematic review. Psychiatry Res 2016 Dec 30;243:255-262. [CrossRef] [Medline]
  19. Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth chronic disease management on treatment adherence and patient outcomes: a systematic review. J Med Internet Res 2015;17(2):e52 [FREE Full text] [CrossRef] [Medline]
  20. Robinson S, Perkins S, Bauer S, Hammond N, Treasure J, Schmidt U. Aftercare intervention through text messaging in the treatment of bulimia nervosa—feasibility pilot. Int J Eat Disord 2006;38(8). [CrossRef] [Medline]
  21. Broom MA, Ladley AS, Rhyne EA, Halloran DR. Feasibility and Perception of Using Text Messages as an Adjunct Therapy for Low-Income, Minority Mothers With Postpartum Depression. JMIR Ment Health 2015;2(1):e4 [FREE Full text] [CrossRef] [Medline]
  22. Pisco Almeida AM, Almeida HS, Figueiredo-Braga M. Mobile solutions in depression: enhancing communication with patients using an SMS-based intervention. Procedia Computer Science 2018;138:89-96. [CrossRef]
  23. Bendtsen M, Müssener U, Linderoth C, Thomas K. A Mobile Health Intervention for Mental Health Promotion Among University Students: Randomized Controlled Trial. JMIR Mhealth Uhealth 2020 Mar 20;8(3):e17208. [CrossRef]
  24. Hadjistavropoulos HD, Peynenburg V, Mehta S, Adlam K, Nugent M, Gullickson KM, et al. An Internet-Delivered Cognitive Behavioral Therapy for Depression and Anxiety Among Clients Referred and Funded by Insurance Companies Compared With Those Who Are Publicly Funded: Longitudinal Observational Study. JMIR Ment Health 2020 Feb 04;7(2):e16005 [FREE Full text] [CrossRef] [Medline]
  25. Salamanca-Sanabria A, Richards D, Timulak L, Connell S, Mojica Perilla M, Parra-Villa Y, et al. A Culturally Adapted Cognitive Behavioral Internet-Delivered Intervention for Depressive Symptoms: Randomized Controlled Trial. JMIR Ment Health 2020 Jan 31;7(1):e13392 [FREE Full text] [CrossRef] [Medline]
  26. Dowling M, Rickwood D. A naturalistic study of the effects of synchronous online chat counselling on young people's psychological distress, life satisfaction and hope. Couns. Psychother. Res 2015 Jul 14;15(4):274-283. [CrossRef]
  27. Fukkink RG, Hermanns JMA. Children's experiences with chat support and telephone support. J Child Psychol Psychiatry 2009 Jun;50(6):759-766. [CrossRef] [Medline]
  28. Fukkink RG, Hermanns JM. Counseling children at a helpline: chatting or calling? J Community Psychol 2009 Nov;37(8):939-948. [CrossRef]
  29. King R, Bambling M, Reid W, Thomas I. Telephone and online counselling for young people: A naturalistic comparison of session outcome, session impact and therapeutic alliance. Counselling and Psychotherapy Research 2006 Sep;6(3):175-181. [CrossRef]
  30. Hull TD, Mahan K. A Study of Asynchronous Mobile-Enabled SMS Text Psychotherapy. Telemedicine and e-Health 2017 Mar;23(3):240-247. [CrossRef]
  31. DellaCrosse M, Mahan K, Hull TD. The effect of messaging therapy for depression and anxiety on employee productivity. J Technol Behav Sci 2019 Mar;4(8):1-5. [CrossRef]
  32. Nitzburg GC, Farber BA. Patterns of utilization and a case illustration of an interactive text-based psychotherapy delivery system. J. Clin. Psychol 2019 Jan 10;75(2):247-259. [CrossRef]
  33. Marcelle ET, Nolting L, Hinshaw SP, Aguilera A. Effectiveness of a Multimodal Digital Psychotherapy Platform for Adult Depression: A Naturalistic Feasibility Study. JMIR Mhealth Uhealth 2019 Jan 23;7(1):e10948 [FREE Full text] [CrossRef] [Medline]
  34. Kessler D, Lewis G, Kaur S, Wiles N, King M, Weich S, et al. Therapist-delivered Internet psychotherapy for depression in primary care: a randomised controlled trial. Lancet 2009 Aug 22;374(9690):628-634. [CrossRef] [Medline]
  35. Kramer J, Conijn B, Oijevaar P, Riper H. Effectiveness of a web-based solution-focused brief chat treatment for depressed adolescents and young adults: randomized controlled trial. J Med Internet Res 2014;16(5):e141 [FREE Full text] [CrossRef] [Medline]
  36. Topooco N, Berg M, Johansson S, Liljethörn L, Radvogin E, Vlaescu G, et al. Chat- and internet-based cognitive-behavioural therapy in treatment of adolescent depression: randomised controlled trial. BJPsych Open 2018 Jul;4(4):199-207 [FREE Full text] [CrossRef] [Medline]
  37. Goldin PR, Lindholm R, Ranta K, Hilgert O, Helteenvuori T, Raevuori A. Feasibility of a Therapist-Supported, Mobile Phone-Delivered Online Intervention for Depression: Longitudinal Observational Study. JMIR Form Res 2019 Jan 22;3(1):e11509 [FREE Full text] [CrossRef] [Medline]
  38. Williams K, Fildes D, Kobel C, Grootemaat P, Bradford S, Gordon R. Evaluation of Outcomes for Help Seekers Accessing a Pilot SMS-Based Crisis Intervention Service in Australia. Crisis 2020 Apr 28:1-8. [CrossRef] [Medline]
  39. Blankers M, Koeter MWJ, Schippers GM. Internet therapy versus internet self-help versus no treatment for problematic alcohol use: A randomized controlled trial. J Consult Clin Psychol 2011 Jun;79(3):330-341 [FREE Full text] [CrossRef] [Medline]
  40. Schaub MP, Wenger A, Berg O, Beck T, Stark L, Buehler E, et al. A Web-Based Self-Help Intervention With and Without Chat Counseling to Reduce Cannabis Use in Problematic Cannabis Users: Three-Arm Randomized Controlled Trial. J Med Internet Res 2015;17(10):e232 [FREE Full text] [CrossRef] [Medline]
  41. Lelutiu-Weinberger C, Pachankis JE, Gamarel KE, Surace A, Golub SA, Parsons JT. Feasibility, Acceptability, and Preliminary Efficacy of a Live-Chat Social Media Intervention to Reduce HIV Risk Among Young Men Who Have Sex With Men. AIDS Behav 2015 Jul;19(7):1214-1227 [FREE Full text] [CrossRef] [Medline]
  42. Wentz E, Nydén A, Krevers B. Development of an internet-based support and coaching model for adolescents and young adults with ADHD and autism spectrum disorders: a pilot study. Eur Child Adolesc Psychiatry 2012 Nov;21(11):611-622. [CrossRef] [Medline]
  43. McCall T, Schwartz TA, Khairat S. The Acceptability of Text Messaging to Help African American Women Manage Anxiety and Depression: Cross-Sectional Survey Study. JMIR Ment Health 2020 Feb 03;7(2):e15801 [FREE Full text] [CrossRef] [Medline]
  44. Kordy H, Wolf M, Aulich K, Bürgy M, Hegerl U, Hüsing J, et al. Internet-Delivered Disease Management for Recurrent Depression: A Multicenter Randomized Controlled Trial. Psychother Psychosom 2016 Jan;85(2):91-98. [CrossRef] [Medline]
  45. Crutzen R, Bosma H, Havas J, Feron F. What can we learn from a failed trial: insight into non-participation in a chat-based intervention trial for adolescents with psychosocial problems. BMC Res Notes 2014 Nov 20;7(1):824 [FREE Full text] [CrossRef] [Medline]
  46. Dowling M, Rickwood D. Online Counseling and Therapy for Mental Health Problems: A Systematic Review of Individual Synchronous Interventions Using Chat. Journal of Technology in Human Services 2013 Jan;31(1):1-21. [CrossRef]
  47. Hoermann S, McCabe KL, Milne DN, Calvo RA. Application of Synchronous Text-Based Dialogue Systems in Mental Health Interventions: Systematic Review. J Med Internet Res 2017 Jul 21;19(8):e267 [FREE Full text] [CrossRef] [Medline]
  48. Andersson G. Internet-Delivered Psychological Treatments. Annu Rev Clin Psychol 2016;12:157-179. [CrossRef] [Medline]
  49. Andersson G, Carlbring P, Titov N, Lindefors N. Internet Interventions for Adults with Anxiety and Mood Disorders: A Narrative Umbrella Review of Recent Meta-Analyses. Can J Psychiatry 2019 Jul;64(7):465-470 [FREE Full text] [CrossRef] [Medline]
  50. Andersson G, Rozental A, Shafran R, Carlbring P. Long-term effects of internet-supported cognitive behaviour therapy. Expert Rev Neurother 2018 Jan;18(1):21-28. [CrossRef] [Medline]
  51. Derks D, Fischer AH, Bos AE. The role of emotion in computer-mediated communication: A review. Computers in Human Behavior 2008 May;24(3):766-785. [CrossRef]
  52. Bargh JA, McKenna KYA, Fitzsimons GM. Can you see the real me? Activation and expression of the 'True self' on the internet. J Social Isssues 2002 Jan;58(1):33-48. [CrossRef]
  53. Stubbings DR, Rees CS, Roberts LD. New Avenues to Facilitate Engagement in Psychotherapy: The Use of Videoconferencing and Text-Chat in a Severe Case of Obsessive-compulsive Disorder. Australian Psychologist 2015 Jul 20;50(4):265-270. [CrossRef]
  54. Beattie A, Shaw A, Kaur S, Kessler D. Primary-care patients' expectations and experiences of online cognitive behavioural therapy for depression: a qualitative study. Health Expect 2009 Mar;12(1):45-59. [CrossRef] [Medline]
  55. Rodda S, Lubman DI. Characteristics of gamblers using a national online counselling service for problem gambling. J Gambl Stud 2014 Jun;30(2):277-289. [CrossRef] [Medline]
  56. Tausczik YR, Pennebaker JW. The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. Journal of Language and Social Psychology 2009 Dec 08;29(1):24-54. [CrossRef]
  57. Fast LA, Funder DC. Gender differences in the correlates of self-referent word use: authority, entitlement, and depressive symptoms. J Pers 2010 Feb;78(1):313-338. [CrossRef] [Medline]
  58. Pyszczynski T, Greenberg J. Toward an Integration of Cognitive and Motivational Perspectives on Social Inference: a Biased Hypothesis-testing Model. Advances in experimental social psychology: Academic Press; 1987:297-340.
  59. Weintraub W. Verbal Behavior: Adaptation and Psychopathology. New York, USA: Springer Publishing Company; 1981.
  60. Lyons M, Aksayli ND, Brewer G. Mental distress and language use: Linguistic analysis of discussion forum posts. Computers in Human Behavior 2018 Oct;87:207-211. [CrossRef]
  61. Tackman AM, Sbarra DA, Carey AL, Donnellan MB, Horn AB, Holtzman NS, et al. Depression, negative emotionality, and self-referential language: A multi-lab, multi-measure, and multi-language-task research synthesis. J Pers Soc Psychol 2019 May;116(5):817-834. [CrossRef] [Medline]
  62. Stirman SW, Pennebaker JW. Word use in the poetry of suicidal and nonsuicidal poets. Psychosom Med 2001;63(4):517-522. [Medline]
  63. Rude S, Gortner E, Pennebaker J. Language use of depressed and depression-vulnerable college students. Cognition & Emotion 2004 Dec;18(8):1121-1133. [CrossRef]
  64. Molendijk ML, Bamelis L, van Emmerik AA, Arntz A, Haringsma R, Spinhoven P. Word use of outpatients with a personality disorder and concurrent or previous major depressive disorder. Behaviour Research and Therapy 2010 Jan;48(1):44-51. [CrossRef]
  65. Brockmeyer T, Zimmermann J, Kulessa D, Hautzinger M, Bents H, Friederich H, et al. Me, myself, and I: self-referent word use as an indicator of self-focused attention in relation to depression and anxiety. Front. Psychol 2015 Oct 09;6. [CrossRef]
  66. Zimmermann J, Brockmeyer T, Hunn M, Schauenburg H, Wolf M. First-person Pronoun Use in Spoken Language as a Predictor of Future Depressive Symptoms: Preliminary Evidence from a Clinical Sample of Depressed Patients. Clin. Psychol. Psychother 2016 Jan 27;24(2):384-391. [CrossRef]
  67. Guntuku SC, Yaden DB, Kern ML, Ungar LH, Eichstaedt JC. Detecting depression and mental illness on social media: an integrative review. Current Opinion in Behavioral Sciences 2017 Dec;18:43-49. [CrossRef]
  68. Park A, Conway M, Chen AT. Examining Thematic Similarity, Difference, and Membership in Three Online Mental Health Communities from Reddit: A Text Mining and Visualization Approach. Comput Human Behav 2018 Jan;78:98-112. [CrossRef] [Medline]
  69. De Choudhury M, Gamon M, Counts S, Horvitz E. Predicting Depression via Social Media. In: Proceedings of the Seventh International AAAI Conference on WeblogsSocial Media. 2013 Presented at: AAAI'13; April 7-9, 2013; Boston, MA, USA p. 128-137.
  70. Schwartz HA, Eichstaedt J, Kern ML, Park G, Sap M, Stillwell D. Towards Assessing Changes in Degree of Depression Through Facebook. In: Proceedings of the Workshop on Computational Linguistics and Clinical Psychology. 2014 Presented at: CLCP'14; May 2-4, 2014; Baltimore, MD, USA p. 118-125. [CrossRef]
  71. Babu S. Predicting Depression From Social Media Updates (Honors Thesis). University of California, Irvine. 2018.   URL: [accessed 2020-08-03]
  72. Davcheva E. Text mining mental health forums - Learning from user experiences. In: Bednar PM, Frank U, Kautz K. editors. Proceedings of the Twenty-sixth European Conference on Information Systems: Beyond Digitization - Facets of Socio-Technical Change. 2018 Presented at: Twenty-sixth European Conference on Information Systems: Beyond Digitization - Facets of Socio-Technical Change; 2018; Portsmouth, UK p. 91-101.
  73. Al-Mosaiwi M, Johnstone T. In an Absolute State: Elevated Use of Absolutist Words Is a Marker Specific to Anxiety, Depression, and Suicidal Ideation. Clinical Psychological Science 2018 Jan 05;6(4):529-542. [CrossRef]
  74. Fekete S. The Internet - A New Source of Data on Suicide, Depression and Anxiety: A Preliminary Study. Archives of Suicide Research 2002 Nov;6(4):351-361. [CrossRef]
  75. Owen JE, Klapow JC, Roth DL, Shuster JL, Bellis J, Meredith R, et al. Randomized pilot of a self-guided internet coping group for women with early-stage breast cancer. Ann Behav Med 2005 Aug;30(1):54-64. [CrossRef] [Medline]
  76. Seabrook EM, Kern ML, Fulcher BD, Rickard NS. Predicting Depression From Language-Based Emotion Dynamics: Longitudinal Analysis of Facebook and Twitter Status Updates. J Med Internet Res 2018 May 08;20(5):e168 [FREE Full text] [CrossRef] [Medline]
  77. Morales MR. Multimodal depression detection: An investigation of features and fusion techniques for automated systems (PhD thesis). New York NY, USA: City University of New York; 2018.
  78. Havigerová JM, Haviger J, Kučera D, Hoffmannová P. Text-Based Detection of the Risk of Depression. Front. Psychol 2019 Mar 18;10(513). [CrossRef]
  79. Banham JA, Schweitzer RD. Comparative exploration of narrative processes for better and poorer outcomes for depression. Couns Psychother Res 2015;15(3):228-238. [CrossRef]
  80. Bento T, Ribeiro AP, Salgado J, Mendes I, Gonçalves MM. The Narrative Model of Therapeutic Change: An Exploratory Study Tracking Innovative Moments and Protonarratives Using State Space Grids. Journal of Constructivist Psychology 2013 Dec 13;27(1):41-58. [CrossRef]
  81. Reyes L, Aristegui R, Krause M, Strasser K, Tomicic A, Valdés N, et al. Language and therapeutic change: a speech acts analysis. Psychother Res 2008 May;18(3):355-362. [CrossRef] [Medline]
  82. Paxling B, Lundgren S, Norman A, Almlöv J, Carlbring P, Cuijpers P, et al. Therapist behaviours in internet-delivered cognitive behaviour therapy: analyses of e-mail correspondence in the treatment of generalized anxiety disorder. Behav Cogn Psychother 2013 May;41(3):280-289. [CrossRef] [Medline]
  83. Arntz A, Hawke LD, Bamelis L, Spinhoven P, Molendijk ML. Changes in natural language use as an indicator of psychotherapeutic change in personality disorders. Behaviour Research and Therapy 2012 Mar;50(3):191-202. [CrossRef]
  84. van der Zanden R, Curie K, van Londen M, Kramer J, Steen G, Cuijpers P. Web-based depression treatment: associations of clients' word use with adherence and outcome. J Affect Disord 2014 May;160:10-13 [FREE Full text] [CrossRef] [Medline]
  85. Dirkse D, Hadjistavropoulos HD, Hesser H, Barak A. Linguistic analysis of communication in therapist-assisted internet-delivered cognitive behavior therapy for generalized anxiety disorder. Cogn Behav Ther 2015 Sep;44(1):21-32. [CrossRef] [Medline]
  86. Greenberg L, Watson J. Experiential therapy of depression: Differential effects of client-centered relationship conditions and process experiential interventions. Psychotherapy Res 1998;8(2):210-224. [CrossRef]
  87. Huston J, Meier S, Faith M, Reynolds A. Exploratory study of automated linguistic analysis for progress monitoring and outcome assessment. Couns Psychother Res 2019 Apr 16;19(3):321-328. [CrossRef]
  88. Althoff T, Clark K, Leskovec J. Large-scale Analysis of Counseling Conversations: An Application of Natural Language Processing to Mental Health. Trans Assoc Comput Linguist 2016;4:463-476 [FREE Full text] [Medline]
  89. Tay D. A Computerized Text and Cluster Analysis Approach to Psychotherapy Talk. Language and Psychoanalysis 2020 Mar 07;9(1):4-25. [CrossRef]
  90. Chen AT, Swaminathan A, Kearns WR, Alberts NM, Law EF, Palermo TM. Understanding User Experience: Exploring Participants’ Messages With a Web-Based Behavioral Health Intervention for Adolescents With Chronic Pain. J Med Internet Res 2019 Apr 15;21(4):e11756. [CrossRef]
  91. Calvo RA, Milne DN, Hussain MS, Christensen H. Natural language processing in mental health applications using non-clinical texts. Nat Lang Eng 2017 Jan 30;23(5):649-685. [CrossRef]
  92. Ruiz V, Shi L, Guerra J, Quan W, Ryan N, Biernesser C, et al. Predicting users' suicide risk levels from their reddit posts on multiple forums. In: Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology.: ACL; 2019 Presented at: Sixth Workshop on Computational Linguistics and Clinical Psychology; 2019; Minneapolis MN, USA p. 162-166. [CrossRef]
  93. Calvo RA, D'Mello S. Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications. IEEE Trans. Affective Comput 2010 Jan;1(1):18-37. [CrossRef]
  94. Losada DE, Gamallo P. Evaluating and improving lexical resources for detecting signs of depression in text. Lang Resources & Evaluation 2020 Jun;54(1):1-24. [CrossRef]
  95. Jones LS, Anderson E, Loades M, Barnes R, Crawley E. Can linguistic analysis be used to identify whether adolescents with a chronic illness are depressed? Clin Psychol Psychother 2020 Mar;27(2):179-192 [FREE Full text] [CrossRef] [Medline]
  96. Kim BYB, Sharafoddini A, Tran N, Wen EY, Lee J. Consumer Mobile Apps for Potential Drug-Drug Interaction Check: Systematic Review and Content Analysis Using the Mobile App Rating Scale (MARS). JMIR Mhealth Uhealth 2018 Mar 28;6(3):e74 [FREE Full text] [CrossRef] [Medline]

LIWC: Linguistic Inquiry and Word Count
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-analyses

Edited by G Strudwick; submitted 19.04.20; peer-reviewed by D Gratzer, M Berg, N Tran; comments to author 02.06.20; revised version received 18.07.20; accepted 15.01.21; published 24.02.21


©Anne Dwyer, Abílio de Almeida Neto, Dominique Estival, Weicong Li, Christa Lam-Cassettari, Mark Antoniou. Originally published in JMIR Mental Health (, 24.02.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.