Published on in Vol 8, No 12 (2021): December

Preprints (earlier versions) of this paper are available at, first published .
Language, Speech, and Facial Expression Features for Artificial Intelligence–Based Detection of Cancer Survivors’ Depression: Scoping Meta-Review

Language, Speech, and Facial Expression Features for Artificial Intelligence–Based Detection of Cancer Survivors’ Depression: Scoping Meta-Review

Language, Speech, and Facial Expression Features for Artificial Intelligence–Based Detection of Cancer Survivors’ Depression: Scoping Meta-Review


1Faculty of Electrical Engineering and Computer Science, University of Maribor, Maribor, Slovenia

2Science Department, Symptoma, Vienna, Austria

3Department of Internal Medicine, Paracelsus Medical University, Salzburg, Austria

4Department of Psychology, Faculty of Arts, University of Maribor, Maribor, Slovenia

Corresponding Author:

Urška Smrke, MA

Faculty of Electrical Engineering and Computer Science

University of Maribor

Koroška cesta 46

Maribor, 2000


Phone: 386 31262861


Background: Cancer survivors often experience disorders from the depressive spectrum that remain largely unrecognized and overlooked. Even though screening for depression is recognized as essential, several barriers prevent its successful implementation. It is possible that better screening options can be developed. New possibilities have been opening up with advances in artificial intelligence and increasing knowledge on the connection of observable cues and psychological states.

Objective: The aim of this scoping meta-review was to identify observable features of depression that can be intercepted using artificial intelligence in order to provide a stepping stone toward better recognition of depression among cancer survivors.

Methods: We followed a methodological framework for scoping reviews. We searched SCOPUS and Web of Science for relevant papers on the topic, and data were extracted from the papers that met inclusion criteria. We used thematic analysis within 3 predefined categories of depression cues (ie, language, speech, and facial expression cues) to analyze the papers.

Results: The search yielded 1023 papers, of which 9 met the inclusion criteria. Analysis of their findings resulted in several well-supported cues of depression in language, speech, and facial expression domains, which provides a comprehensive list of observable features that are potentially suited to be intercepted by artificial intelligence for early detection of depression.

Conclusions: This review provides a synthesis of behavioral features of depression while translating this knowledge into the context of artificial intelligence–supported screening for depression in cancer survivors.

JMIR Ment Health 2021;8(12):e30439



While cancer incidence is increasing worldwide [1], so are 5-year survival rates, from 49% in the 1970s to 69% in 2017 [2,3]. After having cancer, individuals can be faced with a wide array of challenges, such as fatigue, pain, impaired cognitive functions, and fear of cancer recurrence [1]. Among these challenges, depressive spectrum and mood-related disorders are among the most common psychological conditions [4]; it is estimated that in the first 2 years after diagnosis, 12% to 20% of cancer survivors meet diagnostic criteria for major depression disorder [1,5,6]. This is even more pronounced in cancer survivors with other comorbid chronic diseases [7] and breast cancer survivors [8]. At any point during survivorship, a dysphoric mood, anxiety, appetite changes, insomnia, or irritability can present and last weeks or even months [9]; however, these conditions frequently remain underrecognized and overlooked in clinical practice [10] because the signs and symptoms of depression in patients with cancer are heterogeneous [11]. Many survivors are also more likely to report somatic complaints rather than an overtly depressed mood. Moreover, only a minority of posttreatment survivors report experiencing clinically significant psychological distress during the treatment phase of their disease [12]. Although cancer distress screening has been recognized as an important tool, health care systems struggle to implement these tools in practice [13]. Lack of staff, difficulty in differentiating between mental health distress and symptoms of the disease, time constraints, accessibility restrictions, and availability of services represent the major barriers to regular screening [14].

Patient-gathered health data and patient-reported outcomes, in particular, have become a valuable tool in understanding the symptomatology of cancer patients and survivors [15]. Patient-reported outcomes give voice to the patient’s perspective and can inform decision-making by providing insight into the quality of life that is complementary to clinician-rated adverse events [16]. Digital interventions that collect patient-reported outcomes have been recognized as feasible and acceptable by clinicians and patients alike [17]. However, self-reporting often involves reporting bias, which may result in erroneous judgments such as the reconstruction of memories and excessive reliance on cognitive heuristics [18]. Retrospective self-reports of negative mood states experienced in the past (eg, the most recent 2 weeks) tend to be exaggerated in a negative direction [19]. Reporting and interpretation biases are even more pronounced for people suffering from symptoms of depression [20,21]. Moreover, inattentive responding and social desirability may distort the quality of results even further [22,23]. Currently, big data and artificial intelligence offer new opportunities for the screening and prediction of mental health problems. Specifically, there is a growing interest in examining relationships between observable cues, such as language use, speech, and facial expressions, and the psychological characteristics of the communicator [24]. These observable cues are generated spontaneously, are less impacted by cognitive and other biases related to desirability and crassness, and can contribute to improving the objectivity of psychiatric assessments [25]. The main motivation of this scoping review was to identify the observable features of depressive symptoms that are expressed during conversation (often without awareness), and that can be intercepted using artificial intelligence to be developed in the project Patient-Centered Survivorship Care Plan After Cancer Treatments Based on Big Data and Artificial Intelligence Technologies (PERSIST [26]). To our knowledge, past reviews (eg, [25,27-29]) mainly focused on specific modalities (ie, text, language, vocal features, or facial expressions). None, however, has focused on integration and the analysis of the complementary role of these modalities when expressed concurrently during a conversation; therefore, in this scoping meta-review, we integrated information from systematic reviews and meta-analyses on specific modalities and observable features of depression that are capable of being exploited by artificial intelligence.


We followed the methodological framework outlined by Arksey and O’Malley [30] and Levac and colleagues [31]. The framework proposes 6 stages: (1) identifying the research questions; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting results; and (6) consultation exercises. To ensure the process of this scoping review was systematic, transparent, and complete, we followed PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews [32]) guidelines.

Identifying the Research Questions

We first established the purpose of the review and developed specific research questions to guide our scoping review (eg, search terms, eligibility criteria)—which (1) text-related cues, (2) speech-related cues, and (3) facial expressions offer a valid insight into individuals’ depression?

Identifying Relevant Studies

Two large and commonly used databases, SCOPUS and Web of Science, which complement each other well [33,34], were used to identify the relevant papers. After a preliminary search in both databases, which helped us to refine the search strategy and ensure that the databases provide adequate coverage of the research topic, we conducted the main search on March 16, 2021.

Our search strategy combined terms related to depression (depression, “major depressive disorder,” "depressive episode,” depressivity, depressed), cues (feature*, indicator*, marker*, sign, signs, signal, signals, cue*, symbol*, pattern*, style*, clue*, manifestation*, expression*), behavior (language, linguistic, speech, acoustic, “facial expression”), and reviews (review, meta-analysis, “state of the art,” state-of-the-art). It should be noted that the last group of keywords (ie, those pertaining to reviews) was added in the process of search strategy refinement, as the number of papers was otherwise too high and not feasible for a scoping review. As such, the present scoping review is a scoping meta-review, which uses high-level evidence to answer research questions and policy dilemmas [35]. Additionally, the search strategy included keywords related to different disorders that affect behavior and display of emotions (dementia, ”Parkinson's disease,” autism, schizophrenia, Alzheimer's, “neurological disorder,” stroke, “Huntington's disease,” paralysis, mutism, “multiple sclerosis,” “cerebral palsy,” “Down syndrome”). Since comorbidity of depression and these disorders could skew our conclusions, such papers were already excluded in the first step. These groups of keywords were later combined into a nested format using Boolean operators (AND, OR, NOT), and titles, abstracts, and keywords were searched. The exact search string for SCOPUS was “TITLE-ABS-KEY ( ( depression OR “major depressive disorder” OR “depressive episode” OR depressivity OR depressed ) AND ( sign OR signs OR signal OR signals OR cue* OR symbol* OR pattern* OR style* OR clue* OR manifestation* OR expression* OR feature* OR indicator* OR marker* ) AND ( speech OR “facial expression” OR language OR linguistic OR acoustic ) AND ( review OR meta-analysis OR “state of the art” OR state-of-the-art ) AND NOT ( dementia OR “Parkinson's disease” OR autism OR schizophrenia OR “Alzheimer's disease” OR “neurological disorder*” OR stroke OR “Huntington's disease” OR paralysis OR mutism OR “multiple sclerosis” OR “cerebral palsy” OR “Down syndrome” ) ).” In order to identify potential additional papers, different combinations of search terms were used in Google Scholar, because this database can lead to the identification of additional unique entries [36].

The inclusion criteria were derived from the research questions guiding this scoping review and were set a priori. In order to be considered for inclusion in the scoping review, studies had to (1) be available in English and (2) published (ie, preprints and other unpublished papers were not considered). Studies were then excluded if (1) they were not based on empirical primary studies (ie, no quantitative evidence, obtained by collecting data on human participants), (2) they included human participants who suffered from other disorders that affect behavior and display of emotions, (3) they did not focus on adults, and (4) they did not offer evidence on the observable features (ie, text, speech, or facial expression) of depression. Since we aimed to provide a complete overview of all the research activity related to our research questions, we did not treat the methodological quality of reviews and primary studies as a reason for exclusion [30,37].

Study Selection

All citations identified in the electronic databases were exported to Excel spreadsheets (Microsoft Inc). Database searches yielded a total of 1203 papers, and 1 additional paper was identified through other sources (Figure 1). After duplicates were removed, 1023 titles and abstracts were screened in 2 stages. In the first stage, the authors (IM, NP, and US) individually excluded highly irrelevant papers. In the second stage, 2 reviewers (NP and US) independently reviewed the titles and abstracts of the remaining citations and settled disagreements through discussion. After this step, 18 papers underwent a thorough full-text review independently by 2 authors. Disagreements in this phase were settled through discussion and adjudication by a third reviewer (IM). In the end, 9 papers fulfilled the predetermined criteria and were included in the scoping review.

Figure 1. Flowchart of search and study selection.
View this figure

Charting the Data

We developed a spreadsheet form based on research questions to determine variables to be extracted from the reviewed papers. Two authors (NP and US) then extracted the following data from each paper: (1) authors, (2) year of publication, (3) type of paper, (4) number of primary studies included, (5) inclusion criteria, and (6) important results or information relevant to our research questions. During this process, results were categorized into the following categories: (1) text cues, (2) speech cues, and (3) facial expression cues. Data extraction and categorization of results were refined and updated in an iterative process as the papers were reviewed. The resulting chart was used for analysis.

Collating, Summarizing, and Reporting Results

We did not follow a well-structured meta-analytic approach in comparing and summarizing the results given the aim and nature of a scoping review. Since the aim of a scoping review includes mapping the existing findings on a topic and providing their descriptive overview [30,37], the results were analyzed by 2 authors (NP and US) using thematic analysis [38] within 3 predefined categories of cues of depression (text, speech, and facial expression cues). To maintain rigor in collating and summarizing results, this process was reviewed by 2 authors (IM and SL).

Consulting With Stakeholders

We developed a review protocol that ensured psychological (BM, NP, and US), medical/oncological (SL), technological (IM), and methodological (NP, US) expertise were included. Health-care professionals involved in PERSIST were consulted to develop relevant inclusion and exclusion criteria (eg, specific conditions that could impact the expression of depression).

Characteristics of Reviewed Studies

The final selection of papers in this review (Table 1) included 2 meta-analyses (22%), 2 systematic reviews (22%), 2 literature reviews (22%), 1 scoping review (11%), and 2 empirical studies (22%) that were included because their state-of-the-art met the criteria of this scoping review. Papers were published from 1997 to 2021, with most (7/9, 78%) published after 2013. Papers included from 8 to 60 primary studies on the topics in line with this review, and most focused on more than 1 category of depression cues.

Table 1. An overview of the characteristics of the papers reviewed.
ReferenceType of paperPrimary studies, naDepression cue categories addressed
Bylsma et al [39]Meta-analysis19Facial expression
Cummins et al [28]Literature review60Facial expression, speech, otherb
Dwyer et al [40]Scoping review22Language use
Edwards and Holtzman [41]Meta-analysis21Language use, speech
Kim et al [27]Systematic review15Language use
Pampouchidou et al [29]Systematic review43Facial expression, other
Scherer et al [42]Review within empirical studyc8Facial expression, speech, other
Shan et al [43]Review within empirical studyc13Facial expression, speech, other
Sobin and Sackeim [44]Literature review10Facial expression, speech, other

aThe number of primary studies that provided information on depression cues in 1 of 3 categories of interest.

bOther represents depression cues that are outside of the scope of this review.

cThese papers include a comprehensive review and an empirical study.

Features Related to Language Use

All 3 studies that focused on features related to language use [27,40,41] report that people with depression tend to focus on themselves, which is manifested in increased use of first-person singular pronouns, such as “I,” “my,” and "me." While the relationship between using such words and depression is relatively weak, findings of the relationship are ubiquitous and well replicated; based on their meta-analysis [41], Edwards and Holtzman suggest that it is equally present in different subsamples (eg, age, gender, clinical or nonclinical samples) and contexts (eg, written or spoken text, private or public language).

Depression can also be inferred from other linguistic features, particularly those that—either directly or indirectly—express depressive symptoms. Analyses of text-based communication have revealed that depressed individuals tend to use more negative-valence words (eg, words related to pain, expressions of sadness and anger, aggressive emotions, and rumination) and fewer positive emotion words than their nondepressed counterparts. For example, some of the specific language markers of depression include words such as ”hurt,” “tears,” ”alone,” “hate,” “sleep,” and “worry” [27,40]. Depression can also be inferred from explicit mentions of the treatment of depression (words such as “side effects” and “therapy”) [40].

Reviews [27,40] also suggest the existence of other linguistic features of depression. Patients with mental health issues, including those with depression, use more absolutist words (eg, “always,” “never”) [40] and tend to be focused on the past (eg, “learned,” “remember”) [27]. Furthermore, for example, depressed users on Twitter were found to generally have fewer words per Tweet, while other studies show that people with depression in general also exhibit lower complexity of language (ie, use fewer complex sentences) [27]. Reviews have highlighted that depressed individuals tend to avoid complex sentences containing adverbial clauses, which is a type of dependent clause that functions as an adverb (eg, “as soon as,” “even though”) [40].

Features Related to Speech

To some extent, features related to speech overlap with features related to language use and written communication. As mentioned above, increased usage of first-person singular pronouns has been identified in spoken language [41]. Moreover, similarly to written text, depressed individuals tend to engage less in verbal communication and speak in shorter phrases [42,43]. However, spoken language is generally an even richer source of information than written text— the information that it contains includes not only the message but also the manner in which the words are spoken.

Prosody, which describes the properties of intonation, stress, and rhythm (among others), is an additional source of information that has been well studied. Reviews and studies [28,42-44] indicate that the most promising prosodic feature of depression is related to speech rate. Depressed individuals speak at a slower rate than controls (particularly at the phoneme—the smallest unit of sound in speech—level) and exhibit longer pauses when answering questions, during the conversation, and at speech initiation [28,42-44]. The speech pause time is not only a good discriminator between depressed individuals and controls but also between depressed individuals and individuals with other mental health diagnoses, including bipolar disorder [44]. Although some authors report that there are conflicting results on the effect of depression on loudness and variation in loudness [28], others emphasize that especially reduced variation in loudness seems to be a common feature of depression [42,43]. Another variable that is often studied in the context of depression is fundamental frequency (also known as pitch or F0). All papers included in our scoping review concluded that depressed individuals exhibit lower pitch variability (also known as change of pitch) compared to controls, although some authors acknowledge that there was heterogeneity in primary studies [28,42-44]. The combination of lower variation in loudness as well as lower variation in pitch can result in monotonous speech among depressed individuals [42,43]. Similarly, depression has also been linked to a lack of linguistic stress (ie, relative emphasis given to a certain syllable or word), reduced intonation [28], repetitious pitch inflections and stress patterns [43], as well as poorer articulation (specifically, in terms of diphthong production, which refers to a sound made by combining 2 vowels) [44].

Fewer papers have studied other speech-related sources of information. Studies on source features (ie, features related to the source of voice production) have primarily focused on voice quality, with results showing that depression is linked to decreased voice quality (eg, aspiration, jitter, shimmer, and breathy phonation) and that the voices of depressed individuals are generally harsher [28,42]. Several studies have focused on comparisons of the glottal spectrum and flow parameters between depressed patients and controls. For example, depressed individuals exhibit higher energy in the upper frequency bands of the glottal spectrum [28]. Depression has also been linked to increased tension in the vocal tract and the vocal folds, with differences in parameters capturing glottal flow between moderate to severe depression and speakers without depression. Differences can specifically be found in their Normalized Amplitude Quotient, which is an amplitude-based measure of the glottal flow and glottal flow derivative, and their Quasi-Open-Quotient, which refers to an amplitude-based measure of the glottal flow pulse that offers important insight into the open period of the vocal folds [28,42].

Some studies also suggest that depressed individuals differ from their nondepressed counterparts in terms of the formant (information on acoustic resonances of the vocal tract) and spectral features (information on the speech spectrum). First, depressed speech is associated with decreased formant frequencies, particularly in the second formant location (phoneme/a1), although studies also show that the first 3 formant frequencies and bandwidths, grouped together, are significantly different between depressed patients and controls [28]. Second, while spectral analysis seems to have limited usefulness in classification systems, some studies have reported a relative shift in energy from lower to higher frequency bands, while others report a reduction in subband energy variability among depressed individuals [28].

Features Related to Facial Expression

Bylsma and colleagues [39] suggest that observing the behavioral indicators, there is reduced positive emotional reactivity in depressed patients and people with major depressive disorder compared to controls but found no differences in negative emotional response in comparison to controls. Their review also found “more pronounced blunting of positive emotional reactivity compared to negative emotional reactivity" [39]. Pampouchidou and colleagues [29] suggest that depression is associated with the variability and intensity of facial expressions (eg, reduced or decreased emotional facial expressivity [28,29,42]) with fewer animated facial expressions [43] and generally decreased facial mobility [28]. Links between depression and more frowns [29,42] and the occurrence of sad, negative, and neutral expressions [29] were also suggested.

Differences between people with depression and nondepressed individuals have been observed in eyebrow activity [29], through generally reduced eyebrow movements [28] and lower frequency and duration movements [44]. Differences can be observed in the region of the Veraguth fold (skin fold on the upper eyelid and between the eyebrows) and in the extended activity on the corrugator muscle (in the medial extremity of the eyebrow) [29].

Depression is associated with reduced saccadic eye movements [28,29], reduced horizontal pursuit [28], and increased visual fixation [28,29]. People with depression generally tend to engage less in mutual gazes [42] and limit eye contact [29]. Eye contact tends to be shorter in duration compared to normal controls [29,44] and people with schizophrenia [44] and occurs less frequently than in controls [44]. Depression tends to be associated with avoiding eye contact [43], low frequency and duration of glances [29], more gaze aversion, more downward gaze, and more nonspecific gaze [42]. Additionally, association with depression was observed in pupil dilation responses and bias, pupillary response, iris movement, and eyelid activity (eg, openings, blinking) [29].

Differences between people with depression and nondepressed individuals tend to be observed in mouth animation [29]. Specifically, they tend to present fewer mouth movements [42], more frequent lip presses, down-angled mouth corners, and reduced activity on the zygomaticus muscle (which moves the mouth angle in producing a smile) [29]. Depression is generally associated with less smiling [28] (ie, smiling less often [29,42,44] and shorter duration smiles [29,44]) and can be observed in smile intensity [29]. People with depression tend to exhibit more smile controls [42] and listening smiles (smiles presented when not speaking) [29].

Associations between depression and head pose (ie, orientation and movement) [29] were also suggested. People with depression tended to turn the head away [42] and were more likely to hold their head in a downward position [43] than nondepressed individuals. People with depression, in comparison to those with schizophrenia, tended to exhibit more large head movements and a higher occurrence and duration of small head movements [44].


Even though cancer survivors often find themselves experiencing disorders from the depressive spectrum, these conditions frequently remain underrecognized and overlooked [4,10]. Screening for psychological distress in this population is recognized as essential, but several barriers for its successful implementation persist [13,14]. For instance, the most used method of self-reporting of symptoms can result in significant bias in reporting experiences, which is even more pronounced in people with depression [18-23]. Therefore, a better way to detect psychological disorders in cancer survivors is needed. Big data and artificial intelligence, together with a growing body of knowledge on the connection between observable cues and psychological states of a person, offer new opportunities to better detect psychological disorders in cancer survivors. As Low and colleagues [25] suggest, spontaneously generated cues of psychological states and their recognition by artificial intelligence could result in improved screening for psychological disorders.

In this scoping review, we reviewed 9 meta-analyses, systematic and literature reviews, and similar papers to conduct a meta-review of observable cues of depression. The findings of our meta-review revealed several observable characteristics of depression in each of the 3 categories. In language use, people with depression tend to show increased use of first-person singular pronouns, use more negatively valenced words, and less positive emotion words than nondepressed individuals. Additionally, the use of absolutist words, the use of words focusing on the past, and low language complexity are often present. In speech, people with depression engage in less verbal communication and speak in shorter phrases; lower speech rate, longer speech pauses, and potentially lower variation in loudness are exhibited. Some authors [42,43] also suggest lower pitch variability and change rate in people with depression, resulting in generally monotonous speech. Reduced intonation, repetitious pitch inflections and stress patterns, poorer articulation, decreased voice quality, generally harsher voice, and higher vocal tension also tend to be present. In the category of facial expressions, people with depression in comparison to nondepressed individuals show reduced positive emotional reactivity and emotional facial expressivity. Generally, they produce fewer animated facial expressions and facial mobility and exhibit more frowns and more sad, negative, and neutral expressions. There is less movement in the eyebrow region, and eyes exhibit reduced saccadic eye movements, reduced horizontal pursuit, and increased visual fixation. People with depression tend to engage in less mutual gaze, avoid or limit their eye contact, show more gaze aversion, and show more downward and nonspecific gaze. Fewer mouth movements, more lip presses, and down-angled mouth corners also proved to be evident in people with depression. Additionally, decreased smiling can be observed, together with more smile controls and listening smiles. They also turn their head away more often and hold their head in a downward position.

The diagnosis of cancer and subsequent treatments can have a large impact on patients’ psychological well-being. Along with the physical remnants of treatment, cancer survivors often continue to grapple with anxiety and depression. Almost one-third of cancer patients suffer from a comorbid mental health condition [45]. Although professional support is available after diagnosis and during treatment, the symptoms are still often overlooked in patients and long-term survivors [46], partially due to significant overlap between symptoms of emotional distress and late effects of cancer or side effects of treatment, but mainly due to depression’s complexity and subjective nature. Namely, as symptoms of depression may be normal for some, the that are symptoms present may impose significant psychological strain on other patients [47]. The difficulty in diagnosing depression in cancer patients has led to the development of several diagnostic approaches [48]. However, since screening is mostly based on self-reports during 6-month or 1-year follow-up, symptoms can remain unnoticed for a long period of time or can even be completely overlooked. The technological advances and reliability of machine learning–supported feature extraction classification methods may allow new, less intrusive, and more reliable ways to detect symptoms of depression. As highlighted in this scoping review, completely unrelated spoken or even written interactions, as well as visual cues, may reveal early warning signs that should trigger further clinical assessment.

While the technological advances may be of great benefit in recognizing these cues and signs, it is particularly important to note that many existing studies tend to exploit only 1 modality, such as text analysis [49], though it may be combined with other aspects of users’ writing (eg, the time gap between 2 consecutive writings; [50]), for artificial intelligence–based recognition of depression and other mental health issues. These approaches can be informative and helpful in the early detection of depression but tend to perform suboptimally due to their relatively narrow focus [51]. Our review shows that depression is likely expressed through all 3 forms of communication simultaneously; therefore, the prevailing unimodal approaches to artificial intelligence recognition of depression (eg, [43]) may be inadequate. Trimodal approaches are especially important in improving the accuracy of predictive models and reducing the chance of false classification of individuals as those presenting or not presenting signs of depression. For example, when using unimodal speech analysis models, an error could occur in the instance of nonnative speakers who often use first-person singular pronouns, which could be recognized as a meaningful feature of depression, whereas the result might be different when the other 2 modes are also examined. In PERSIST [26], we intend to exploit findings of this review to deliver an explainable artificial intelligence capable of sensing, detecting, and interpreting the affective states spontaneously expressed through language, speech, and mimicry during interaction (ie, diary recordings).

Study Limitations

While this scoping meta-review offers a valuable synthesis of research on the observable cues of depression and translates this knowledge into the context of screening cancer survivors’ depression with artificial intelligence, it does not offer an all-encompassing picture of the current state of research on the behavioral features of depression. First, the papers included in our scoping review highlight that depression is also associated with cues, such as body movement (including gestures, fidgeting) and posture, that are outside of the scope of this meta-review. As these features could explain the variance in depression expressions above and beyond the features described in our review (and hence improve the accuracy of depression detection algorithms), we argue that attempts to synthesize research on these additional cues would be highly beneficial. Second, the findings of our meta-review represent the features of depression that are characteristic of the average person with depression. Future research should take into consideration potential moderators that determine whether features are present and to what extent. In other words, future research should explore which cues are relevant in which cases, to allow the development of algorithms that are robust to individuals’ characteristics. Third, because we focused only on English-language papers, based on studies that were largely conducted on Anglophone participants in industrialized countries, our conclusions may be culturally biased. Further research is thus needed to understand whether our findings can be generalized. Lastly, since we included reviews instead of original empirical papers, it is possible that we overlooked more recent papers related to this topic that have not yet been included in any reviews.

Clinical Implications

This review provides valuable theoretical background and ideas for technological implementation that could facilitate the development of improved artificial intelligence solutions to detecting cancer survivors’ depression. We argue that such solutions may benefit clinicians as well as cancer survivors. For clinicians, these solutions may be more cost-effective and efficient for recognizing distress in patients after cancer than existing methods. As such, artificial intelligence could replace some elements of current screening procedures and supplement others. Additionally, such solutions may be used for ecological momentary assessments (and over longer periods of time), which is not possible with traditional clinical assessment and self-report instruments. For cancer survivors, on the other hand, artificial intelligence may be an accessible and nonobtrusive way of monitoring their mental health that does not require any conscious effort.


Evidence from 9 reviews (based on more than 200 primary empirical studies) show that there is a robust association between depression and a wide array of specific observable cues. Such associations are an excellent theoretical underpinning for the development of artificial intelligence algorithms; therefore, it is time to move from the question of whether artificial intelligence can support the process of detecting cancer survivors’ depression to the question of how this can be done.


This work has received funding by the European Union Horizon 2020 Research and Innovation Programme (PERSIST, grant number 875406). The content of this paper does not reflect the official opinion of the European Union or any other institution. Responsibility for the information and views expressed herein lies entirely with the authors.

Conflicts of Interest

None declared.

  1. Martin F, Wright H, Moody L, Whiteman B, McGillion M, Clyne W, et al. Help to overcome problems effectively for cancer survivors: development and evaluation of a digital self-management program. J Med Internet Res 2020 May 19;22(5):e17824 [FREE Full text] [CrossRef] [Medline]
  2. Baseman J, Revere D, Baldwin L. A mobile breast cancer survivorship care app: pilot study. JMIR Cancer 2017 Sep 26;3(2):e14 [FREE Full text] [CrossRef] [Medline]
  3. Cancer facts and figures 2017. American Cancer Society. 2017.   URL: https:/​/www.​​content/​dam/​cancer-org/​research/​cancer-facts-and-statistics/​annual-cancer-facts-and-figures/​2017/​cancer-facts-and-figures-2017.​pdf [accessed 2017-09-18]
  4. Caruso R, Nanni MG, Riba M, Sabato S, Mitchell AJ, Croce E, et al. Depressive spectrum disorders in cancer: prevalence, risk factors and screening for depression: a critical review. Acta Oncol 2017 Feb;56(2):146-155. [CrossRef] [Medline]
  5. Mitchell AJ, Ferguson DW, Gill J, Paul J, Symonds P. Depression and anxiety in long-term cancer survivors compared with spouses and healthy controls: a systematic review and meta-analysis. Lancet Oncol 2013 Jul;14(8):721-732. [CrossRef] [Medline]
  6. Inhestern L, Beierlein V, Bultmann JC, Möller B, Romer G, Koch U, et al. Anxiety and depression in working-age cancer survivors: a register-based study. BMC Cancer 2017 May 19;17(1):347 [FREE Full text] [CrossRef] [Medline]
  7. Yan R, Xia J, Yang R, Lv B, Wu P, Chen W, et al. Association between anxiety, depression, and comorbid chronic diseases among cancer survivors. Psychooncology 2019 Jun;28(6):1269-1277. [CrossRef] [Medline]
  8. Doege D, Thong MSY, Koch-Gallenkamp L, Jansen L, Bertram H, Eberle A, et al. Age-specific prevalence and determinants of depression in long-term breast cancer survivors compared to female population controls. Cancer Med 2020 Nov;9(22):8713-8721 [FREE Full text] [CrossRef] [Medline]
  9. Mehta RD, Roth AJ. Psychiatric considerations in the oncology setting. CA Cancer J Clin 2015;65(4):300-314 [FREE Full text] [CrossRef] [Medline]
  10. Caruso R, GiuliaNanni M, Riba MB, Sabato S, Grassi L. Depressive spectrum disorders in cancer: diagnostic issues and intervention: a critical review. Curr Psychiatry Rep 2017 Jun;19(6):33 [FREE Full text] [CrossRef] [Medline]
  11. Godby RC, Dai C, Al-Obaidi M, Giri S, Young-Smith C, Kenzik K, et al. Depression among older adults with gastrointestinal malignancies. J Geriatr Oncol 2021 May;12(4):599-604. [CrossRef] [Medline]
  12. Philip EJ, Merluzzi TV, Zhang Z, Heitzmann CA. Depression and cancer survivorship: importance of coping self-efficacy in post-treatment survivors. Psychooncology 2013 May;22(5):987-994 [FREE Full text] [CrossRef] [Medline]
  13. Knies AK, Jutagir DR, Ercolano E, Pasacreta N, Lazenby M, McCorkle R. Barriers and facilitators to implementing the commission on cancer's distress screening program standard. Palliat Support Care 2019 Jun;17(3):253-261 [FREE Full text] [CrossRef] [Medline]
  14. Granek L, Nakash O, Ariad S, Shapira S, Ben-David M. Oncologists' identification of mental health distress in cancer patients: strategies and barriers. Eur J Cancer Care (Engl) 2018 May;27(3):e12835. [CrossRef] [Medline]
  15. Jim HSL, Hoogland AI, Brownstein NC, Barata A, Dicker AP, Knoop H, et al. Innovations in research and clinical care using patient-generated health data. CA Cancer J Clin 2020 May;70(3):182-199 [FREE Full text] [CrossRef] [Medline]
  16. Austin E, LeRouge C, Hartzler AL, Segal C, Lavallee DC. Capturing the patient voice: implementing patient-reported outcomes across the health system. Qual Life Res 2020 Feb;29(2):347-355 [FREE Full text] [CrossRef] [Medline]
  17. Wright AA, Raman N, Staples P, Schonholz S, Cronin A, Carlson K, et al. The HOPE pilot study: harnessing patient-reported outcomes and biometric data to enhance cancer care. JCO Clin Cancer Inform 2018 Dec;2:1-12 [FREE Full text] [CrossRef] [Medline]
  18. Nahum M, Van Vleet TM, Sohal VS, Mirzabekov JJ, Rao VR, Wallace DL, et al. Immediate mood scaler: tracking symptoms of depression and anxiety using a novel mobile mood scale. JMIR Mhealth Uhealth 2017 Apr 12;5(4):e44. [CrossRef] [Medline]
  19. Sato H, Kawahara J. Selective bias in retrospective self-reports of negative mood states. Anxiety Stress Coping 2011 Jul;24(4):359-367. [CrossRef] [Medline]
  20. Everaert J, Podina IR, Koster EHW. A comprehensive meta-analysis of interpretation biases in depression. Clin Psychol Rev 2017 Dec;58:33-48. [CrossRef] [Medline]
  21. Urban EJ, Charles ST, Levine LJ, Almeida DM. Depression history and memory bias for specific daily emotions. PLoS One 2018;13(9):e0203574 [FREE Full text] [CrossRef] [Medline]
  22. Maniaci MR, Rogge RD. Caring about carelessness: Participant inattention and its effects on research. J Res Pers 2014 Feb;48:61-83. [CrossRef]
  23. Robinson MA, Boies K. On the quest for quality self-report data: HEXACO and indicators of careless responding. Can J Behav Sci 2021 Jul;53(3):377-380. [CrossRef]
  24. Liang Y, Zheng X, Zeng DD. A survey on big data-driven digital phenotyping of mental health. Inf Fusion 2019 Dec;52:290-307. [CrossRef]
  25. Low DM, Bentley KH, Ghosh SS. Automated assessment of psychiatric disorders using speech: a systematic review. Laryngoscope Investig Otolaryngol 2020 Feb;5(1):96-116 [FREE Full text] [CrossRef] [Medline]
  26. Project PERSIST. PERSIST.   URL: [accessed 2021-04-30]
  27. Kim J, Uddin ZA, Lee Y, Nasri F, Gill H, Subramanieapillai M, et al. A systematic review of the validity of screening depression through Facebook, Twitter, Instagram, and Snapchat. J Affect Disord 2021 May 01;286:360-369. [CrossRef] [Medline]
  28. Cummins N, Scherer S, Krajewski J, Schnieder S, Epps J, Quatieri TF. A review of depression and suicide risk assessment using speech analysis. Speech Commun 2015 Jul;71:10-49. [CrossRef]
  29. Pampouchidou A, Simos PG, Marias K, Meriaudeau F, Yang F, Pediaditis M, et al. Automatic assessment of depression based on visual cues: a systematic review. IEEE Trans Affective Comput 2019 Oct 1;10(4):445-470. [CrossRef]
  30. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol Theory Pract 2005 Feb;8(1):19-32. [CrossRef]
  31. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010;5:69 [FREE Full text] [CrossRef] [Medline]
  32. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, Tunçalp, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018 Oct 02;169(7):467-473. [CrossRef] [Medline]
  33. Burnham JF. Scopus database: a review. Biomed Digit Libr 2006 Mar 08;3:1 [FREE Full text] [CrossRef] [Medline]
  34. Zhu J, Liu W. A tale of two databases: the use of Web of Science and Scopus in academic papers. Scientometrics 2020 Feb 22;123(1):321-335. [CrossRef]
  35. Sarrami-Foroushani P, Travaglia J, Debono D, Clay-Williams R, Braithwaite J. Scoping meta-review: introducing a new methodology. Clin Transl Sci 2015 Feb;8(1):77-81 [FREE Full text] [CrossRef] [Medline]
  36. Martín-Martín A, Orduna-Malea E, Thelwall M, Delgado López-Cózar E. Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories. J Informetr 2018 Nov;12(4):1160-1177. [CrossRef]
  37. Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods 2014 Dec;5(4):371-385 [FREE Full text] [CrossRef] [Medline]
  38. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
  39. Bylsma LM, Morris BH, Rottenberg J. A meta-analysis of emotional reactivity in major depressive disorder. Clin Psychol Rev 2008 Apr;28(4):676-691. [CrossRef] [Medline]
  40. Dwyer A, de Almeida Neto A, Estival D, Li W, Lam-Cassettari C, Antoniou M. Suitability of text-based communications for the delivery of psychological therapeutic services to rural and remote communities: scoping review. JMIR Ment Health 2021 Feb 24;8(2):e19478 [FREE Full text] [CrossRef] [Medline]
  41. Edwards T, Holtzman NS. A meta-analysis of correlations between depression and first person singular pronoun use. J Res Pers 2017 Jun;68:63-68 [FREE Full text] [CrossRef]
  42. Scherer S, Stratou G, Lucas G, Mahmoud M, Boberg J, Gratch J, et al. Automatic audiovisual behavior descriptors for psychological disorder analysis. Image Vis Comput 2014 Oct;32(10):648-658. [CrossRef]
  43. Guohou S, Lina Z, Dongsong Z. What reveals about depression level? The role of multimodal features at the level of interview questions. Inf Manag 2020 Nov;57(7):103349. [CrossRef]
  44. Sobin C, Sackeim HA. Psychomotor symptoms of depression. Am J Psychiatry 1997 Jan;154(1):4-17. [CrossRef] [Medline]
  45. Singer S. Psychosocial impact of cancer. Recent Results Cancer Res 2018;210:1-11. [CrossRef] [Medline]
  46. Kim JH, Paik H, Jung YJ, Kim D, Jo HJ, Lee S, et al. A prospective longitudinal study about change of sleep, anxiety, depression, and quality of life in each step of breast cancer patients. Oncology 2019;97(4):245-253. [CrossRef] [Medline]
  47. Smith HR. Depression in cancer patients: pathogenesis, implications and treatment (review). Oncol Lett 2015 Apr;9(4):1509-1514 [FREE Full text] [CrossRef] [Medline]
  48. Weinberger MI, Bruce ML, Roth AJ, Breitbart W, Nelson CJ. Depression and barriers to mental health care in older cancer patients. Int J Geriatr Psychiatry 2011 Jan;26(1):21-26 [FREE Full text] [CrossRef] [Medline]
  49. Aladağ AE, Muderrisoglu S, Akbas NB, Zahmacioglu O, Bingol HO. Detecting suicidal ideation on forums: proof-of-concept study. J Med Internet Res 2018 Jun 21;20(6):e215 [FREE Full text] [CrossRef] [Medline]
  50. Cacheda F, Fernandez D, Novoa FJ, Carneiro V. Early detection of depression: social network analysis and random forest techniques. J Med Internet Res 2019 Jun 10;21(6):e12554 [FREE Full text] [CrossRef] [Medline]
  51. Wongkoblap A, Vadillo MA, Curcin V. Researching mental health disorders in the era of social media: systematic review. J Med Internet Res 2017 Dec 29;19(6):e228 [FREE Full text] [CrossRef] [Medline]

PERSIST: Patient-Centered Survivorship Care Plan After Cancer Treatments Based on Big Data and Artificial Intelligence Technologies
PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension for Scoping Reviews

Edited by J Torous; submitted 14.05.21; peer-reviewed by J Erickson, E Toki; comments to author 10.08.21; revised version received 25.08.21; accepted 06.09.21; published 06.12.21


©Urška Smrke, Izidor Mlakar, Simon Lin, Bojan Musil, Nejc Plohl. Originally published in JMIR Mental Health (, 06.12.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.