This paper is in the following e-collection/theme issue:

Mobile Health in Psychiatry (403) Depression and Mood Disorders; Suicide Prevention (1290) Mobile Health (mhealth) (2777) Diagnostic Tools in Mental Health (224)

Published on in Vol 9, No 10 (2022): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/33871, first published .
An App-Based Digit Symbol Substitution Test for Assessment of Cognitive Deficits in Adults With Major Depressive Disorder: Evaluation Study

An App-Based Digit Symbol Substitution Test for Assessment of Cognitive Deficits in Adults With Major Depressive Disorder: Evaluation Study

An App-Based Digit Symbol Substitution Test for Assessment of Cognitive Deficits in Adults With Major Depressive Disorder: Evaluation Study

Authors of this article:

Roger S McIntyre1, 2, 3, 4 Author Orcid Image ;   Orly Lipsitz1, 4 Author Orcid Image ;   Nelson B Rodrigues1, 4 Author Orcid Image ;   Mehala Subramaniapillai1, 4 Author Orcid Image ;   Flora Nasri4 Author Orcid Image ;   Yena Lee1 Author Orcid Image ;   Ben Fehnert5, 6 Author Orcid Image ;   James King6 Author Orcid Image ;   Lambros Chrones7 Author Orcid Image ;   Kevin Kratiuk4 Author Orcid Image ;   Sharif Uddin7 Author Orcid Image ;   Joshua D Rosenblat1, 2, 3, 4 Author Orcid Image ;   Rodrigo B Mansur1, 3 Author Orcid Image ;   Maggie McCue7 Author Orcid Image
Article Authors Cited by (3) Tweetations (12) Metrics

    Original Paper

    1University Health Network, Mood Disorders Psychopharmacology Unit, Toronto, ON, Canada

    2Brain and Cognition Discovery Foundation, Toronto, ON, Canada

    3Department of Psychiatry, University of Toronto, Toronto, ON, Canada

    4Canadian Rapid Treatment Center of Excellence, Mississauga, ON, Canada

    5Ctrl Group, London, United Kingdom

    6Cognition Kit, Cambridge, United Kingdom

    7Takeda Pharmaceuticals U.S.A., Inc., Lexington, MA, United States

    Corresponding Author:

    Roger S McIntyre, MD

    Brain and Cognition Discovery Foundation

    73 Mathersfield Drive

    Toronto, ON, M4W 3W4

    Canada

    Phone: 1 416 603 5279

    Email: roger.mcintyre@bcdf.org


    Background: Cognitive dysfunction is an impairing core symptom of depression. Among adults with major depressive disorder (MDD) treated with antidepressants, residual cognitive symptoms interfere with patient-reported outcomes. The foregoing characterization of cognitive symptoms provides the rationale for screening and assessing the severity of cognitive symptoms at point of care. However, clinical neurocognitive assessments are time-consuming and difficult, and they require specialist expertise to interpret them. A smartphone-delivered neurocognitive test may offer an effective and accessible tool that can be readily implemented into a measurement-based care framework.

    Objective: We aimed to evaluate the use of a smartphone-delivered app-based version of the established Cognition Kit Digit Symbol Substitution Test (DSST) neurocognitive assessment compared to a traditional paper-and-pencil version.

    Methods: Convergent validity and test-retest reliability of the 2 versions were evaluated. Patient satisfaction with the app was also assessed.

    Results: Assessments made using the app-based Cognition Kit DSST were highly correlated with the standard paper-and-pencil version of the test, both at the baseline visit (r=0.69, df=27; P<.001) and at the end-of-study visit (r=0.82, df=27; P<.001), and they were positively evaluated by 30 patients as being user-friendly, easy to navigate, and preferable over the paper-and-pencil version of the DSST. However, although the app-based Cognition Kit DSST was validated in patients with MDD, it still needs to be evaluated in healthy controls.

    Conclusions: App-based DSST may facilitate a more personalized, convenient, and cost-effective method of cognitive assessment, helping to guide measurement-based care and psychotherapeutic and pharmacologic treatment options for patients with MDD.

    Trial Registration: ClinicalTrials.gov NCT03999567; https://tinyurl.com/2p8pnyv7

    JMIR Ment Health 2022;9(10):e33871

    doi:10.2196/33871

    Keywords

    depression; DSST; Digit Symbol Substitution Test; smartphone; technology; measurement-based care; cognition 


    Major depressive disorder (MDD) is an episodic illness characterized by a persistently depressed mood or loss of interest in activities that causes significant impairment in daily life [1,2]. Cognitive impairment is a core domain disturbance in MDD, with disturbances in cognitive function being listed among the criteria that define a major depressive episode [2].

    Approximately 90%-95% of adults with MDD experience cognitive symptoms during a depressive episode [3], including impairments in executive function, attention, learning and memory, and processing speed [4,5]. The foregoing cognitive deficits have been demonstrated to mediate social, functional, and work-related disability associated with MDD and have a long-term impact that often persists when depressive symptoms have abated [5,6]. Indeed, during periods of remission, approximately half of all depressed patients will continue to experience cognitive deficits, which presage patient-reported outcomes (PROs) in adults with MDD [3,7].

    Addressing cognitive symptoms can be a clinical priority when treating some patients with MDD, particularly those whose MDD appears to be significantly impairing their daily functioning and treatment has failed to resolve these symptoms [8]. Impaired cognitive functioning is also progressive in some patients with MDD [5,9], and there is evidence to suggest that cognitive function deteriorates further with each major depressive episode [4]. Accordingly, there is a need to address cognitive dysfunction in patients with MDD, as it substantially interferes with daily psychosocial functioning and can lead to adverse long-term outcomes. For example, patients with MDD may have poorer workplace performance, which is a result of impaired cognitive functioning [5,6,10]. Cognitive deficits, therefore, influence PROs and reduce individuals’ quality of life and functioning [11].

    The critical and pervasive role that cognitive deficits play in the functioning PROs in patients with MDD invites the need for accessible, convenient, and effective measurement-based care (MBC) assessment tools that offer more than a simple evaluation of the presence or absence of symptoms. Tools that collect valuable information about symptoms and potential changes that could impact general well-being may provide greater insight into a patient’s condition, supporting individualized care that can improve overall treatment outcomes. Guiding principles of MBC are more likely to be implemented at the point of care as tools that are brief and patient administered, provide actionable information, and are preferably digitalized in keeping with busy office practices [12].

    Many commonly used comprehensive neurocognitive tests are effective MBC assessment tools, but they are lengthy and cumbersome to administer and complete and often require professional interpretation, limiting their implementation outside of a clinical environment [13,14]. Web-based tools integrate both subjective and objective measures of cognition, are typically free of charge for the patient, digitalized, implemented remotely (ie, using a tablet), and require less time to complete (ie, approximately 10 minutes) [12,15]. However, web-based tools may not be accessible for all patients if a paid software subscription is required or the test has not been optimized for smartphone delivery and must be delivered using a computer or tablet with a large screen [16]. Recent progress in smartphone technology and mobile apps presents a unique prospect in this scenario. Several health-related smartphone apps have already been implemented in other chronic diseases (eg, diabetes mellitus), where it has been shown to be acceptable to end users, provide actionable data, and facilitate health outcomes [17]. The ubiquity of smartphones provides an opportunity to screen and measure the presence of cognitive functions in patients with MDD via smartphone-based neurocognitive assessments. Similar to web-based tools, smartphone-based neurocognitive assessments can also be free of charge for the patient, easy to administer, and may require even less time to complete.

    The Digit Symbol Substitution Test (DSST) is an MBC assessment tool that provides multidomain assessment of neurocognitive functions and has been extensively validated in psychiatric, medical, and general populations [16]. This study was designed to evaluate an app-based Cognition Kit DSST as a screening and assessment tool for cognition in MDD that can be delivered via a smartphone.


    Ethics Approval

    This study was approved by the institutional review board of Advarra (Pro00037042) prior to initiation of the study. All participants provided written informed consent prior to enrollment.

    Study Design and Participants

    Patients enrolled at the Canadian Rapid Treatment Center of Excellence were approached and asked to participate. Adult patients (aged 18-65 years) experiencing a moderate-to-severe major depressive episode (based on the Montgomery-Åsberg Depression Rating Scale [MADRS] with a total score ≥20) in the context of MDD were enrolled in this prospective, longitudinal validation study (Clinical Trial Identifier NCT03999567). Patients were not under treatment during participation and could not have had a change in their medication up to 2 weeks prior to participation. Participants were recruited by 2 research coordinators who were not responsible for patient care at the center.

    The diagnosis of MDD was ascertained clinically and confirmed using the Mini-International Neuropsychiatric Interview (MINI). Patients were excluded if they had a comorbid psychiatric condition primary to MDD, used benzodiazepines within 12 hours of cognitive assessment, consumed alcohol within 8 hours of cognitive assessment, or used marijuana in an inconsistent or abusive manner. Patients were also excluded if they had current alcohol or substance use disorder confirmed by the MINI; physical, cognitive, or language impairments; diagnosed reading disability; dyslexia; or a clinically significant learning disorder. Use of electroconvulsive therapy in the past 6 months or a history of moderate-to-severe head trauma, neurological disorders, or unstable medical conditions that affect the central nervous system were criteria for exclusion from the study. If patients had a previous history or were currently experiencing symptoms of mania or hypomania or had a history of seizures and epilepsy, they were not eligible to participate. Patients were asked not to change medications 1 week prior to the baseline study visit and in the week between the baseline and end-of-study visits.

    Procedure

    Participants completed both the app-based Cognition Kit DSST (Cognition Kit Ltd; Figure 1) on an Apple iPhone with finger-screen interaction and the paper-and-pencil DSST at 2 study visits: first at the baseline visit and then 1 week later at the end-of-study visit. The order of the app-based and paper-and-pencil assessments was counterbalanced between the 2 visits and between study participants. The DSST was based on the Wechsler Adult Intelligence Scale–Revised version [16]. Data from each app-based Cognition Kit DSST was stored in a system compliant with the Health Insurance Portability and Accountability Act to ensure data privacy.

    Overall, depressive symptom severity (based on MADRS), consummatory anhedonia (based on Snaith-Hamilton Pleasure Scale [SHAPS]) [18], and anxiety (based on Hamilton Anxiety Rating Scale [HAM-A]) [19] were assessed at each visit. At the end-of-study visit, patients completed a 10-item app satisfaction survey. Each item of this survey was scored on a 5-point Likert scale, where 1 indicated “strongly disagree” and 5 indicated “strongly agree.”

    Convergent validity of the app-based Cognition Kit DSST versus the paper-and-pencil DSST and the test-retest reliability of each instrument were assessed by calculating the Pearson correlation coefficient (partial correlation) using SPSS (version 23.0; IBM Corp), controlling for age.

    ‎
    Figure 1. App-based Cognition Kit Digit Symbol Substitution Test (DSST) screenshots.
    View this figure

    Patient Demographic and Clinical Characteristics

    Prescreening was performed for 47 potential patients, of which a total of 30 patients were eligible for inclusion (Multimedia Appendix 1). All patients completed both study visits; 17 (57%) were female, and the mean age was 42 (SD 13) years. Approximately two-thirds (19/30, 63%) of patients had completed a college or university education. At baseline, 57% (17/30) of patients were currently taking an antidepressant medication (Table 1). Patients had a mean MADRS total score of 29.9 (SD 4.9) at baseline. Mean MADRS, HAM-A, and SHAPS scores did not differ greatly between the baseline and end-of-study visits (Figure 2).

    Table 1. Patient demographic and clinical characteristics at baseline.
    Demographics or clinical characteristicsAll study participants (N=30)
    Age (years), mean (SD)42 (13)
    Sex, n (%)

    		Female17 (57)

    		Male13 (43)
    Race, n (%)

    		White25 (83)
    Asian2 (6.7)
    Multiracial2 (6.7)
    Latin American1 (3.3)
    Highest level of education completed,n(%)

    		High school7 (23)

    		College or university19 (63)

    		Graduate school4 (13)
    Antidepressant medication

    		Taking antidepressant medication, n (%)19 (63)

    		Current antidepressants, mean (SD)1.0 (0.98)

    		Lifetime antidepressantsa, mean (SD)7 (7)

    aNumber of antidepressants used throughout the patient’s lifetime.

    ‎
    Figure 2. Clinical measures at baseline and end-of-study (N=30). HAM-A: Hamilton Anxiety Rating Scale; MADRS: Montgomery-Åsberg Depression Rating Scale; SHAPS: Snaith-Hamilton Pleasure Scale.
    View this figure

    Convergent Validity

    Patients had mean DSST scores of 29 (SD 7) and 50 (SD 13) at baseline for the app-based and paper-and-pencil DSST, respectively. At the end of the study, the mean DSST scores were 31 (SD 9) and 55 (SD 14), respectively, for the app-based and paper-and-pencil DSST. The app-based Cognition Kit DSST and the paper-and-pencil DSST were positively correlated at both the baseline visit (r=0.64, df=27; Figure 3A) and the end-of-study visit (r=0.80, df=27; Figure 3B). The corresponding partial correlations after adjusting for age were r=0.69 (df=27; P<.001) and r=0.82 (df=27; P<.001) at baseline and end-of-study visits, respectively. Differences in the Cognition Kit DSST score from the baseline visit to the end-of-study visit versus the corresponding differences in the paper-and-pencil DSST score trended toward positive correlation (r=0.24, df=27; P=.21).

    ‎
    Figure 3. Correlation between app-based DSST and paper-and-pencil DSST scores at (A) baseline visit and (B) end-of-study visit (N=30). The correlation is not adjusted for age. DSST: Digit Symbol Substitution Test.
    View this figure

    Test-Retest Reliability

    A positive correlation was found between scores at the baseline visit and scores at the end-of-study visit for both the app-based Cognition Kit DSST (r=0.82, df=27; Figure 4A) and the paper-and-pencil DSST (r=0.92, df=27; Figure 4B). The corresponding partial correlations after adjusting for age were r=0.75 (df=27; P<.001) and r=0.92 (df=27; P<.001) for the app-based Cognition Kit DSST and the paper-and-pencil DSST, respectively.

    ‎
    Figure 4. Test-retest reliability of (A) app-based DSST and (B) paper-and-pencil DSST (N=30). Results are not adjusted for age. DSST: Digit Symbol Substitution Test.
    View this figure

    Cognition Kit DSST App Satisfaction

    In the app satisfaction survey, 87% (26/30) of patients reported that they agreed or strongly agreed that the Cognition Kit DSST app was user-friendly and easy to navigate. Overall, patients reported the highest levels of agreement with the following statements, each of which received a mean score >4 on the 5-point Likert scale (where a score of 4 indicated “agree” and a score of 5 indicated “strongly agree”): “I like when my symptoms of depression are evaluated with measurement tools”; “The time required to complete the DSST app is reasonable”; “I found the DSST app user-friendly and easy to navigate”; and “The instructions for the DSST app are understandable” (Figure 5). Additionally, 57% (17/30) of patients reported that they preferred to use the Cognition Kit DSST over the paper-and-pencil DSST.

    ‎
    Figure 5. App satisfaction survey mean scores (N=30). Each question was scored on a 5-point Likert scale where 1=strongly disagree and 5=strongly agree. DSST: Digit Symbol Substitution Test; QOL: quality of life.
    View this figure

    Principal Results

    The Cognition Kit DSST was capable of detecting cognitive deficits in adults with MDD. The app-based DSST was highly correlated with the standard paper-and-pencil version of the test and was positively evaluated by 30 patients as being user-friendly, easy to navigate, and preferable to the paper-and-pencil version of the DSST. The change in mean DSST scores from baseline to end of the study observed in both the paper-and-pencil and app-based DSST may be due to intervention effects, random effects, regression to mean, or practice effects [16,20].

    Comparison With Prior Work

    Antidepressant therapy may relieve depressive symptoms, but resolution of these symptoms is not well correlated with functional recovery and quality of life, which are often higher priority outcomes for patients with MDD [21]. Therefore, regular assessment of functional outcomes using validated measures, such as the DSST, may assist health care professionals in optimizing treatment for patients with MDD [21].

    MBC has been highly evaluated in clinical practice in adult patients with MDD [22], but uptake remains low (<20%) in mental health settings [21]. Therefore, it is important that a convenient and accessible method of assessment is made available to patients with MDD, especially those who may be experiencing a degree of cognitive impairment. Indeed, the National Institute of Mental Health’s public health trial—Sequenced Treatment Alternatives to Relieve Depression—demonstrated the usefulness of MBC for guiding psychotherapeutic and pharmacologic treatment approaches in patients with depression [22]. Strategies to reduce barriers to MBC and appropriately implement it as part of routine practice should be prioritized.

    Neurocognitive tests via smartphones offer an effective and accessible tool that can be readily implemented into an MBC framework. In particular, individual tests such as the Cognition Kit DSST can be integrated into a suite of clinical measures within a single app, allowing multiple clinically relevant assessments to be performed. For example, depressive symptom, functioning, and quality of life measures may be offered alongside medication adherence and adverse event reporting, as well as reminders.

    The THINC-it Tool has been previously validated as a screening tool and as a repeat measure for cognitive function in adults with MDD [12,23]. A variant of the DSST was included in the THINC-it Tool and accounted for significant variance in the tool’s overall performance, suggesting that the DSST alone may provide sufficient conceptual coverage [12,23,24]. This study demonstrated the validity of the app-based Cognition Kit DSST to assess cognitive impairment in patients with MDD and represents a personalized assessment approach that may help guide MBC to inform psychotherapeutic and pharmacologic treatment options.

    Moreover, applying smartphone technology may help clinicians to more fully understand the mediational role of cognition in MDD, particularly the extent to which it interferes with daily life in patients with persistent psychosocial and workplace impairment [6]. Integrating cognitive functions as part of the assessment of MDD may inform suicide risk, as suicidality in some cases may be linked to cognitive function [25]. Therefore, there is a need to assess cognition in these patients and in patients who do not functionally recover and will continue to exhibit cognitive impairment despite treatment. Deploying an easily accessible smartphone-based testing regimen that can be completed outside the clinical environment at no cost and with limited inconvenience to the patient may help remove barriers to routine assessment.

    There is a risk, however, that assessments performed using a paper-and-pencil approach may differ compared with a smartphone-based assessment. The DSST is a polyfactorial test that assesses motor speed, attention, and visuoperceptual functions, all of which may be subject to subtle inter- and intraindividual differences when evaluated using a paper-and-pencil approach versus a smartphone-based approach [16]. However, high test-retest reliability has previously been demonstrated with the paper-and-pencil version [26]. DSST performance is known to be reduced with increasing age and may be negatively influenced by physical impairments relating to vision or motor skills, but level of education does not appear to significantly influence performance [27]. Women may also perform better than men [28]. Therefore, additional neurocognitive testing may be required alongside the DSST to confirm any deficit.

    Preliminary evidence suggests that there are differences among antidepressants in their ability to affect cognitive function in patients with MDD [5], indicating that cognitive function assessment is highly relevant when initiating pharmacologic therapy. In fact, DSST has been effectively used to assess improved performance in patients with MDD when comparing 2 different antidepressants [5,29]. That the Cognition Kit DSST was able to detect cognitive dysfunction in patients with MDD demonstrates that it may be implemented into an MBC framework because it is capable of guiding treatment decision-making for clinicians.

    Implementing the Cognition Kit DSST in routine clinical practice has several benefits for both clinicians and patients. Enabling patients to complete neurocognitive assessment tools on their smartphones conserves clinical resources and streamlines the assessment process. For example, having a preassessed electronic record of cognitive symptom status can improve testing accuracy and consistency, while enabling clinicians to focus appointment time on treatment as opposed to administering and interpreting paper-and-pencil versions of cognitive assessments [30]. Furthermore, patients can easily and privately access app-based assessments at their convenience. Currently, assessment is recommended every 2-4 weeks if clinically appropriate, but patients could complete the Cognition Kit DSST more frequently, if required, to build a rich longitudinal picture of their cognition during a clinical study or as part of a health care pathway. However, when deploying these electronic assessments, consideration needs to be given to applying appropriate data protection measures to ensure patient privacy [30], especially given that individuals with depression may be concerned about employers, for example, becoming aware of a diagnosis of MDD or a degree of cognitive impairment [30].

    This study had several strengths, including the patient population being representative of patients with MDD based on MADRS score, diagnosis confirmed by the MINI, and anxiety and anhedonia assessed by HAM-A and SHAPS, respectively. Furthermore, patients with heterogeneous illness presentation and course were eligible, including those receiving psychotropic medications in combination with medications for concurrent comorbidities.

    Limitations and Future Directions

    Although the app-based Cognition Kit DSST was validated in patients with MDD, it still needs to be evaluated in healthy controls and individuals with MDD in other settings (eg, primary care). Additionally, Cognition Kit DSST sensitivity to change remains to be assessed. Our study excluded patients whose principal diagnosis was not MDD, which was another limitation. Further, only one standardized measure of cognitive function was validated in a relatively small sample, and patients were recruited from treatment-resistant depression centers. These factors may limit the generalizability of study results to a broader patient population, including those with treatment-responsive MDD or those who are treatment naive. Likewise, these results may not be generalizable to individuals with neurological disorders other than MDD, such as dementia, or those with learning differences, such as dyslexia.

    The DSST has been extensively studied and is well regarded as a multidomain assessment of cognitive function [16]; however, it may disproportionately evaluate processing speed, and therefore, may not provide adequate conceptual coverage of other subdomains of neurocognition. Furthermore, the Cognition Kit DSST does not contain any self-reported measures of cognitive function, and we acknowledge that self-reported cognitive function does not correlate with objective cognitive function.

    Conclusions

    This study demonstrated that cognitive function assessments performed using the Cognition Kit DSST app correlated with the paper-and-pencil version of the test, detecting cognitive deficits in adults with MDD. Future research efforts should focus on validating the Cognition Kit DSST in a healthy control population and in a larger MDD patient population. Research is needed into the Cognition Kit DSST app’s sensitivity to change with treatment, the cost-effectiveness and impact on therapeutic outcomes of implementing the app, as well as the app’s implications for health outcomes.

    Acknowledgments

    Under direction of the authors, medical writing assistance was provided by Blair Hesp, PhD, CMPP, and Namita Satija, PhD, on behalf of Syneos Health Medical Communications, LLC. Takeda Pharmaceuticals U.S.A., Inc., and H. Lundbeck A/S provided funding to Syneos Health for support in writing this manuscript.

    This study was a collaborative study funded by Takeda Pharmaceuticals U.S.A., Inc., and Lundbeck LLC, and the sponsors were responsible for codevelopment of the study design; data collection, analysis, and interpretation; the writing of the study report; and the decision to submit the paper for publication.

    Authors' Contributions

    RSM, OL, NBR, MS, FN, YL, BF, JK, LC, KK, SU, JDR, RBM, and MM contributed to the design and implementation of the study, the analysis of results, and the writing of the manuscript. All authors reviewed and approved the final manuscript for submission.

    Conflicts of Interest

    MM, LC, and SU are employees of Takeda Pharmaceuticals U.S.A., Inc. RSM has received research grant support from the Canadian Institutes of Health Research (CIHR), Global Alliance for Chronic Diseases (GADC), and the National Natural Science Foundation of China and the Milken Institute; he has received speaker or consultation fees from Lundbeck, Janssen, Alkermes, Neumora Therapeutics, Boehringer Ingelheim, Sage, Biogen, Mitsubishi Tanabe, Purdue, Pfizer, Otsuka, Takeda, Neurocrine, Sunovion, Bausch Health, Axsome, Novo Nordisk, Kris Pharma, Sanofi, Eisai, Intra-Cellular, NewBridge Pharmaceuticals, Viatris, AbbVie, and Atai Life Sciences; and he is CEO of Braxia Scientific Corp. JDR has received research grant support from the Canadian Cancer Society, Canadian Psychiatric Association, American Psychiatric Association, American Society of Psychopharmacology, University of Toronto, University Health Network Centre for Mental Health, Joseph M. West Family Memorial Fund, and Timeposters Fellowship, as well as industry funding for speaker, consultation, or research fees from Janssen, Allergan, Lundbeck, Sunovion, and COMPASS. JDR is also the medical director of a private clinic providing intravenous ketamine infusions and intranasal esketamine for depression. KK is an employee of the Canadian Rapid Treatment Center of Excellence in Mississauga, ON, Canada. JK is an employee of Ctrl Group. BF is an employee of Ctrl Group and director of Cognition Kit. YL received a personal fee from Champignon Brand Inc. OL, MS, NBR, RBM, and FN have no conflicts of interest related to this study.

    ‎
    Multimedia Appendix 1

    Evaluation of an app-based Digit Symbol Substitution Test for assessment of cognitive deficits in adults with major depressive disorder.

    PDF File (Adobe PDF File), 232 KB
    1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). 5th ed. Washington, DC: American Psychiatric Association; 2013.
    2. Practice guideline for the treatment of patients with major depressive disorder. American Psychiatric Association. 2010.   URL: https://psychiatryonline.org/pb/assets/raw/sitewide/practice_guidelines/guidelines/mdd.pdf [accessed 2021-09-21]
    3. Conradi HJ, Ormel J, de Jonge P. Presence of individual (residual) symptoms during depressive episodes and periods of remission: a 3-year prospective study. Psychol Med 2011 Jun;41(6):1165-1174. [CrossRef] [Medline]
    4. Harrison JE, Lophaven S, Olsen CK. Which cognitive domains are improved by treatment with vortioxetine? Int J Neuropsychopharmacol 2016 May 26;19(10):1-6 [FREE Full text] [CrossRef] [Medline]
    5. McIntyre RS, Harrison J, Loft H, Jacobson W, Olsen CK. The effects of vortioxetine on cognitive function in patients with major depressive disorder: a meta-analysis of three randomized controlled trials. Int J Neuropsychopharmacol 2016 Jun 15;19(10):1-9 [FREE Full text] [CrossRef] [Medline]
    6. McIntyre RS, Soczynska JZ, Woldeyohannes HO, Alsuwaidan MT, Cha DS, Carvalho AF, et al. The impact of cognitive impairment on perceived workforce performance: results from the International Mood Disorders Collaborative Project. Compr Psychiatry 2015 Jan;56:279-282. [CrossRef] [Medline]
    7. McIntyre RS, Cha DS, Soczynska JK, Woldeyohannes HO, Gallaugher LA, Kudlow P, et al. Cognitive deficits and functional outcomes in major depressive disorder: determinants, substrates, and treatment interventions. Depress Anxiety 2013 Jun;30(6):515-527. [CrossRef] [Medline]
    8. Chiauzzi E, Drahos J, Sarkey S, Curran C, Wang V, Tomori D. Patient perspective of cognitive symptoms in major depressive disorder: retrospective database and prospective survey analyses. J Particip Med 2019 May 16;11(2):e11167 [FREE Full text] [CrossRef] [Medline]
    9. Gorwood P, Corruble E, Falissard B, Goodwin GM. Toxic effects of depression on brain function: impairment of delayed recall and the cumulative length of depressive disorder in a large sample of depressed outpatients. Am J Psychiatry 2008 Jun;165(6):731-739. [CrossRef] [Medline]
    10. Lee Y, Smofsky A, Nykoliation P, Allain SJ, Lewis-Daly L, Schwartz J, et al. Cognitive impairment mediates workplace impairment in adults with type 2 diabetes mellitus: results from the motivaction study. Can J Diabetes 2018 Jun;42(3):289-295. [CrossRef] [Medline]
    11. Knight MJ, Lyrtzis E, Baune BT. The association of cognitive deficits with mental and physical quality of life in major depressive disorder. Compr Psychiatry 2020 Feb;97:152147 [FREE Full text] [CrossRef] [Medline]
    12. McIntyre RS, Lee Y, Mansur RB. Treating to target in major depressive disorder: response to remission to functional recovery. CNS Spectr 2015 Dec;20 Suppl 1:20-30; quiz 31. [CrossRef] [Medline]
    13. McIntyre RS, Best MW, Bowie CR, Carmona NE, Cha DS, Lee Y, et al. The THINC-Integrated Tool (THINC-it) screening assessment for cognitive dysfunction: validation in patients with major depressive disorder. J Clin Psychiatry 2017 Jul;78(7):873-881. [CrossRef] [Medline]
    14. Lee Y, Brietzke E, Cao B, Chen Y, Linnaranta O, Mansur RB, Global Alliance for Chronic Diseases (GACD) Mental Health Guidelines Working Group. Development and implementation of guidelines for the management of depression: a systematic review. Bull World Health Organ 2020 Oct 01;98(10):683-697H [FREE Full text] [CrossRef] [Medline]
    15. McIntyre RS, Anderson N, Baune BT, Brietzke E, Burdick K, Fossati P, et al. Expert consensus on screening and assessment of cognition in psychiatry. CNS Spectr 2019 Feb;24(1):154-162. [CrossRef] [Medline]
    16. Jaeger J. Digit symbol substitution test: the case for sensitivity over specificity in neuropsychological testing. J Clin Psychopharmacol 2018 Oct;38(5):513-519 [FREE Full text] [CrossRef] [Medline]
    17. Doupis J, Festas G, Tsilivigos C, Efthymiou V, Kokkinos A. Smartphone-Based Technology in Diabetes Management. Diabetes Ther 2020 Mar;11(3):607-619 [FREE Full text] [CrossRef] [Medline]
    18. Nakonezny PA, Morris DW, Greer TL, Byerly MJ, Carmody TJ, Grannemann BD, et al. Evaluation of anhedonia with the Snaith-Hamilton Pleasure Scale (SHAPS) in adult outpatients with major depressive disorder. J Psychiatr Res 2015 Jun;65:124-130 [FREE Full text] [CrossRef] [Medline]
    19. Baldwin DS, Florea I, Jacobsen PL, Zhong W, Nomikos GG. A meta-analysis of the efficacy of vortioxetine in patients with major depressive disorder (MDD) and high levels of anxiety symptoms. J Affect Disord 2016 Dec;206:140-150 [FREE Full text] [CrossRef] [Medline]
    20. Morton V, Torgerson DJ. Effect of regression to the mean on decision making in health care. BMJ 2003 May 17;326(7398):1083-1084 [FREE Full text] [CrossRef] [Medline]
    21. Hong RH, Murphy JK, Michalak EE, Chakrabarty T, Wang Z, Parikh SV, et al. Implementing measurement-based care for depression: practical solutions for psychiatrists and primary care physicians. Neuropsychiatr Dis Treat 2021;17:79-90 [FREE Full text] [CrossRef] [Medline]
    22. Lewis CC, Scott K, Marti CN, Marriott BR, Kroenke K, Putz JW, et al. Implementing measurement-based care (iMBC) for depression in community mental health: a dynamic cluster randomized trial study protocol. Implement Sci 2015 Sep 07;10:127 [FREE Full text] [CrossRef] [Medline]
    23. Harrison JE, Barry H, Baune BT, Best MW, Bowie CR, Cha DS, et al. Stability, reliability, and validity of the THINC-it screening tool for cognitive impairment in depression: A psychometric exploration in healthy volunteers. Int J Methods Psychiatr Res 2018 Dec;27(3):e1736 [FREE Full text] [CrossRef] [Medline]
    24. McIntyre RS, Subramaniapillai M, Park C, Zuckerman H, Cao B, Lee Y, et al. The THINC-it tool for cognitive assessment and measurement in major depressive disorder: sensitivity to change. Front Psychiatry 2020;11:546 [FREE Full text] [CrossRef] [Medline]
    25. Lee Y, Syeda K, Maruschak NA, Cha DS, Mansur RB, Wium-Andersen IK, et al. A new perspective on the anti-suicide effects with ketamine treatment: a procognitive effect. J Clin Psychopharmacol 2016 Feb;36(1):50-56. [CrossRef] [Medline]
    26. Matarazzo JD, Herman DO. Base rate data for the WAIS-R: test-retest stability and VIQ-PIQ differences. J Clin Neuropsychol 1984 Nov;6(4):351-366. [CrossRef] [Medline]
    27. Hoyer WJ, Stawski RS, Wasylyshyn C, Verhaeghen P. Adult age and digit symbol substitution performance: a meta-analysis. Psychol Aging 2004 Mar;19(1):211-214. [CrossRef] [Medline]
    28. Kertzman S, Ben-Nahum Z, Gotzlav I, Grinspan H, Birger M, Kotler M. Digit Symbol Substitution test performance: sex differences in a Hebrew-readers' health population. Percept Mot Skills 2006 Aug;103(1):121-130. [CrossRef] [Medline]
    29. Rosenblat JD, Kakar R, McIntyre RS. The cognitive effects of antidepressants in major depressive disorder: a systematic review and meta-analysis of randomized clinical trials. Int J Neuropsychopharmacol 2015 Jul 25;19(2):1-13 [FREE Full text] [CrossRef] [Medline]
    30. Gold M, Amatniek J, Carrillo MC, Cedarbaum JM, Hendrix JA, Miller BB, et al. Digital technologies as biomarkers, clinical outcomes assessment, and recruitment tools in Alzheimer's disease clinical trials. Alzheimers Dement (N Y) 2018;4:234-242 [FREE Full text] [CrossRef] [Medline]

    ‎
    DSST: Digit Symbol Substitution Test
    HAM-A: Hamilton Anxiety Rating Scale
    MADRS: Montgomery-Åsberg Depression Rating Scale
    MBC: measurement-based care
    MDD: major depressive disorder
    MINI: Mini-International Neuropsychiatric Interview
    PROs: patient-reported outcomes
    SHAPS: Snaith-Hamilton Pleasure Scale

    Edited by J Torous; submitted 04.10.21; peer-reviewed by L Culpepper, P Harvey, A Silven; comments to author 03.01.22; revised version received 29.06.22; accepted 20.07.22; published 27.10.22

    Copyright

    ©Roger S McIntyre, Orly Lipsitz, Nelson B Rodrigues, Mehala Subramaniapillai, Flora Nasri, Yena Lee, Ben Fehnert, James King, Lambros Chrones, Kevin Kratiuk, Sharif Uddin, Joshua D Rosenblat, Rodrigo B Mansur, Maggie McCue. Originally published in JMIR Mental Health (https://mental.jmir.org), 27.10.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.