JMH JMIR Ment Health JMIR Mental Health 2368-7959 JMIR Publications Toronto, Canada v7i5e14505 32436853 10.2196/14505 Letter to the Editor Letter to the Editor Comment on “Web-Based Measure of Life Events Using Computerized Life Events and Assessment Record (CLEAR): Preliminary Cross-Sectional Study of Reliability, Validity, and Association With Depression”: Validity and Methodological Issues Torous John Bifulco Antonia Rahmani Jamal MSc, PhD 1 https://orcid.org/0000-0002-2007-9241 Karimi Roya MSc, PhD 2 https://orcid.org/0000-0003-4746-7575 Mohtasham Farideh MSc, PhD 3 https://orcid.org/0000-0003-0987-7485 Sabour Siamak MD, PhD 2
Department of Clinical Epidemiology School of Health and Safety Shahid Beheshti University of Medical Sciences Chamran Highway Tehran Iran 98 21 22421814 s.sabour@sbmu.ac.ir
4 https://orcid.org/0000-0002-1928-992X National Nutrition and Food Technology Research Institute Shahid Beheshti University of Medical Sciences Tehran Iran Department of Clinical Epidemiology School of Health and Safety Shahid Beheshti University of Medical Sciences Tehran Iran Department of Clinical Epidemiology School of Health and Research Institute for Gastroenterology and Liver Diseases Shahid Beheshti University of Medical Sciences Tehran Iran Safety Promotions and Injury Prevention Research Centre Shahid Beheshti University of Medical Sciences Tehran Iran Corresponding Author: Siamak Sabour s.sabour@sbmu.ac.ir 5 2020 21 5 2020 7 5 e14505 26 4 2019 10 7 2019 ©Jamal Rahmani, Roya Karimi, Farideh Mohtasham, Siamak Sabour. Originally published in JMIR Mental Health (http://mental.jmir.org), 21.05.2020. 2020

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.

 https://mental.jmir.org/2019/1/e10675/ https://mental.jmir.org/2020/5/e15434/ validity methodological issues diagnostic test

We were interested in the article titled, “Web-Based Measure of Life Events Using Computerized Life Events and Assessment Record (CLEAR): Preliminary Cross-Sectional Study of Reliability, Validity, and Association With Depression” published in JMIR Mental Health [1].

One of the aims of the abovementioned study was to assess the validity of Computerized Life Events and Assessment Record (CLEAR), considering the Life Events and Difficulties Schedule (LEDS) and the List of Threatening Experiences Questionnaire (LTE-Q) as gold standards among 328 participants (126 students; 202 matched midlife sample: 127 unaffected controls, 75 recurrent depression cases). The authors concluded that CLEAR has acceptable validity and great potential for effective use in research and clinical practice. However, there are some methodological issues in this conclusion that are mentioned below.

First, there are some measures that can be applied to the assessment of the validity of a test including sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio (LR+; ranging from 1 to infinity; the higher the LR+, the more accurate is the test), negative likelihood ratio (LR–; ranging from 0 to 1; the lower the LR–, the more accurate is the test), and odds ratio (ratio of true to false results) [2-5]. According to the results, sensitivity of CLEAR was 59.1% and 43.1% compared to LEDS and LTE-Q, respectively, as gold standards. Likewise, specificity of CLEAR was 65.4% and 78.6%, respectively, compared to the abovementioned gold standards.

It is good to know that sensitivity is an important measure in public health aspects instead of clinical fields. Likewise, the positive predictive value and negative predictive value are among measures that are more appropriate for advice about the validity of a diagnostic test for clinical purposes [3-5]. Therefore, we suggest applying predictive values, likelihood ratios, odds ratio, and diagnostic accuracy to decide the validity of CLEAR. Moreover, according to the data of study, LR+, LR–, odds ratio, and diagnostic accuracy of CLEAR will be 1.6, 0.6, 2.6, and 62%, respectively, compared to LEDS (Tables 1 and 2) and 1.9, 0.7, 2.6, and 60%, respectively compared to LTE-Q (Tables 3 and 4). Therefore, there is a high level of uncertainty for decisions based on these values, and there is insufficient evidence to conclude that the validity of the CLEAR test is acceptable.

Two by two table of Computerized Life Events and Assessment Record compared to Life Events and Difficulties Schedule as the gold standard.

CLEARa LEDSb (gold standard)
Positive Negative Total
Positive 59 35 94
Negative 41 65 106
Total 100 100 200

aCLEAR: Computerized Life Events and Assessment Record.

bLEDS: Life Events and Difficulties Schedule.

Assessing the validity of Computerized Life Events and Assessment Record compared to Life Events and Difficulties Schedule as the gold standard.

Parameter Estimate
Sensitivity (%) 59
Specificity (%) 65
Positive predictive value (%) 63
Negative predictive value (%) 61
Diagnostic accuracy (%) 62
Likelihood ratio of a positive test 1.6
Likelihood ratio of a negative test 0.6
Diagnostic odds 2.6

Two by two table of Computerized Life Events and Assessment Record compared to List of Threatening Experiences Questionnaire as the gold standard.

CLEARa LTE-Qb (gold standard)
Positive Negative Total
Positive 43 22 65
Negative 57 78 135
Total 100 100 200

aCLEAR: Computerized Life Events and Assessment Record.

bLTE-Q: List of Threatening Experiences Questionnaire.

Assessing the validity of Computerized Life Events and Assessment Record compared to List of Threatening Experiences Questionnaire as the gold standard.

Parameter Estimate
Sensitivity (%) 43
Specificity (%) 78
Positive predictive value (%) 66
Negative predictive value (%) 58
Diagnostic accuracy (%) 60
Likelihood ratio of a positive test 1.9
Likelihood ratio of a negative test 0.7
Diagnostic odds 2.6
Abbreviations CLEAR

Computerized Life Events and Assessment Record

 LEDS

Life Events and Difficulties Schedule

 LR–

negative likelihood ratio

 LR+

positive likelihood ratio

 LTE-Q

List of Threatening Experiences Questionnaire

None declared.

 Bifulco A Spence R Nunn S Kagan L Bailey-Rodriguez D Hosang GM Taylor M Fisher HL Web-Based Measure of Life Events Using Computerized Life Events and Assessment Record (CLEAR): Preliminary Cross-Sectional Study of Reliability, Validity, and Association With Depression JMIR Ment Health 2019 01 08 6 1 e10675 10.2196/10675 30622088 v6i1e10675 PMC6329417 Szklo M Nieto J Epidemiology: beyond the basics. 3rd ed 2014 Manhattan, NY Jones and Bartlett Learning Sabour S Ghassemi F The validity and reliability of a signal impact assessment tool: statistical issue to avoid misinterpretation Pharmacoepidemiol Drug Saf 2016 10 25 10 1215 1216 10.1002/pds.4061 27696610 Sabour S Validity and reliability of the new Canadian Nutrition Screening Tool in the 'real-world' hospital setting: Methodological issues Eur J Clin Nutr 2015 07 69 7 864 10.1038/ejcn.2015.69 25920428 ejcn201569 Sabour S Validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia in Mexican adults; statistical issues in validity and reliability analysis Clin Chem Lab Med 2014 12 52 12 e295 6 10.1515/cclm-2014-0453 24933629 /j/cclm.ahead-of-print/cclm-2014-0453/cclm-2014-0453.xml