Published on in Vol 9, No 4 (2022): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/34105, first published .
Assessment of Population Well-being With the Mental Health Quotient: Validation Study

Assessment of Population Well-being With the Mental Health Quotient: Validation Study

Assessment of Population Well-being With the Mental Health Quotient: Validation Study

Original Paper

Corresponding Author:

Jennifer Jane Newson, BSc, DPhil

Sapien Labs

1201 Wilson Blvd, 27th floor

Arlington, VA, 22209

United States

Phone: 1 7039976657

Email: jennifer@sapienlabs.org


Background: The Mental Health Quotient (MHQ) is an anonymous web-based assessment of mental health and well-being that comprehensively covers symptoms across 10 major psychiatric disorders, as well as positive elements of mental function. It uses a novel life impact scale and provides a score to the individual that places them on a spectrum from Distressed to Thriving along with a personal report that offers self-care recommendations. Since April 2020, the MHQ has been freely deployed as part of the Mental Health Million Project.

Objective: This paper demonstrates the reliability and validity of the MHQ, including the construct validity of the life impact scale, sample and test-retest reliability of the assessment, and criterion validation of the MHQ with respect to clinical burden and productivity loss.

Methods: Data were taken from the Mental Health Million open-access database (N=179,238) and included responses from English-speaking adults (aged≥18 years) from the United States, Canada, the United Kingdom, Ireland, Australia, New Zealand, South Africa, Singapore, India, and Nigeria collected during 2021. To assess sample reliability, random demographically matched samples (each 11,033/179,238, 6.16%) were compared within the same 6-month period. Test-retest reliability was determined using the subset of individuals who had taken the assessment twice ≥3 days apart (1907/179,238, 1.06%). To assess the construct validity of the life impact scale, additional questions were asked about the frequency and severity of an example symptom (feelings of sadness, distress, or hopelessness; 4247/179,238, 2.37%). To assess criterion validity, elements rated as having a highly negative life impact by a respondent (equivalent to experiencing the symptom ≥5 days a week) were mapped to clinical diagnostic criteria to calculate the clinical burden (174,618/179,238, 97.42%). In addition, MHQ scores were compared with the number of workdays missed or with reduced productivity in the past month (7625/179,238, 4.25%).

Results: Distinct samples collected during the same period had indistinguishable MHQ distributions and MHQ scores were correlated with r=0.84 between retakes within an 8- to 120-day period. Life impact ratings were correlated with frequency and severity of symptoms, with a clear linear relationship (R2>0.99). Furthermore, the aggregate MHQ scores were systematically related to both clinical burden and productivity. At one end of the scale, 89.08% (8986/10,087) of those in the Distressed category mapped to one or more disorders and had an average productivity loss of 15.2 (SD 11.2; SEM [standard error of measurement] 0.5) days per month. In contrast, at the other end of the scale, 0% (1/24,365) of those in the Thriving category mapped to any of the 10 disorders and had an average productivity loss of 1.3 (SD 3.6; SEM 0.1) days per month.

Conclusions: The MHQ is a valid and reliable assessment of mental health and well-being when delivered anonymously on the web.

JMIR Ment Health 2022;9(4):e34105

doi:10.2196/34105

Keywords



Background

The World Health Organization defines mental health as “a state of well-being in which the individual realizes his or her own abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community” [1]. On the basis of this definition, assessments of mental health should reflect the presence of dysfunction and also provide insight into the positive aspects of mental functioning [2-5]. However, the clinical heritage of mental health assessment means that most assessment tools are built around specific psychiatric disorder categories taken from the clinical classification systems of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) [6], or the International Classification of Diseases [7] and, therefore, are not designed to provide a perspective on the continuum of mental health and well-being across the general population. In contrast, assessments of mental health that are more relevant across the spectrum of the general population can support the early identification of at-risk individuals before symptoms escalate, improve uptake in help-seeking behaviors, and reveal relevant social determinants to support the active management of mental health and well-being through self-care behaviors and preventative strategies and interventions at various scales, from organizations to countries [8-10].

Population-level assessments also provide an opportunity for understanding the scale of mental health challenges at a global level. For example, in 2017, 792 million people were estimated to be living with a mental health disorder worldwide [11], whereas depression is the leading cause of disability as measured by years lived with disability [12]. In addition, suicide was the fourth leading cause of death among people aged 15-29 years worldwide in 2019 [13] and is still poorly understood [14,15]. However, presently, there are few reliable and valid tools that can provide an aggregate and measurable view across the full spectrum of a global population from distressed to thriving as well as estimating clinical burden in an aggregate, disorder-agnostic way. Furthermore, as mental health and well-being can change substantially based on external circumstances, as evidenced by the COVID-19 pandemic [16-18], it is important to have metrics that track the extent and nature of these changes and their impact on clinical burden as well as on the productive capacity of a population. For example, evidence suggests a clear relationship between mental health and well-being and productivity, resulting in both absenteeism and presenteeism [19-25], an increased prevalence of burnout [26,27], and significant personal and economic loss [28-32]. This further highlights the importance of preventative measures for actively managing the mental health and well-being of working-age adults in the general population.

To address this need for a population-based, disorder-agnostic assessment that spans the spectrum of mental health and well-being and for a single measurable metric of mental health and well-being, we developed a new web-based assessment tool delivering a metric called the Mental Health Quotient (MHQ) [33]. The MHQ assesses the complete breadth of mental health elements spanning the range from symptoms to positive mental assets using a unique life impact scale and aims to enable a paradigm for managing and improving the lives and well-being of all people, not just those with a clinical disorder.

The MHQ Assessment

The MHQ was developed based on a comprehensive review of symptoms by coding questions across 126 commonly used psychiatric assessment tools spanning depression, anxiety, bipolar disorder, attention-deficit/hyperactivity disorder, posttraumatic stress disorder, obsessive-compulsive disorder (OCD), addiction, schizophrenia, eating disorders, and autism spectrum disorder as well as cross-disorder tools (see Newson et al [34] for a complete list of assessment tools). These disorders were selected based on a review of the disorders included in the Structured Clinical Interview for the DSM-5, Clinician Version [35]. In addition, autism spectrum disorder and eating disorders were included because of both their prevalence and their broad public and scientific interest. Symptoms from these 126 assessments were consolidated into a set of 43 symptom categories and reviewed and expanded in the context of the Research Domain Criteria constructs put forward by the National Institute of Mental Health [36-38]. The resultant 47 elements were then split into two formats: mental functions that could manifest as a spectrum from positive to negative (spectrum questions) and those symptoms that purely represented detractions from overall mental health (problem questions).

Spectrum and problem questions within the MHQ are answered using a 9-point scale reflecting the consequences on one’s life functioning and impact on their ability to carry out tasks and activities in their daily life. Therefore, the scale is different from traditional mental health assessments, which typically focus on the frequency, severity, duration, or timing of symptoms [34]. An aggregate MHQ score developed using an algorithm that nonlinearly transforms the life impact scale based on different categories of symptom seriousness is provided on completion of the assessment [33]. This score is intended as a representation of the overall mental health and well-being of the individual and is categorized from Distressed (−100) to Thriving (+200).

The MHQ is currently used on the web as part of an open data project called the Mental Health Million Project, which is a web-based platform that monitors the status of population mental health across the globe and currently spans 30 countries and 4 languages (English, French, Spanish, and Arabic). In this paper, we evaluate the potential of the MHQ to be used as a valid and reliable measure of mental health and well-being both at the individual and population levels to determine how mental health and well-being evolves over time across the globe and the impact of these changes on clinical burden and productivity. We aim to address (1) how the unique MHQ life impact scale relates to more commonly used metrics of frequency and severity; (2) whether an anonymous web-based assessment serves as a true measure of the population by demonstrating the population and test-retest reliability of the MHQ; and (3) how well the composite MHQ score relates to functional criteria such as clinical diagnostic criteria, workdays missed, and overall life productivity. We hypothesize that the MHQ will show good validity and reliability as an assessment of mental health and well-being when delivered anonymously on the web.


Recruitment of Participants

The data were taken from the Mental Health Million open-access database [39] and included responses from 179,238 English-speaking individuals from the United States, Canada, the United Kingdom, Ireland, Australia, New Zealand, South Africa, Singapore, India, and Nigeria collected during 2021. Participants were recruited via outreach campaigns on Facebook and using Google Ads by targeting a broad cross-section of adults aged 18-85 years across a wide geographic and socioeconomic demographic. The anonymous assessment was freely available on the web for anyone to complete, and individuals took the assessment for the purpose of obtaining their personalized mental health and well-being report on completion. The provision of a personal report aimed to ensure greater interest of the respondent in answering questions thoughtfully and accurately. Only respondents who found the assessment easy to understand (ie, responded Yes to the question Did you find this assessment easy to understand?) were included in the analysis. This resulted in the exclusion of 2.58% (4620/179,238) of respondents, leaving 174,618 for the full analysis.

Ethics Approval

The study received ethics approval from the Health Media Lab Institutional Review Board (Office for Human Research Protections Institutional Review Board #00001211, Federal Wide Assurance #00001102, IORG #0000850).

Assessment of Reliability

Reliability Across Randomly Selected Samples

All respondents from the United States, India, Australia, and the United Kingdom between January 2021 and June 2021 were pooled together (44,132/174,618, 25.27%). A total of 4 randomly selected and nonoverlapping samples of 11,033 people with similar demographic composition were selected. The average rating (1-9) of each MHQ-scored element for each sample, the average MHQ score for each sample, and the statistical differences between the samples were then computed.

Internal Consistency Analysis

The MHQ is designed to be as parsimonious as possible without repetition. However, the internal consistency of the MHQ was evaluated by looking at the relative correlations between elements that would be expected to be correlated compared with those that would not. First, the correlation between 2 questions about sleep quality within the MHQ was computed (N=174,618). Sleep question 1 asked respondents to Assess your sleep quality, and sleep question 2 asked respondents: In general, I get as much sleep as I need. The 1-9 rating score from sleep question 1 was correlated with the transformed answers to sleep question 2, where each answer option was assigned a number that was roughly equivalent to the text description: all the time=7, most of the time=5, some of the time=3, and hardly ever=1. Second, the correlation between 2 questions about mood was computed. Mood question 1 asked respondents to Assess your feelings of sadness, distress, or hopelessness and was rated on a 1-9 life impact scale. Mood question 2 asked respondents How would you describe your overall mood right now? and was rated on a 1-9 scale from Very negative to Very positive. Comparisons were also made between responses to the related MHQ elements of Self-worth and confidence and Self-image. In addition, comparisons were made between the elements of Physical intimacy and Memory, Emotional control and Coordination, and Memory and Emotional control, which would not be expected to have a significant correlation.

Test-Retest Reliability

The MHQ is designed to measure changes in the mental health and well-being of the population and, therefore, in individual mental health and well-being status over time. Therefore, the MHQ scores of individuals could change over time. However, over short time frames of less than a year, most individuals would not be expected to change significantly. Within the sample of 174,618 respondents, email addresses were provided by 80,955 (46.36%) to receive their MHQ report. These email addresses were automatically converted into anonymous unique identifiers to identify repeat respondents. Of these 80,955 respondents, 2231 (2.76%) had taken the MHQ twice at varying time intervals up to 15 months from the time of the first assessment. Those who took the MHQ twice within the same day or immediately the next day were excluded as they were more likely to be experimenting with answer choices than evaluating their own change over time in an honest way. Thus, only those who had at least 3 days between attempts were included in the analysis (1907/2231, 85.48%). We examined the test-retest reliability of the MHQ in this sample by looking at the correlation between the element ratings on the first and second attempts as well as the correlation between MHQ scores across both attempts.

Validation of the Life Impact Scale

Clinical assessments are heterogeneous in their evaluation of the frequency and severity of symptoms. For example, a review of 126 assessment tools found that, across 19 commonly used depression scales, 51% of questions asked about frequency of symptoms and 32% asked about severity, whereas, across 9 posttraumatic stress disorder assessment tools, 17% of questions asked about frequency and 53% asked about severity [34]. Given the lack of a clear understanding of which aspect (eg, frequency or duration) of a symptom matters most, the MHQ uses a 9-point life impact scale reflecting the impact of a particular mental aspect on one’s ability to function [33]. For example, for questions pertaining to mental health challenges, 1 referred to Never causes me any problems, 9 referred to Has a constant and severe impact on my ability to function, and 5 referred to Sometimes causes me difficulties or distress but I can manage. For the purpose of validation, for the question that asked individuals to rate the impact of their Feelings of sadness, distress, or hopelessness on this 9-point scale, two additional questions were asked when a value of ≥5 was selected: (1) How many days in the last week did you experience these feelings? with options for selection from 0-7 (similar to the format in depression screening tools such as the Center for Epidemiologic Studies–Depression scale [40,41]) and (2) On these days, how did these feelings impact your ability to function in life? with five options of increasing severity—1=They would come and go while I went about my life as normal; 2=I did what I had to do, but they were always there in the back of my mind; 3=I managed but it took extreme effort; 4=They stopped me doing the things I usually do, or would want to do; and 5=They consumed me so much I was unable to get out of bed. The average frequency and severity and the standard error of measurement (SEM) were then computed for each selection from 5 to 9 on the life impact scale.

Relationship Between MHQ Score and Clinical Burden

The computation of the MHQ score takes into account the number of severe symptoms (ie, those scored as having a highly negative life impact). Thus, the number of elements with a rating that signifies a highly negative life impact decreases as the MHQ score increases, although the nonlinear weighting differs for different types of symptoms [33]. To assess how effectively the MHQ score relates to clinical burden, we mapped elements of the MHQ to the diagnostic criteria for each of the 10 major DSM-5 disorders on which the MHQ is based (see The MHQ Assessment section) and examined (1) the percentage of individuals meeting the diagnostic criteria for at least one disorder and (2) the average number of diagnoses per person for each MHQ score bin of 25. Full details of the thresholds used to determine the presence of a clinical symptom and the mapping to the DSM-5 disorder criteria are described in Newson et al [42]. In brief, MHQ elements were first mapped to the symptoms described within the diagnostic criteria for each of the 10 DSM-5 disorders based on the closest semantic match. For each of the 47 MHQ elements, responses were determined to be clinically significant symptoms if they met a particular threshold of impact on the individual’s ability to function, approximately equivalent to experiencing the symptom 5 days a week (≥8 for problem elements and ≤1 for spectrum elements). The specific diagnostic criteria rules of the DSM-5 (eg, must be experiencing ≥5 symptoms) were then applied to arrive at a disorder diagnostic mapping. A set of rules using combinations of the DSM-5–mapped MHQ elements was then developed to align with these criteria descriptions for each of the 10 disorders. For each respondent (N=174,618), these rules were applied to their MHQ clinical symptom profile to determine the diagnostic match to each of the 10 disorders. See Newson et al [42] and the Limitations section below for further discussion of this approach.

Relationship Between MHQ Score and Productivity Criterion

To assess the relationship between the MHQ score and measures of functional productivity, a subset of participants (7625/174,618, 4.37%) were asked two additional questions: (1) How many days during the past month were you totally unable to work or carry out your normal activities because of problems with your physical or mental health and (2) How many days during the past month were you able to work and carry out your normal activities, but could not get as much done because of problems with your physical or mental health? with options to select a number between 0 and 31. Individuals were then grouped by MHQ score in bins of 25, and the average and SEM of days of work missed (M) and days with reduced productivity (R) were then computed for each bin. We then computed the overall loss of life productivity for each individual as M + n *R, where n represented an assumed loss of productivity on those days ranging from 20% to 50%. Data were examined for all respondents together and for a subset of respondents who answered Employed/Self-Employed to the MHQ question Please select which best describes your occupational status? (alternative answer options included Homemaker, Unemployed, Retired, Studying, and Not able to work).


Reliability and Internal Consistency in the MHQ

Assessment Reliability

Figure 1A shows the average rating for each element of the assessment for spectrum (left) and problem (right) elements across the 4 randomly selected demographically matched samples. Across all samples, the ratings were correlated with r>0.8 for all pairs, and the distributions of ratings for individual elements were highly similar and statistically indistinguishable (analysis of variance; P=.99), with an example from the element Self-image shown in Figure 1B. Similarly, the distributions of the resulting MHQ scores for each of these 4 samples were highly similar (Figure 1C, analysis of variance; P=.18). These results confirm that the MHQ, when offered anonymously and on the web, produces similar results across similar samples. Should responses have been randomly generated (eg, by bots) or if individuals had highly inconsistent interpretations of the life impact scale, this would not have been the case.

Figure 1. Reliability of the Mental Health Quotient (MHQ). (A) The average ratings of each of the 27 MHQ spectrum elements (left) and 20 MHQ problem elements (right) in 4 separate samples of the MHQ obtained over a similar period were indistinguishable (each bar is a sample). (B) Distribution of ratings for an example MHQ element (Self-image) in each of the 4 samples (each line is a sample). (C) Distribution of MHQ scores in each of the 4 samples (each line is a sample) for an example MHQ element (Self-image).
View this figure
Internal Consistency

Among the related elements, the 2 questions relating to sleep quality and sleep sufficiency had a 0.63 correlation. Thus, those who had challenges with sleep quality were also likely to have fewer days of sufficient sleep. Similarly, the 2 questions relating to mood had a 0.64 correlation, indicating that those with a more significant impact of Feelings of sadness, distress, or hopelessness were also more likely to have a negative mood at the time of taking the assessment. Finally, the life impact rating of the MHQ element Self-image had a 0.77 correlation with the rating of the element Self-worth and confidence. In contrast, ratings of unrelated elements had lower correlations. For example, Memory and Physical intimacy had a correlation of 0.35, Emotional control and Coordination had a correlation of 0.36, and Memory and Emotional control had a correlation of 0.39. Therefore, related elements within the MHQ were more highly correlated than the unrelated elements examined.

Test-Retest Reliability

Among all those who could be identified as having taken the MHQ twice at least 3 days apart, MHQ scores were correlated with r=0.84 (P<.001). Figure 2 shows the MHQ scores for the test plotted against the MHQ scores for the retest, demonstrating that points fall around the line y=x. Furthermore, this correlation did not change significantly as the interval between attempts increased, although correlations were as high as r=0.88 for retest intervals of 8-120 days. The correlations (r) were as follows for MHQ scores and MHQ items ratings respectively: 3 to 7 days=0.7, 0.58; 8 to 30 days=0.88, 0.73; 31 to 60 days=0.88, 0.72; 61 to 120 days=0;.83, 0.7; 121 to 450 days=0.79, 0.68). Finally, the correlation between the ratings of individual elements on each attempt was r=0.70 (P<.001) and did not change as the interval between attempts increased. Thus, the MHQ had high test-retest reliability but also reflected changes that can occur in mental health and well-being over time.

Figure 2. Test-retest reliability of the Mental Health Quotient (MHQ). MHQ score for the assessment retake (retest) versus MHQ score for the first take (test). The red line represents y=x. MHQ: Mental Health Quotient.
View this figure

Relationship Between Life Impact Score and Frequency and Severity

MHQ life impact ratings ≥5 (corresponding to the negative end of the scale) for the MHQ element Feelings of sadness, distress, or hopelessness (data subset of 4247/174,618, 2.43%) were correlated with symptom frequency measured as the number of days in the past week where they experienced the symptom, with r=0.5. At the aggregate level, the mean and SEM of frequency for each rating on the life impact scale were linearly related, with R2=0.99 (Figure 3A). Extrapolation of this function to life impact ratings <5 shows that those selecting 1 (the lowest end of the scale, indicating no impact) would have experienced that symptom at a frequency of <1 day in the previous week. Across all data, life impact was similarly positively correlated with the level of severity selected (where levels of severity were coded from 1 to 5) but less so than with frequency (r=0.32). However, the aggregate mean and SEM of severity for each life impact rating were also linearly related (Figure 3B; R2=0.99). Finally, in the aggregate, a composite measure of frequency × severity was nonlinearly related to the life impact rating, with R2=0.98 (Figure 3C). Therefore, we demonstrated a strong relationship between the rating on the life impact scale and both frequency and severity of symptoms.

Figure 3. Example relationship between life impact and frequency and severity. (A) The selection on the Mental Health Quotient (MHQ) life impact scale for Feelings of sadness, distress, and hopelessness was linearly related to the frequency of these feelings. (B) The selection on the MHQ life impact scale was similarly linearly related to the severity of the symptom. (C) Frequency × severity was nonlinearly related to the MHQ life impact selection.
View this figure

Relationship Between MHQ Score and Clinical Burden

First, as would be expected, the average number of clinical symptoms increased as the MHQ score decreased given the nonlinear weighting of symptom severity within the MHQ score (Figure 4A). Beyond this, the percentage of people with clinical symptom profiles that aligned with any of the 10 DSM-5–defined disorder criteria increased as the MHQ score decreased such that 89.08% (8986/10,087) of those in the Distressed category (MHQ score <−50) had symptom profiles that aligned with at least one of the 10 DSM-5–defined disorders, whereas 0.03% (21/70,367) in the categories of Succeeding and Thriving (MHQ score >100) had profiles that aligned with at least one disorder (Figure 4B). Similarly, the number of disorders per individual decreased systematically as MHQ scores increased, with the average number of disorders per person at 3.8 (SD 2.7) for those in the Distressed group and 0.0 (SD 0.02) for those in the Succeeding and Thriving groups (Figure 4C). Thus, the MHQ score is also reflective of the overall clinical burden of mental health.

Figure 4. Relationship between Mental Health Quotient (MHQ) score and clinical symptoms and diagnosis. (A) The average number of symptoms (life impact ≥8 for problem elements and ≤1 for spectrum elements) decreased as the MHQ score increased. The grey area represents the negative side of the scale or the MHQ score categories of Distressed and Struggling (all panels). (B) Approximately 97.83% (3926/4013) of those with the lowest MHQ scores (−100 to −75) mapped to at least one of 10 major clinical disorders and decreased systematically. (C) The average number of disorders per person decreased with MHQ score.
View this figure

Loss of Function Criterion Validation

As MHQ scores increased, the average number of days of work missed in the past month (Figure 5A) decreased systematically and was best fit by an exponential function, with R2=0.98. Those in the lowest MHQ score bin (−75 to −100) were unable to work or carry out their daily activities 15.0 (SD 11.3; SEM 0.9) days on average, whereas those who were employed (as opposed to studying, unable to work, unemployed, retired, or occupied with household work; 3306/174,618, 1.89%) were unable to work 9.3 (SD 10.0; SEM 1.6) days on average in the last month. In contrast, those who were in the highest MHQ bin (175-200) lost only an average of 0.2 (SD 1.6; SEM 0.1) days, whereas those who were employed lost an average of 0.2 (SD 0.54; SEM 0.07) days. Furthermore, as MHQ scores increased, the average number of days where people reported not being as productive as usual at work (presenteeism) or in their daily activities decreased linearly (Figure 5B; R2=0.98 for all respondents and employed respondents alone). Here, those in the lowest MHQ bin (−75 to −100) were not productive an average of 14.2 (SD 11.4; SEM 0.9) days, whereas those employed alone were not productive an average of 15.6 (SD 10.2; SEM 1.6) days. This decreased to an average of 3.2 (SD 9.5; SEM 0.8) and 2.2 (SD 8.5; SEM 1.1) days for all respondents and employed respondents, respectively, in the highest MHQ score bin (175-200). Figure 5C shows the total loss of life productivity as a function of the MHQ score considering both days of work missed and days that were less productive, assuming a range of 20% to 50% loss of productivity on less productive days. Altogether, those with the lowest MHQ scores had an overall reduction in life productivity of anywhere from 18 to 23 days per month on average. Although those with the highest MHQ scores did not often miss a day of work, even this group reported a few unproductive days per month. Thus, MHQ scores are a good representation of behavioral loss of function.

Figure 5. Relationship between Mental Health Quotient (MHQ) score and productivity. (A) The days unable to work in the past month decreased nonlinearly as the MHQ score increased (closed circles, exponential fit, R2=0.98). Employed people (open circles) with low MHQ scores missed fewer days of work or productive activity. (B) The days in the past month with reduced productivity (presenteeism) decreased linearly as the MHQ score increased (closed circles, exponential fit, R2=0.98). Employed people (open circles) with low MHQ scores had more days of presenteeism. (C) Total productivity loss for employed (dotted line) and all respondents together (solid line) as a function of the MHQ score (calculated as days missed + n * days with reduced productivity, where n is assumed to be a range between 0.2 [lower dotted or solid line] and 0.5 [upper dotted or solid line]).
View this figure

Principal Findings

In this study, we have demonstrated that the MHQ taken anonymously on the web has excellent sample reliability, internal consistency, and test-retest reliability; that the life impact scale used in the MHQ reflects a combination of both severity and frequency of symptoms; and that the MHQ score relates systematically to clinical burden in the population as well as loss of function from the perspective of days of work missed and loss of productivity. Specifically, the results showed that (1) the MHQ scores were highly similar and statistically indistinguishable between multiple randomly selected, demographically matched samples of respondents; (2) MHQ scores were correlated between retakes with an r=0.84; (3) the life impact rating scale of the MHQ was systematically related to both symptom frequency and severity (R2=0.99); (4) ratings on related elements were more correlated than unrelated elements; and, finally, (5) MHQ scores decreased systematically with clinical burden and productivity (both R2=0.98). Thus, the MHQ provides a valid and reliable estimation of population mental health and well-being that, in turn, reflects the clinical burden of mental health and the productive capacity of a population. Therefore, it is also well-suited to measure changes in the status of mental health and well-being of a population. In addition, the test-retest reliability establishes the MHQ as a useful tool for individuals to track their mental health and well-being trajectory over time.

MHQ and Clinical Burden

Surveillance of population mental health requires assessments that are reliable, valid, and accessible to the general population and that provide a comprehensive profile of mental health and well-being that has clinical and real-world relevance. Currently, many of the assessments used in epidemiological studies evaluate the prevalence of individual disorders rather than overall mental health and well-being. These tools typically consider mental illness as an all-or-nothing phenomenon, raising challenges relating to where the border between normal and disordered should lie [42-44] and leading to wide-ranging prevalence estimates that are dependent on the tool used and the thresholds considered as well as on geography and time period [18,45-51]. In addition, focusing only on individual disorders creates a siloed landscape of clinical burden that is at odds with the real-life heterogeneous and comorbid nature of symptomatic experiences and profiles [42,52-62]. Thus, generally, the aggregate burden of clinical-level mental health challenges beyond the domain of individual disorders is unknown in the general population. In this study, we have established the MHQ as a valid and reliable measure of mental health and well-being that can provide a view of overall mental distress and clinical burden. Rolled out at scale as it is currently being actioned as part of the Mental Health Million Project, the MHQ thus provides a solid foundation for the global surveillance of population mental health across different countries. This will help identify relevant risk factors to support the rollout of preventative strategies and the development of interventions or policies that could induce large-scale shifts in population well-being [8-10].

MHQ and Productivity

Over the past few decades, there has been mounting evidence supporting the relationship between mental health and well-being and productivity [19-25] as well as the resultant economic loss to society as a consequence of days lost and unproductive days (eg, presenteeism) [28-31]. With the increased prevalence of mental distress as a result of the COVID-19 pandemic [16-18] and increased levels of burnout in the population [26,27], there is a need to better understand the relationship between mental health and well-being and productivity in the general population. The systematic relationship observed between the spectrum of MHQ scores from Distressed to Thriving and productivity loss along with the general reliability and validity of the MHQ support its use as an assessment of the productive capacity of a population independent of any disorder classification. It also positions the MHQ as an important tool for companies to assess the mental health and well-being of their workforce, providing relevant metrics that can help them address challenges such as employee burnout and work-home imbalance [26,27], as well as for university student bodies, where young adults are disproportionally affected by mental health challenges [63-65]. This will allow and encourage organizations and institutions to be more strategic in their management of mental health and well-being.

Limitations and Future Directions

It is important that we acknowledge some limitations of these data and study. First, the validation of the life impact rating against symptom frequency and severity was performed for a single MHQ element (Feelings of sadness, distress, or hopelessness). However, it is possible that the correspondence between frequency or severity and life impact rating may differ from element to element. Furthermore, these results were used to select an appropriate threshold value for clinical significance [42] to determine clinical burden, indicating that a threshold of 8 was equivalent, on average, to experiencing the symptom 5 days per week. However, it could be the case that other threshold values may have been more appropriate for other elements.

Second, the mapping of MHQ elements to DSM-5 diagnostic criteria was constrained by the presence of broad or imperfect matches for certain symptoms pertaining to OCD and bipolar disorder that could have affected the accuracy of the mapping [42]. For example, for bipolar disorder, symptoms denoting extreme versions of positive assets (eg, grandiosity and decreased need for sleep) were not fully articulated within the MHQ, whereas, for OCD, the MHQ elements were broader (eg, obsessive thoughts were incorporated within a general element reflecting strange, unwanted, and obsessive thoughts). Furthermore, a specific criterion of symptom timing was not included as this is not included in the MHQ, which assesses an individual’s current perception.

Third, in the future, it will be important to compare the MHQ outcomes with more commonly used assessments (eg, mapping against the 9-item Patient Health Questionnaire [66] and the 7-item Generalized Anxiety Disorder scale [67]) to determine the alignment between DSM-5–mapped MHQ symptom profiles for depression and anxiety and the scores from these 2 questionnaires, respectively. Triangulating data arising from the Mental Health Million Project against other external data metrics that provide insight into clinical burden (eg, quality-adjusted life years or disability-adjusted life years) will also be important.

Altogether, the MHQ supports a valid and reliable monitoring of population mental health and well-being. As the MHQ continues to underpin large-scale initiatives such as the Mental Health Million Project, it will provide deeper insights into social determinants and the societal impact of changes in mental health and well-being. These insights can, in turn, enable preventative strategies for better management of global mental health and well-being.

Acknowledgments

This work was supported by internal funding from Sapien Labs.

Authors' Contributions

JJN and TCT conceptualized, designed, and led the study. JJN and VP performed the analysis. JJN and TCT interpreted the data and drafted the manuscript. JJN created the figures. TCT and JJN made critical revisions. JJN, TCT, and VP approved the final version of the manuscript and agreed to be accountable for all aspects of the work.

Conflicts of Interest

TCT received a grant award from the National Institute of Mental Health to develop a commercial version of the Mental Health Quotient tool referenced herein.

  1. Promoting mental health: concepts, emerging evidence, practice: a report of the World Health Organization, Department of Mental Health and Substance Abuse in collaboration with the Victorian Health Promotion Foundation and the University of Melbourne. World Health Organization. 2005.   URL: https://www.who.int/mental_health/evidence/en/promoting_mhh.pdf [accessed 2021-09-01]
  2. Cloninger CR. The science of well being: an integrated approach to mental health and its disorders. Psychiatr Danub 2006;18(3-4):218-224. [Medline]
  3. Slade M. Mental illness and well-being: the central importance of positive psychology and recovery approaches. BMC Health Serv Res 2010;10:26 [FREE Full text] [CrossRef] [Medline]
  4. Bohlmeijer E, Westerhof G. The model for sustainable mental health: future directions for integrating positive psychology into mental health care. Front Psychol 2021;12:747999 [FREE Full text] [CrossRef] [Medline]
  5. Schotanus-Dijkstra M, Ten Have M, Lamers SM, de Graaf R, Bohlmeijer ET. The longitudinal relationship between flourishing mental health and incident mood, anxiety and substance use disorders. Eur J Public Health 2017;27(3):563-568. [CrossRef] [Medline]
  6. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th edition. Washington, D.C: American Psychiatric Association; 2013.
  7. International statistical classification of diseases and related health problems. 11th revision. World Health Organization. 2018.   URL: https://www.who.int/standards/classifications/classification-of-diseases [accessed 2021-09-01]
  8. Purtle J, Nelson KL, Counts NZ, Yudell M. Population-based approaches to mental health: history, strategies, and evidence. Annu Rev Public Health 2020;41:201-221 [FREE Full text] [CrossRef] [Medline]
  9. Fuhrer R, Keyes KM. Population mental health in the 21st century: time to act. Am J Public Health 2019;109(S3):S152-S153. [CrossRef] [Medline]
  10. Huppert FA. A new approach to reducing disorder and improving well-being. Perspect Psychol Sci 2009;4(1):108-111. [CrossRef] [Medline]
  11. Dattani S, Ritchie H, Roser M. Mental health. Our World In Data. 2021.   URL: https://ourworldindata.org/mental-health [accessed 2021-09-01]
  12. Friedrich MJ. Depression is the leading cause of disability around the world. JAMA 2017;317(15):1517. [CrossRef] [Medline]
  13. Suicide worldwide in 2019: global health estimates. World Health Organization. 2021.   URL: https://www.who.int/publications/i/item/9789240026643 [accessed 2021-09-01]
  14. Orsolini L, Latini R, Pompili M, Serafini G, Volpe U, Vellante F, et al. Understanding the complex of suicide in depression: from research to clinics. Psychiatry Investig 2020;17(3):207-221 [FREE Full text] [CrossRef] [Medline]
  15. De Berardis D, Fornaro M, Valchera A, Cavuto M, Perna G, Di Nicola M, et al. Eradicating suicide at its roots: preclinical bases and clinical evidence of the efficacy of ketamine in the treatment of suicidal behaviors. Int J Mol Sci 2018;19(10):2888 [FREE Full text] [CrossRef] [Medline]
  16. Pierce M, Hope H, Ford T, Hatch S, Hotopf M, John A, et al. Mental health before and during the COVID-19 pandemic: a longitudinal probability sample survey of the UK population. Lancet Psychiatry 2020;7(10):883-892 [FREE Full text] [CrossRef] [Medline]
  17. Serafini G, Parmigiani B, Amerio A, Aguglia A, Sher L, Amore M. The psychological impact of COVID-19 on the mental health in the general population. QJM 2020;113(8):531-537 [FREE Full text] [CrossRef] [Medline]
  18. Xiong J, Lipsitz O, Nasri F, Lui LM, Gill H, Phan L, et al. Impact of COVID-19 pandemic on mental health in the general population: a systematic review. J Affect Disord 2020;277:55-64 [FREE Full text] [CrossRef] [Medline]
  19. Kessler RC, Frank RG. The impact of psychiatric disorders on work loss days. Psychol Med 1997;27(4):861-873. [CrossRef] [Medline]
  20. Alonso J, Petukhova M, Vilagut G, Chatterji S, Heeringa S, Üstün TB, et al. Days out of role due to common physical and mental conditions: results from the WHO World Mental Health surveys. Mol Psychiatry 2011;16(12):1234-1246 [FREE Full text] [CrossRef] [Medline]
  21. Burton WN, Schultz AB, Chen C, Edington DW. The association of worker productivity and mental health: a review of the literature. Int J Workplace Health Manag 2008;1(2):78-94. [CrossRef]
  22. Hemp P. Presenteeism: at work--but out of it. Harv Bus Rev 2004;82(10):49-58. [Medline]
  23. Lim D, Sanderson K, Andrews G. Lost productivity among full-time workers with mental disorders. J Ment Health Policy Econ 2000;3(3):139-146. [CrossRef] [Medline]
  24. Evans-Lacko S, Knapp M. Global patterns of workplace productivity for people with depression: absenteeism and presenteeism costs across eight diverse countries. Soc Psychiatry Psychiatr Epidemiol 2016;51(11):1525-1537 [FREE Full text] [CrossRef] [Medline]
  25. Bubonya M, Cobb-Clark D, Wooden M. Mental health and productivity at work: does what you do matter? Labour Econ 2017;46(C):150-165. [CrossRef]
  26. Threlkeld K. Employee burnout report: COVID-19’s impact and 3 strategies to curb it. Indeed. 2021.   URL: https://www.indeed.com/lead/preventing-employee-burnout-report [accessed 2022-01-12]
  27. Employee burnout is ubiquitous, alarming—and still underreported. McKinsey. 2021.   URL: https:/​/www.​mckinsey.com/​featured-insights/​coronavirus-leading-through-the-crisis/​charting-the-path-to-the-next-normal/​employee-burnout-is-ubiquitous-alarming-and-still-underreported [accessed 2021-01-12]
  28. Marcotte DE, Wilcox-Gök V. Estimating the employment and earnings costs of mental illness: recent developments in the United States. Soc Sci Med 2001;53(1):21-27. [CrossRef] [Medline]
  29. Stewart WF, Ricci JA, Chee E, Hahn SR, Morganstein D. Cost of lost productive work time among US workers with depression. JAMA 2003;289(23):3135-3144. [CrossRef] [Medline]
  30. Trautmann S, Rehm J, Wittchen HU. The economic costs of mental disorders: do our societies react appropriately to the burden of mental disorders? EMBO Rep 2016;17(9):1245-1249 [FREE Full text] [CrossRef] [Medline]
  31. Whiteford H. The productivity commission inquiry into mental health. Aust N Z J Psychiatry 2021:48674211031159. [CrossRef] [Medline]
  32. Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, et al. Scaling-up treatment of depression and anxiety: a global return on investment analysis. Lancet Psychiatry 2016;3(5):415-424 [FREE Full text] [CrossRef] [Medline]
  33. Newson JJ, Thiagarajan TC. Assessment of population well-being with the mental health quotient (MHQ): development and usability study. JMIR Ment Health 2020;7(7):e17935 [FREE Full text] [CrossRef] [Medline]
  34. Newson JJ, Hunter D, Thiagarajan TC. The heterogeneity of mental health assessment. Front Psychiatry 2020;11:76 [FREE Full text] [CrossRef] [Medline]
  35. First MB, Williams JB, Karg RS, Spitzer RL. Structured clinical interview for DSM-5 disorders, clinician version (SCID-5-CV). Arlington, VA: American Psychiatric Association; 2016.
  36. Cuthbert BN. The role of RDoC in future classification of mental disorders
. Dialogues Clin Neurosci 2020;22(1):81-85 [FREE Full text] [CrossRef] [Medline]
  37. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry 2010;167(7):748-751. [CrossRef] [Medline]
  38. Insel TR. The NIMH research domain criteria (RDoC) project: precision medicine for psychiatry. Am J Psychiatry 2014;171(4):395-397. [CrossRef] [Medline]
  39. Newson JJ, Thiagarajan T. Dynamic dataset of global population mental wellbeing. 2021.   URL: https:/​/sapienlabs.​org/​wp-content/​uploads/​2021/​06/​NewsonThiagarajan_2021_Dynamic-Dataset-of-Global-Population-Mental-Wellbeing.​pdf [accessed 2021-09-01]
  40. Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas 1977;1(3):385-401. [CrossRef]
  41. Eaton WM, Smith C, Ybarra M, Muntaner C, Tien A. Center for Epidemiologic Studies Depression Scale: review and revision (CESD and CESD-R). In: Maruish ME, editor. The use of psychological testing for treatment planning and outcomes assessment: volume 3: instruments for adults. 3rd edition. Mahwah, NJ: Lawrence Erlbaum Associates; 2004:363-377.
  42. Newson JJ, Pastukh V, Thiagarajan TC. Poor separation of clinical symptom profiles by DSM-5 disorder criteria. Front Psychiatry 2021;12:775762 [FREE Full text] [CrossRef] [Medline]
  43. Rössler W. What is normal? The impact of psychiatric classification on mental health practice and research. Front Public Health 2013;1:68 [FREE Full text] [CrossRef] [Medline]
  44. Wakefield JC. Diagnostic issues and controversies in DSM-5: return of the false positives problem. Annu Rev Clin Psychol 2016;12:105-132. [CrossRef] [Medline]
  45. Wu T, Jia X, Shi H, Niu J, Yin X, Xie J, et al. Prevalence of mental health problems during the COVID-19 pandemic: a systematic review and meta-analysis. J Affect Disord 2021;281:91-98 [FREE Full text] [CrossRef] [Medline]
  46. Cénat JM, Blais-Rochette C, Kokou-Kpolou CK, Noorishad PG, Mukunzi JN, McIntee SE, et al. Prevalence of symptoms of depression, anxiety, insomnia, posttraumatic stress disorder, and psychological distress among populations affected by the COVID-19 pandemic: a systematic review and meta-analysis. Psychiatry Res 2021;295:113599 [FREE Full text] [CrossRef] [Medline]
  47. Baxter AJ, Scott KM, Vos T, Whiteford HA. Global prevalence of anxiety disorders: a systematic review and meta-regression. Psychol Med 2013;43(5):897-910. [CrossRef] [Medline]
  48. Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980-2013. Int J Epidemiol 2014;43(2):476-493 [FREE Full text] [CrossRef] [Medline]
  49. Lim GY, Tam WW, Lu Y, Ho CS, Zhang MW, Ho RC. Prevalence of depression in the community from 30 countries between 1994 and 2014. Sci Rep 2018;8(1):2861 [FREE Full text] [CrossRef] [Medline]
  50. McGrath J, Saha S, Welham J, El Saadi O, MacCauley C, Chant D. A systematic review of the incidence of schizophrenia: the distribution of rates and the influence of sex, urbanicity, migrant status and methodology. BMC Med 2004;2:13 [FREE Full text] [CrossRef] [Medline]
  51. Atwoli L, Stein DJ, Koenen KC, McLaughlin KA. Epidemiology of posttraumatic stress disorder: prevalence, correlates and consequences. Curr Opin Psychiatry 2015;28(4):307-311 [FREE Full text] [CrossRef] [Medline]
  52. Angold A, Costello EJ, Erkanli A. Comorbidity. J Child Psychol Psychiatry 1999;40(1):57-87. [Medline]
  53. Kessler RC, Nelson CB, McGonagle KA, Liu J, Swartz M, Blazer DG. Comorbidity of DSM-III-R major depressive disorder in the general population: results from the US National Comorbidity Survey. Br J Psychiatry Suppl 1996(30):17-30. [Medline]
  54. Maj M. "Psychiatric comorbidity": an artefact of current diagnostic systems? Br J Psychiatry 2005;186:182-184. [CrossRef] [Medline]
  55. Feczko E, Miranda-Dominguez O, Marr M, Graham AM, Nigg JT, Fair DA. The heterogeneity problem: approaches to identify psychiatric subtypes. Trends Cogn Sci 2019;23(7):584-601 [FREE Full text] [CrossRef] [Medline]
  56. Fried EI, Nesse RM. Depression is not a consistent syndrome: an investigation of unique symptom patterns in the STAR*D study. J Affect Disord 2015;172:96-102 [FREE Full text] [CrossRef] [Medline]
  57. Caspi A, Houts RM, Ambler A, Danese A, Elliott ML, Hariri A, et al. Longitudinal assessment of mental health disorders and comorbidities across 4 decades among participants in the Dunedin birth cohort study. JAMA Netw Open 2020;3(4):e203221 [FREE Full text] [CrossRef] [Medline]
  58. McElroy E, Patalay P. In search of disorders: internalizing symptom networks in a large clinical sample. J Child Psychol Psychiatry 2019;60(8):897-906 [FREE Full text] [CrossRef] [Medline]
  59. Plana-Ripoll O, Pedersen CB, Holtz Y, Benros ME, Dalsgaard S, de Jonge P, et al. Exploring comorbidity within mental disorders among a Danish national population. JAMA Psychiatry 2019;76(3):259-270 [FREE Full text] [CrossRef] [Medline]
  60. Buch AM, Liston C. Dissecting diagnostic heterogeneity in depression by integrating neuroimaging and genetics. Neuropsychopharmacology 2021;46(1):156-175 [FREE Full text] [CrossRef] [Medline]
  61. Luo Y, Weibman D, Halperin JM, Li X. A review of heterogeneity in attention deficit/hyperactivity disorder (ADHD). Front Hum Neurosci 2019;13:42 [FREE Full text] [CrossRef] [Medline]
  62. Masi A, DeMayo MM, Glozier N, Guastella AJ. An overview of autism spectrum disorder, heterogeneity and treatment options. Neurosci Bull 2017;33(2):183-193 [FREE Full text] [CrossRef] [Medline]
  63. Kessler RC, Amminger GP, Aguilar-Gaxiola S, Alonso J, Lee S, Ustün TB. Age of onset of mental disorders: a review of recent literature. Curr Opin Psychiatry 2007;20(4):359-364 [FREE Full text] [CrossRef] [Medline]
  64. Twenge JM, Cooper AB, Joiner TE, Duffy ME, Binau SG. Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005-2017. J Abnorm Psychol 2019;128(3):185-199. [CrossRef] [Medline]
  65. Newson JJ, Pastukh V, Sukhoi O, Taylor J, Thiagarajan TC. Mental state of the world 2020. Mental Health Million Project, Sapien Labs. 2021.   URL: https://sapienlabs.org/wp-content/uploads/2021/03/Mental-State-of-the-World-Report-2020-1.pdf [accessed 2021-09-01]
  66. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient health questionnaire. JAMA 1999;282(18):1737-1744. [CrossRef] [Medline]
  67. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166(10):1092-1097. [CrossRef] [Medline]


DSM-5: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition
MHQ: Mental Health Quotient
OCD: obsessive-compulsive disorder
SEM: standard error of measurement


Edited by J Torous; submitted 07.10.21; peer-reviewed by G Serafini, S Six, L Ospina-Pinillos; comments to author 29.11.21; revised version received 14.01.22; accepted 24.01.22; published 20.04.22

Copyright

©Jennifer Jane Newson, Vladyslav Pastukh, Tara C Thiagarajan. Originally published in JMIR Mental Health (https://mental.jmir.org), 20.04.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.