Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 01.06.20 in Vol 7, No 6 (2020): June

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/19347, first published Apr 14, 2020.

This paper is in the following e-collection/theme issue:

    Original Paper

    Flattening the Mental Health Curve: COVID-19 Stay-at-Home Orders Are Associated With Alterations in Mental Health Search Behavior in the United States

    1Center for Technology and Behavioral Health, Geisel School of Medicine, Dartmouth College, Lebanon, NH, United States

    2Department of Biomedical Data Science, Geisel School of Medicine, Dartmouth College, Lebanon, NH, United States

    3Department of Psychiatry, Geisel School of Medicine, Dartmouth College, Hanover, NH, United States

    4Quantitative Biomedical Sciences Program, Dartmouth College, Hanover, NH, United States

    5Dartmouth-Hitchcock Medical Center, Lebanon, NH, United States

    6The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine, Dartmouth College, Hanover, NH, United States

    Corresponding Author:

    Nicholas C Jacobson, PhD

    Center for Technology and Behavioral Health, Geisel School of Medicine, Dartmouth College

    46 Centerra Parkway

    Suite 300, Office #333S

    Lebanon, NH, 03766

    United States

    Phone: 1 6036467037

    Email: Nicholas.C.Jacobson@dartmouth.edu


    ABSTRACT

    Background: The coronavirus disease (COVID-19) has led to dramatic changes worldwide in people’s everyday lives. To combat the pandemic, many governments have implemented social distancing, quarantine, and stay-at-home orders. There is limited research on the impact of such extreme measures on mental health.

    Objective: The goal of this study was to examine whether stay-at-home orders produced differential changes in mental health symptoms using internet search queries on a national scale.

    Methods: In the United States, individual states vary in their adoption of measures to reduce the spread of COVID-19; as of March 23, 2020, 11 of the 50 states had issued stay-at-home orders. The staggered rollout of stay-at-home measures across the United States allows us to investigate whether these measures impact mental health by exploring variations in mental health search queries across the states. This paper examines the changes in mental health search queries on Google between March 16-23, 2020, across each state and Washington, DC. Specifically, this paper examines differential changes in mental health searches based on patterns of search activity following issuance of stay-at-home orders in these states compared to all other states. The participants were all the people who searched mental health terms in Google between March 16-23. Between March 16-23, 11 states underwent stay-at-home orders to prevent the transmission of COVID-19. Outcomes included search terms measuring anxiety, depression, obsessive-compulsive, negative thoughts, irritability, fatigue, anhedonia, concentration, insomnia, and suicidal ideation.

    Results: Analyzing over 10 million search queries using generalized additive mixed models, the results suggested that the implementation of stay-at-home orders are associated with a significant flattening of the curve for searches for suicidal ideation, anxiety, negative thoughts, and sleep disturbances, with the most prominent flattening associated with suicidal ideation and anxiety.

    Conclusions: These results suggest that, despite decreased social contact, mental health search queries increased rapidly prior to the issuance of stay-at-home orders, and these changes dissipated following the announcement and enactment of these orders. Although more research is needed to examine sustained effects, these results suggest mental health symptoms were associated with an immediate leveling off following the issuance of stay-at-home orders.

    JMIR Ment Health 2020;7(6):e19347

    doi:10.2196/19347

    KEYWORDS



    Introduction

    The 2019 novel coronavirus disease (COVID-19), a symptomatically broad disease transmitted by human-to-human droplets or direct contact, has been declared by the World Health Organization to be an international crisis [1,2]. The rapid, largely uncontrolled spread of COVID-19 has impacted every facet of American life, calling for dramatic shifts in the social and professional behavior of nearly 327 million people.

    In the early phase of an outbreak, reducing physical distance and interactions between individuals in a population is an effective way of stopping or limiting disease spread [3]. In early February 2020, the Centers for Disease Control and Prevention recommended social distancing across the United States as a strategy to prevent the rapid spread of COVID-19 and subsequent overburdening of the health care system [4]. Despite these and other measures, there has been an exponential spread of COVID-19. During this time, a simulation model suggested that only the most stringent social distancing interventions would be effective at reducing the spread [5]. In the absence of federal guidance, individual states have enacted varying degrees of these recommendations. Such enactment has manifested in the closure of schools and nonessential businesses and, more recently, in the issuance of shelter-in-place notices and stay-at-home orders in several states (hereafter referred to as stay-at-home orders). Stay-at-home orders reflect the most disruptive measure, resulting in a mass quarantine that restricts individuals to their place of residence except for essential travel. As of March 23, 2020, the following 11 states have issued these orders: California, Connecticut, Delaware, Illinois, Louisiana, Massachusetts, Michigan, New Jersey, New York, Ohio, and West Virginia. Of these 11 states to announce stay-at-home orders due to COVID-19, the stay-at-home orders for California, New Jersey, New York, Illinois, and Washington had taken effect as of March 23.

    Although social distancing measures are necessary to protect physical health, less is known about the impact of such measures on mental health. A rapid review of the psychological impact of quarantine found that such measures were associated with high levels of psychological distress including posttraumatic stress symptoms, confusion and anger, and high prevalence of low mood and irritability [6]. The authors note that a lack of clear communication from governments to their citizens may heighten uncertainty, which could be a key driver of distress. This suggests that clear governmental action may reduce psychological distress. Nevertheless, none of the included studies in the rapid review assessed psychological distress immediately pre- and postenactment of a quarantine. Consequently, longitudinal data regarding the potential changes of mental health symptoms immediately before and after implementations of stay-at-home orders is needed.

    To date, many studies across the world have employed internet search trends to study epidemiological changes in mental health [7-14]. Nearly all of these published studies have found an association between internet search behavior and real-world mental health data, although the magnitude of this effect has differed across studies. Specifically, studies have found robust associations between suicide search queries and completed suicide rates [7,10,12] with extremely high interrater reliability (intraclass correlation coefficient [ICC]=0.98) between searches using the term “suicides” and observed suicide rates [15]. Many of these studies have also found links between mental health search data and observed rates of life stressors, such as divorce and unemployment [7,10,12]. These findings provide support for using mental health queries as effective proxies for real world mental health outcomes. They in effect “bridge the gap” between internet search trends and epidemiological data, allowing for important public health application to this large corpus of readily available and easily accessible data.

    This paper evaluated the impact of stay-at-home orders on mental health search queries between March 16-23, 2020. This work used Google Trends to quantify changes in search behavior in the 50 states within the United States as well as the District of Columbia (Washington, DC) with the goal of better understanding the acute mental health impact of stay-at-home orders amid COVID-19. Specifically, we sought to determine whether stay-at-home orders were associated with increased affective symptoms as might be suggested by theories related to potential impacts of prolonged social isolation, or, in contrast, whether there might be improved mental health from clear government action rather than continuing to live in a state of uncertainty caused by government inaction [16,17]. We investigated the following research questions:

    1. Would stay-at-home orders significantly alter the trajectory of mental health search queries across time between March 16-23, 2020, in the United States, compared to states that had not yet enacted stay-at-home orders?
    2. Would the effects of stay-at-home orders on search queries be isolated to specific symptom domains (eg, anxiety, depression, suicide) or be consistent across symptom domains?
    3. Would the results of stay-at-home orders uniquely impact search queries related to mental health symptoms or be consistent with queries on physical health symptoms (both related and unrelated to COVID-19)?

    Methods

    Google Trends

    Google is the leading search engine, retaining a dominant market share of all search traffic within the United States and worldwide. Google Trends uses large-scale search volume and allows users to download information about search volume for a given time and place. Google Trends is posted publicly, and data can be downloaded directly through Google’s web portals or through freely available software [18]. Google Trends normalizes search data per search locale (in this case, per state). Normalization occurs using the following process: each data point is divided by the total search volume of the geography and time range it represents to compare relative popularity; the results are then scaled on a range of 0-100 based on a topic’s proportion to all searches on all topics. Such normalization has the effect of controlling for the total volume of internet use across time. We used these normalized values as the outcome; a positive upward trajectory of a given search term means that term increased in relative frequency compared to other terms. The data was obtained for each hour between 11 PM on March 16, 2020, and 10 PM on March 23, 2020, EST. This period was chosen because we wanted to investigate the immediate time trends before and after issuance and implementation of stay-at-home orders. Orders were announced and implemented for 11 states within this period of time. We did not examine data prior to March 16 because Google Trends only reports hourly data for up to 7 days prior. See Figure 1 and Table 1 for a study timeline.

    Figure 1. Timeline of stay-at-home order policy announcement (red) and implementation (blue).
    View this figure
    Table 1. States with stay-at-home orders as of the date of data collection.a
    View this table

    Search Terms

    Mental Health

    We used the following search terms to examine common mental health symptoms: “anxiety,” “depression,” “OCD” (obsessive-compulsive disorder), “hopeless,” “angry,” “afraid,” “apathy,” “worthless,” “worried,” “restless,” “irritable,” “tense,” “scattered,” “tired,” “avoiding,” “procrastinate,” “insomnia,” “suicidal,” and “suicide.” These mental health terms were validated by prior research on mental health using Google Trends [20]. We also adapted other terms that measure Diagnostic and Statistical Manual of Mental Disorders, 5th edition affective disorder symptoms based on prior research assessing rapid affective symptom changes, including single-items to assess anxiety (both in terms of subjective thoughts, including fear, tense, and restless, as well as avoidance behaviors, including avoiding and procrastination), negative thoughts (hopeless, worried), irritability (anger, irritable), fatigue (tired), anhedonia (apathy), diminished ability to think or concentrate (scattered), disturbed sleep (insomnia), and suicidal ideation (suicidal, suicide) [21,22].

    Physical Health Terms Unrelated to Known COVID-19 Symptoms

    To determine whether changes in search trends were specific to searches related to mental health, we contrasted mental health symptoms to physical health symptoms that have no current known association with COVID-19. The physical health searches included: “abrasion,” “allergic,” “angina,” “apnea,” “bleeding,” “blister,” “bruising,” “conjunctivitis,” “constipation,” “discharge,” “earache,” “flatulence,” “fracture,” “hemorrhage,” “incontinence,” “inflammation,” “itching,” “lesions,” “rash,” “spasms,” “swelling,” and “syncope.” These terms were used as prespecified falsification hypotheses to provide useful control conditions to further ensure that conclusions drawn were not an artifact of the methodology alone [23].

    Physical Health Terms Related to Known COVID-19 Symptoms

    We also examined search queries for COVID-19 physical health symptoms to determine whether the changes in mental health symptoms related to stay-at-home orders were distinct from changes in COVID-19 physical health symptoms. Consequently, we also conducted searches on physical health terms among known symptoms of COVID-19 including “bloating,” “blurry,” “congestion,” “cough,” “coughing,” “croup,” “diarrhea,” “dizzy,” “fainting,” “fever,” “pain,” “sneezing,” “strep,” “stuffy,” and “vomiting.”

    Analyses

    The goal of the present analyses was to investigate change in search trends across time. Consequently, it was important to account for potentially nonlinear trends across time and account for interdependence of observations. As such, the current analyses used generalized additive mixed models (GAMMs). GAMMs combine the features of models that allow predictors to have a highly flexible relationship with the outcome so long as the relationship has a smooth functional form (ie, is continuous and differentiable) such as enabled by thin plate regression splines and multilevel models that account for the lack of statistical independence in the observations made on the same units across time (see [24] for a review). Splines within GAMMs were used as they allow the data to take on any smooth functional form, but the models only allow nonlinearity in the predictor-outcome relationship if nonlinearity would provide the best fit to the data. We chose these methods over more popular but less flexible approaches (eg, higher-order polynomial transformations) because thin plate regression splines better address the issue of number and location of the knots. In particular, we made use of thin plate regression splines that use an eigenvalue decomposition to pick the basis coefficients that can explain the greatest variance. This is advantageous as it does not require a researcher to choose knot locations, thereby reducing subjectivity in modeling and otherwise having optimal bases [25], and better accommodating a higher number of predictors [26]. Notably, thin plate regression splines have been shown to not overfit the data in the way that other spline methods may [27].

    We used the following model for each respective outcome:

    Outcomei,j ~ s1(Timei,j) + s2(TimeSincePolicyi,j) * StayAtHomePolicyi,j + u1,i + s3,i(Timei,j) (1)

    Outcome represents each respective search term for state i at time j; Time represents the number of hours since 11 PM on March 16, 2020; TimeSincePolicy represents a time difference variable between the observed time and when the implementation of the stay-at-home policy began (defined to be 0 if the policy had not been enacted); StayAtHomePolicy represents a dummy variable representing 0 for when the policy was not in place and 1 when the policy was in place. The terms represent a smooth, thin plate regression spline, wherein the term is allowed to have a linear or nonlinear relationship with the outcome. However, nonlinearity is penalized such that data will only become nonlinear if it results in a substantially greater model fit. Note that u1 and s3 represent random effects, with u1 representing a vector of random intercepts of state and s3 representing a vector of random smooth slopes of the time trend for each state (thus allowing a nonlinear random effect to account for changes in each state). The default k (10) in mgcv value was chosen for all analyses to balance smooth fitting with computational time (as allowing a smooth spline for each random effect is computationally expensive) [28]. The latter seeks to estimate the search trajectory trend that would have occurred for that state in the absence of it implementing a stay-at-home policy. Thus, the primary term of interest is s2, which estimates the effects of the stay-at-home policy intervention as a deviation from the state-specific counterfactual trend that would have occurred had there not been a stay-at-home order issued.

    Ethics

    This paper was not considered human subjects research because it used publicly available data, and as such was exempt from human subject approval.


    Results

    Volume of Mental Health Search Queries Across the United States

    We estimated the total volume of search queries related to mental health by comparing and using a reported value of search terms from the Google Trends trending terms in the United States for a given day, multiplying the normalized value of each mental health term, and then dividing by the comparison term with the known absolute increase value noted across the United States. Based on this method, there were a total of 9,717,876 total searches on mental health from March 18 through March 23, 2020, (we only calculated the total search volume beginning on March 18 rather than March 16 as we were not aware of the absolute search volume terms being reported until March 25; the hour-to-hour measures reported in Google Trends only goes back a total of 7 days). See Figure 2 for a breakdown of searches per search term. Based on the average number of searches per day between March 18-23, we estimated that approximately 13 million mental health search queries were conducted between March 16-23.

    Figure 2. This plot depicts the estimated total volume of searches between March 18-23, 2020, for each search term. OCD: obsessive-compulsive disorder.
    View this figure

    Mental Health Symptom Searches and Stay-at-Home Orders

    We found a significant association between queries involving 14 (of 19 total) mental health search terms and implementation of stay-at-home orders (see Table 2 for a summary). Our results showed nonlinear changes in 12 of these mental health symptoms over time associated with the announcement of stay-at-home orders. In particular, “afraid,” “anxiety,” “apathy,” “avoiding,” “hopeless,” “insomnia,” “irritable,” “procrastinate,” “restless,” “suicide,” “suicidality,” and “worthless” each showed rapid increases in search queries several days prior to the official enactment of the stay-at-home policies (see Figures 3 and 4 and Supplementary Figures A1-A3 in Multimedia Appendix 1). Moreover, each of these terms was associated with a leveling-off effect approximately 2 days prior to the implementation of the stay-at-home policy (ie, approximately the same time as the stay-at-home policy was announced). For all but 1 of these 12 symptoms (ie, “avoiding”), the leveling off remained consistent through the last day of data collection. Note that the terms associated with the strongest changes in these trends were “afraid,” “suicide,” “anxiety,” and “suicidal” (see Figure 2).

    Table 2. Changes in search behavior related to stay-at-home orders.a
    View this table
    Figure 3. This plot depicts the changes in search patterns relative to the orders going into effect. Negative time of stay-at-home values reflects the time before the stay-at-home orders go into effect, and positive values reflect the time following the order going into effect. Note that in this plot, the predictions are not normalized to show the magnitude of the effects. The centering performed here subtracts the actual value at 0 from the estimates of this term. The escalation phase designates approximately 4 days prior to the stay-at-home orders going into effect. The interruption phase spans between approximately 4 days and 0 days before the orders go into effect, and the new normal phase represents the times the 5 days after orders go into effect.
    View this figure
    Figure 4. This plot depicts the standardized changes in search patterns relative to the orders going into effect. Values are normalized to reflect the relative change in these searches across time. Negative time of stay-at-home values reflect the time before the stay-at-home orders go into effect, and positive values reflect the time following the order going into effect. Values are standardized to show the relative pattern of the effect. The escalation phase designates approximately 4 days prior to the stay-at-home orders going into effect. The interruption phase spans between approximately 4 days and 0 days before the orders go into effect, and the new normal phase represents the times the 5 days after orders go into effect.
    View this figure

    Non-COVID-19 Physical Health Symptom Searches and Stay-at-Home Orders

    In contrast to the mental health terms (where 14 of the 19 symptoms evidenced significant changes related to stay-at-home orders, 12 of which were associated with significant leveling off), only 7 of the 22 nonrelated physical health symptoms demonstrated changes over time related to the timing of stay-at-home orders (see Supplemental Table A1 in Multimedia Appendix 2). Of the 7, only 2 of these symptoms demonstrated a leveling-off effect (ie, flatulence and discharge). In addition, the effects of these symptoms were notably later and less extreme than those of the mental health symptoms (see Supplementary Figures A4 and A5 in Multimedia Appendix 1). Taken together, this suggests that the extreme leveling off associated with mental health symptoms was not replicated in the non-COVID-19 physical health symptoms.

    COVID-19 Physical Health Symptom Searches and Stay-at-Home Orders

    Likewise, in contrast to mental health symptoms, only 6 of the 15 COVID-19 symptoms were significantly related to stay-at-home orders (see Supplemental Table A2 in Multimedia Appendix 2). Of these 6 symptoms, only 2 symptoms (fever and pain) were increasing before the initiation of the stay-at-home order, and they leveled off after the order was officially in place (see Supplementary Figures A6 and A7 in Multimedia Appendix 1). Note that the findings of the COVID-19 physical health symptoms continue to suggest that stay-at-home orders appear to more saliently impact mental health compared to COVID-19–related physical health.


    Discussion

    This paper investigates the consequences of stay-at-home policies to prevent the spread of COVID-19 on changes in mental health symptom searches. The analysis is based on more than 10 million mental health Google searches within the United States. The results suggested that there were large shifts in mental health symptom searches linked to stay-at-home orders between the period of March 16-23, 2020. Particularly, the results indicated that topics related to anxiety (ie, fear, anxiety, avoiding, restlessness, procrastination), negative thoughts about oneself and the future (ie, hopelessness and worthless), sleep disturbances (ie, insomnia), and suicidal ideation (ie, suicide, suicidal) were each associated with dramatic increases prior to stay-at-home orders being announced with subsequent and considerable leveling off during these periods, approximately the same time as the stay-at-home orders were announced and enacted.

    Notably, with few exceptions, these patterns were relatively unique to mental health search behaviors and not characteristic of other physical health conditions more broadly, thereby providing evidence to suggest that the largest changes were associated with mental health symptoms rather than COVID-19 symptoms across this short time frame. The consistency of these findings and their applicability to mental health domains highlights that they are likely not an artifact of the study methodology but rather reflect differences in search behaviors resulting from stay-at-home policy announcements and implementations.

    Suicidality was one of the symptoms most greatly impacted by these stay-at-home orders. Importantly, prior studies have found robust associations between local Google suicide search queries and completed suicide rates [7,12], with extremely high interrater reliability (ICC=0.98) between searches for suicides and observed suicide rates [15]. Moreover, the early observed increase in “suicide” searches corroborates media reports that suicide calls have been increasing during the COVID-19 outbreak in the United States [29]. Thus, the current findings may point toward emerging evidence that stay-at-home orders may have immediately, at least temporarily, mitigated suicide risk. Although this research may point toward potential relationships between stay-at-home orders and actual suicide rates, research regarding observed suicide rates should be conducted to determine whether these patterns hold when examining actual suicide rates before any conclusions can be reached.

    The current observation of rapid rises in anxiety symptom searches before the announcement of stay-at-home orders mirrors the trends of high prevalence rates of generalized anxiety disorder during the COVID-19 outbreak in China [30]. Nevertheless, the flattening of anxiety symptom searches suggest that the announcement and enactment of stay-at-home orders may have had an immediate effect on altering this rising trajectory, preventing further fears and unrest given known governmental action. Notably, the trends observed affirm prior research in China, which suggests that more clarity and action regarding precautionary COVID-19 measures leads to greater calming and lower levels of anxiety [16]. This highlights the importance of reducing uncertainty to foster mental health following disasters [24]. In turn, the societal impacts of governmental interventions in ameliorating the increase of anxiety during the COVID-19 pandemic is corroborated.

    Notably, although these stay-at-home orders may have helped to flatten the staggering curvilinear increase in mental health symptom searches prior to the enactment of stay-at-home orders, the trend was not one in which these symptoms significantly decreased. Rather, the search rates remained relatively stable across the first days of a stay-at-home order going into effect. Certainly, more research will be needed to examine the long-term effects of these stay-at-home orders, especially given that longer quarantine orders have been associated with increased posttraumatic stress symptoms in prior work [31].

    This study has many strengths. First, this study is the first known study to investigate the immediate impacts of stay-at-home measures during a pandemic on mental health. Second, having analyzed over 10 million search queries related to mental health across 1 week, the scope of the study methodology in examining mental health is unprecedented in size. Third, we were able to contrast the pre-post changes from these stay-at-home orders in contrast to other states in the same country to show deviations from the general pattern. Fourth, we isolated that these mental health terms were unique to mental health trajectories within the same time span by comparing the searches to changes in non-COVID-19–related and COVID-19–related physical symptoms.

    Despite the study’s many strengths, the study also raises unanswered questions. Perhaps the greatest question is whether or not the flattening of these surges in mental health symptom searches is short-lived or if long-term stay-at-home orders will result in an abiding dampening of these symptoms. Thus, more research is needed to extend these findings to study the impacts of both COVID-19 and governmental responses to COVID-19 during this unprecedented time.

    Authors' Contributions

    All authors made significant contributions to the manuscript. The following were the respective roles for each author: NJ contributed to the study design, hypothesis generation, data collection, data analysis, data interpretation, and manuscript write-up and review; DL contributed to the literature searches, data collection, figure creation, and manuscript write-up and review; GP contributed to the literature searches, data collection, and manuscript write-up and review; MH contributed to the literature searches and manuscript write-up and review; MS contributed to the literature searches and manuscript write-up and review; JO contributed to the study design, data analyses, data interpretation, and manuscript review; and PB contributed to the study design, literature searches, manuscript write-up, data interpretation, and manuscript review.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Supplementary figures.

    DOCX File , 1380 KB

    Multimedia Appendix 2

    Supplementary tables.

    DOCX File , 18 KB

    References

    1. Lai C, Shih T, Ko W, Tang H, Hsueh P. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): The epidemic and the challenges. Int J Antimicrob Agents 2020 Mar;55(3):105924 [FREE Full text] [CrossRef] [Medline]
    2. Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 2020 Mar;395(10229):1054-1062. [CrossRef]
    3. Hatchett RJ, Mecher CE, Lipsitch M. Public health interventions and epidemic intensity during the 1918 influenza pandemic. Proc Natl Acad Sci U S A 2007 May 01;104(18):7582-7587 [FREE Full text] [CrossRef] [Medline]
    4. Fauci AS, Lane HC, Redfield RR. Covid-19 — navigating the uncharted. N Engl J Med 2020 Mar 26;382(13):1268-1269. [CrossRef]
    5. Lewnard JA, Lo NC. Scientific and ethical basis for social-distancing interventions against COVID-19. Lancet Infect Dis 2020 Mar. [CrossRef]
    6. Wang Y, Xu B, Zhao G, Cao R, He X, Fu S. Is quarantine related to immediate negative psychological consequences during the 2009 H1N1 epidemic? Gen Hosp Psychiatry 2011;33(1):75-77. [CrossRef] [Medline]
    7. Yang A, Tsai SJ, Huang NE, Peng CK. Association of Internet search trends with suicide death in Taipei City, Taiwan, 2004-2009. J Affect Disord 2011 Jul;132(1-2):179-184. [CrossRef] [Medline]
    8. Sueki H. Does the volume of Internet searches using suicide-related search terms influence the suicide death rate: data from 2004 to 2009 in Japan. Psychiatry Clin Neurosci 2011 Jun;65(4):392-394. [CrossRef] [Medline]
    9. Kristoufek L, Moat HS, Preis T. Estimating suicide occurrence statistics using Google Trends. EPJ Data Sci 2016;5(1):32 [FREE Full text] [CrossRef] [Medline]
    10. Parker J, Cuthbertson C, Loveridge S, Skidmore M, Dyar W. Forecasting state-level premature deaths from alcohol, drugs, and suicides using Google Trends data. J Affect Disord 2017 Apr 15;213:9-15. [CrossRef] [Medline]
    11. McCarthy MJ. Internet monitoring of suicide risk in the population. J Affect Disord 2010 May;122(3):277-279 [FREE Full text] [CrossRef] [Medline]
    12. Barros JM, Melia R, Francis K, Bogue J, O'Sullivan M, Young K, et al. The validity of Google Trends search volumes for behavioral forecasting of national suicide rates in Ireland. Int J Environ Res Public Health 2019 Sep 02;16(17) [FREE Full text] [CrossRef] [Medline]
    13. Waszak PM, Górski P, Springer J, Kasprzycka-Waszak W, Duży M, Zagożdżon P. Internet searches for "suicide", its association with epidemiological data and insights for prevention programs. Psychiatr Danub 2018 Dec;30(4):404-409 [FREE Full text] [CrossRef] [Medline]
    14. Gunn JF, Lester D. Using Google searches on the internet to monitor suicidal behavior. J Affect Disord 2013 Jun;148(2-3):411-412. [CrossRef] [Medline]
    15. Lee J. Search trends preceding increases in suicide: a cross-correlation study of monthly Google search volume and suicide rate using transfer function models. J Affect Disord 2020 Feb 01;262:155-164. [CrossRef] [Medline]
    16. Wang C, Pan R, Wan X, Tan Y, Xu L, Ho CS, et al. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int J Environ Res Public Health 2020 Mar 06;17(5) [FREE Full text] [CrossRef] [Medline]
    17. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 2020 Mar;395(10227):912-920. [CrossRef]
    18. Massicotte P, Eddelbuettel D. gtrendsR: perform and display Google trends queries. R package version 2020.
    19. COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU).   URL: https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6
    20. Soreni N, Cameron DH, Streiner DL, Rowa K, McCabe RE. Seasonality patterns of internet searches on mental health: exploratory infodemiology study. JMIR Ment Health 2019 Apr 24;6(4):e12974 [FREE Full text] [CrossRef] [Medline]
    21. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5). Washington, DC: American Psychiatric Pub; 2013.
    22. Fernandez KC, Fisher AJ, Chi C. Development and initial implementation of the Dynamic Assessment Treatment Algorithm (DATA). PLoS One 2017;12(6):e0178806 [FREE Full text] [CrossRef] [Medline]
    23. Prasad V, Jena AB. Prespecified falsification end points: can they validate true observational associations? JAMA 2013 Jan 16;309(3):241-242. [CrossRef] [Medline]
    24. Polansky L, Robbins MM. Generalized additive mixed models for disentangling long-term trends, local anomalies, and seasonality in fruit tree phenology. Ecol Evol 2013 Sep;3(9):3141-3151. [CrossRef] [Medline]
    25. Wood SN. Generalized Additive Models: An Introduction With R. Cleveland, OH: CRC Press; 2006:A.
    26. Jacobson NC, Chow S, Newman MG. The Differential Time-Varying Effect Model (DTVEM): a tool for diagnosing and modeling time lags in intensive longitudinal data. Behav Res Methods 2019 Feb;51(1):295-315 [FREE Full text] [CrossRef] [Medline]
    27. Wood SN. Thin plate regression splines. J Royal Statistical Soc B 2003 Feb;65(1):95-114. [CrossRef]
    28. Wood S. mgcv: mixed GAM computation vehicle with automatic smoothness estimation. R Package 2019.
    29. KGW. 2020 Mar 24. Portland police chief says suicide calls are up during coronavirus outbreak   URL: https:/​/www.​kgw.com/​article/​news/​health/​coronavirus/​watch-portland-mayor-ted-wheeler-gives-updates-on-covid-19-response/​283-20f9b2e2-14d2-45c0-a388-cabcb62b35c2
    30. Huang Y, Zhao N. Generalized anxiety disorder, depressive symptoms and sleep quality during COVID-19 epidemic in China: a web-based cross-sectional survey. medRxiv 2020 Mar 09. [CrossRef]
    31. Hawryluck L, Gold WL, Robinson S, Pogorski S, Galea S, Styra R. SARS control and psychological effects of quarantine, Toronto, Canada. Emerg Infect Dis 2004 Jul;10(7):1206-1212 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    COVID-19: coronavirus disease
    GAMM: generalized additive mixed model
    ICC: intraclass correlation coefficient
    OCD: obsessive-compulsive disorder


    Edited by G Eysenbach, J Torous; submitted 14.04.20; peer-reviewed by A Pratap, V Bremer; comments to author 14.05.20; revised version received 22.05.20; accepted 26.05.20; published 01.06.20

    ©Nicholas C Jacobson, Damien Lekkas, George Price, Michael V Heinz, Minkeun Song, A James O’Malley, Paul J Barr. Originally published in JMIR Mental Health (http://mental.jmir.org), 01.06.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.