This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.
Anxiety symptoms during public health crises are associated with adverse psychiatric outcomes and impaired health decision-making. The interaction between real-time social media use patterns and clinical anxiety during infectious disease outbreaks is underexplored.
We aimed to evaluate the usage pattern of 2 types of social media apps (communication and social networking) among patients in outpatient psychiatric treatment during the COVID-19 surge and lockdown in Madrid, Spain and their short-term anxiety symptoms (7-item General Anxiety Disorder scale) at clinical follow-up.
The individual-level shifts in median social media usage behavior from February 1 through May 3, 2020 were summarized using repeated measures analysis of variance that accounted for the fixed effects of the lockdown (prelockdown versus postlockdown), group (clinical anxiety group versus nonclinical anxiety group), the interaction of lockdown and group, and random effects of users. A machine learning–based approach that combined a hidden Markov model and logistic regression was applied to predict clinical anxiety (n=44) and nonclinical anxiety (n=51), based on longitudinal time-series data that comprised communication and social networking app usage (in seconds) as well as anxiety-associated clinical survey variables, including the presence of an essential worker in the household, worries about life instability, changes in social interaction frequency during the lockdown, cohabitation status, and health status.
Individual-level analysis of daily social media usage showed that the increase in communication app usage from prelockdown to lockdown period was significantly smaller in the clinical anxiety group than that in the nonclinical anxiety group (F1,72=3.84,
Patients who reported severe anxiety symptoms were less active in communication apps after the mandated lockdown and more engaged in social networking apps in the overall period, which suggested that there was a different pattern of digital social behavior for adapting to the crisis. Predictive modeling using digital biomarkers—passive-sensing of shifts in category-based social media app usage during the lockdown—can identify individuals at risk for psychiatric sequelae.
During the early peaks of casualties from the first wave of the COVID-19 pandemic, government lockdown measures in urban centers drastically diminished in-person communication and forced individuals to turn to the digital world to connect with others [
In contrast, positive public health outcomes are driven by individuals’ sound health decisions made based on accurate perceptions of the costs and benefits to self and society [
In recent years, passive smartphone sensor data have been utilized in empirical studies to identify various psychiatric presentations and mental health-related behaviors, including social anxiety severity [
In this study, we focused on analyzing daily time spent on apps in 2 social media categories (communication and social networking) in a sample of psychiatric outpatients in Madrid, Spain, before and during the mandatory COVID-19 lockdown. Communication apps allow direct messaging activity, and social networking apps enable interactions on social networking sites in heterogeneous forms. We hypothesized that differential forms of social media app activity can represent the distinct user behaviors that interplay with the manifestation of anxiety. Specifically, we aimed to employ a machine learning model and individual app usage patterns during this period to predict who would report clinical anxiety symptoms at follow-up.
Data were drawn from 2 ongoing studies [
Sociodemographic and clinical information were collected from all participants at baseline before the onset of the pandemic via an electronic health tool (MEmind [
From February 1 through May 3, 2020, passive smartphone usage data were collected using eB2 Mindcare [
A clinical psychologist collected short-term mental health outcomes, including self-reported intensity of psychosocial stressors during the lockdown and Generalized Anxiety Disorder 7-item scale (GAD-7), by phone follow-up between May 12 and June 3 after the initial lockdown measures had been lifted. Clinical anxiety was defined as a GAD-7 score of 10 or greater, given its diagnostic value in screening for severe GAD, panic disorder, and social phobia [
Group-level differences were evaluated using the 2-sided
We designed a 2-step approach that combined a probabilistic generative model, namely a hidden Markov model (HMM) [
The proposed anxiety prediction pipeline. LR: logistic regression; HMM; hidden Markov model; S1, S2, S3; the 3 states of the hidden Markov model.
HMMs are commonly used for time-series analysis. HMMs model generative sequences, which are characterized by a set of observable sequences. A first-order Markov chain process generates the states of the HMM. The following components specify an HMM:
The state space of the applied HMM is discrete, while the observations can be discrete or continuous. In this study, communication and social networking app usage are treated as continuous variables from a Gaussian distribution. The parameters of an HMM can be trained with the Baum-Welch algorithm, a variation of the expectation-maximization algorithm. The model can deal with missing data using marginalization without requiring imputation before training. To select the optimal number of hidden states, we computed the Akaike information criterion and the Bayesian information criterion [
Once the optimal HMM was selected for each sequence, we computed the state posterior probabilities
The evaluation was performed using
Finally, we performed feature importance analysis by computing Shapley additive explanations (SHAP) values [
Of 142 participants (
Demographic and clinical information.
Variable | All (n=142) | Clinical anxietya (n=66) | Nonclinical anxietyb (n=76) | |||||||||||
|
||||||||||||||
|
Age (years), mean (SD) | 45 (14.2) | 43 (13.6) | 47 (14.6) | –1.8 (139)c | .07 | ||||||||
|
|
0.36 | .72 | |||||||||||
|
|
Male | 43 (30) | 19 (29) | 24 (32) |
|
|
|||||||
|
|
Female | 99 (70) | 47 (71) | 52 (68) |
|
|
|||||||
|
|
|
|
|
0.83 | .40 | ||||||||
|
|
No | 21 (15) | 8 (12) | 13 (17) |
|
|
|||||||
|
|
Yes | 121 (85) | 58 (88) | 63 (83) |
|
|
|||||||
|
|
3.5 (3)d | .32 | |||||||||||
|
|
Single | 46 (32) | 21 (32) | 25 (33) |
|
|
|||||||
|
|
Separated | 26 (18) | 11 (17) | 15 (20) |
|
|
|||||||
|
|
Widowed | 6 (4) | 5 (8) | 1 (1) |
|
|
|||||||
|
|
Married or cohabitating for >6 months | 64 (45) | 29 (44) | 35 (46) |
|
|
|||||||
|
|
7.5 (5)d | .18 | |||||||||||
|
|
Employed, student or homemaker | 51 (36) | 23 (35) | 28 (37) |
|
|
|||||||
|
|
Unemployed without subsidy | 28 (20) | 9 (14) | 19 (25) |
|
|
|||||||
|
|
Unemployed with subsidy | 17 (12) | 8 (12) | 9 (12) |
|
|
|||||||
|
|
Long-term disability | 11 (8) | 7 (11) | 4 (5) |
|
|
|||||||
|
|
Temporarily incapacitated | 26 (18) | 16 (25) | 10 (13) |
|
|
|||||||
|
|
Retired | 8 (6) | 2 (3) | 6 (8) |
|
|
|||||||
|
Anxiety, stress, or trauma disordere, n (%) | 79 (58) | 41 (63) | 38 (54) | 1.1 | .26 | ||||||||
|
Mood disordere, n (%) | 50 (37) | 28 (43) | 22 (31) | 1.5 | .14 | ||||||||
|
Personality disordere, n (%) | 30 (22) | 14 (22) | 16 (23) | –0.14 | .89 | ||||||||
|
Substance use disordere, n (%) | 8 (6) | 5 (8) | 3 (4) | 0.86 | .39 | ||||||||
|
Psychotic disordere, n (%) | 3 (2) | 2 (3) | 1 (1) | 0.66 | .51 | ||||||||
|
Other psychiatric disordere, n (%) | 21 (15) | 10 (15) | 11 (15) | –0.02 | .99 | ||||||||
|
||||||||||||||
|
|
12 (4)d | .01 | |||||||||||
|
|
Not at all | 21 (15) | 8 (12) | 13 (18) |
|
|
|||||||
|
|
Slightly | 32 (23) | 9 (14) | 23 (31) |
|
|
|||||||
|
|
Moderately | 37 (26) | 19 (29) | 18 (24) |
|
|
|||||||
|
|
A lot | 35 (25) | 18 (27) | 17 (23) |
|
|
|||||||
|
|
Extremely | 15 (11) | 12 (18) | 3 (4) |
|
|
|||||||
|
|
6.4 (2)d | .04 | |||||||||||
|
|
Positive | 66 (46) | 23 (35) | 43 (57) |
|
|
|||||||
|
|
Regular | 56 (40) | 32 (49) | 24 (32) |
|
|
|||||||
|
|
Negative | 19 (13) | 10 (15) | 9 (12) |
|
|
|||||||
|
|
3.2 (2)d | .21 | |||||||||||
|
|
Phone calls | 66 (46) | 34 (52) | 32 (43) |
|
|
|||||||
|
|
Video calls | 45 (32) | 16 (24) | 29 (39) |
|
|
|||||||
|
|
Messengers (WhatsApp, Telegram, etc) | 29 (20) | 15 (23) | 14 (19) |
|
|
|||||||
|
|
6.8 (2)d | .03 | |||||||||||
|
|
Less frequent than prepandemic | 63 (44) | 36 (55) | 63 (36) |
|
|
|||||||
|
|
More or less the same | 48 (34) | 21 (32) | 48 (36) |
|
|
|||||||
|
|
More frequent than prepandemic | 31 (22) | 9 (14) | 31 (29) |
|
|
|||||||
|
Tested positive on SARS-CoV-2 PCRf test, n (%) | 1(1) | 0 (0) | 1 (1) | –0.94 | .34 | ||||||||
|
Living with people with COVID-19, n (%) | 16 (11) | 9 (14) | 7 (10) | 0.75 | .45 | ||||||||
|
Living with older adult, n (%) | 6 (4) | 3 (5) | 3 (4) | 0.20 | .84 | ||||||||
|
Essential workers in household, n (%) | 43 (30) | 27 (41) | 16 (23) | 2.3 | .02 | ||||||||
|
Knew people who died of COVID-19, n (%) | 43 (30) | 25 (38) | 18 (24) | 1.7 | .08 | ||||||||
|
Generalized Anxiety Disorder-7 score postlockdown (mean, SD) | 9.6 (5.5) | 14.6 (3.1) | 5.2 (2.8) | 19 (133)c | <.001 |
aGeneralized Anxiety Disorder-7 score ≥10.
bGeneralized Anxiety Disorder-7 score <10.
cA 2-sided
dA chi-square independence test was used.
ePsychiatric diagnosis categories are not mutually exclusive.
fPCR: polymerase chain reaction.
Active users (n=42; Table S1 in
From prelockdown to lockdown period, the mean of individual median usage on communication app (in both anxiety groups) increased from 29 minutes (95% CI 25-35) to 41 minutes (95% CI 35-49;
The effect of lockdown on the increase in communication app usage was lower in the clinical anxiety group. Error bars indicate 95% standard error of the mean.
No significant correlations were found between GAD-7 and median communication or social networking app usage during prelockdown (communication:
Only the patients with communication and social networking app usage data during both the prelockdown (≥1 out of 42 days) and lockdown period (≥1 out of 51 days) were considered for the model training. This resulted in 95 patients in the model with varying sequences of individual app usage data. In these sequences, 8.76% (655/7476) of the communication app and 30.26% (2262/7476) of the social networking app usage data were missing in the data set. Figure S4 in
An HMM with 3 hidden states (
The 3-state hidden Markov model parameters used for temporal data modeling and most probable hidden Markov model states applied to daily communication and social media app usage of example individuals with clinical and nonclinical anxiety. Temporal variables were normalized before model training, providing the negative means. Large state transition probabilities suggest that the states were relatively stable.
Our model achieved a mean accuracy of 62.30% (SD 16%) and an AUROC of 0.70 (SD 0.19) in predicting the clinical anxiety group on the test sets (Table S4 in
The majority of nontemporal features, led by the presence of essential workers in the household, outweighed the aggregated representation of the temporal features in importance. Among temporal features, the aggregated posterior probability of state 2 (higher social networking app use) was the most important predictor of the clinical anxiety group (
Feature importance (Shapley additive explanation [SHAP] value) for the logistic regression model trained for the anxiety prediction task in the order of descending importance (colored by value, from low to high). Each point is for a feature and an instance, and overlapping points are jittered in the y-axis direction. SHAP values encode the feature’s predictability for classifying the participant (positive, in the clinical anxiety group; negative, in the nonclinical anxiety group). For explanation of the encoded feature values see Table S5 in
Our findings demonstrate that among active social media users, those who reported clinical levels of anxiety symptoms after the mandatory lockdown spent less time on communication apps during the lockdown. Active social-networking app users, biased toward younger patients, additionally had a higher likelihood of having an anxiety disorder diagnosis. Our machine learning–based model, trained on the temporal series of communication and social networking app usage and clinically important features of self-report and demographic variables, accurately predicted which individuals were in the clinical anxiety group from higher social networking app usage and lower communication app usage. Our machine learning–based model results suggest that passive tracking of decreased communication app usage and increased social networking app usage through the lockdown period can predict users reporting clinical anxiety symptoms, at risk for impaired decision-making, maladaptive coping, and psychiatric sequelae during public health crises and lockdown periods [
We interpret the findings from the perspective that patients who reported fewer anxiety symptoms proactively harnessed their digital social environment by using communication apps (eg, WhatsApp and Telegram) to initiate contact or respond to others’ direct messages during this highly anxiogenic period (ie, during the COVID-19 pandemic and related lockdown). Our analysis is consistent with the clinical anxiety group’s self-reports that they had less frequent social interactions with others during the lockdown. Social support systems, either in-person or online, are well-known protective factors against physiological and psychological stressors and can mitigate the impact of loneliness in times of uncertainty, including during infectious disease outbreaks [
Conversely, users in our sample who were highly active on social networking apps were more likely to be diagnosed with anxiety disorder and report clinical anxiety symptoms. Social networking apps, such as Facebook, Twitter, TikTok, and Instagram, are examples of web 2.0 technology apps that have shifted the recent web-based environment of health communications, from traditionally one-way communication to interactive and iterative, characterized by passive sharing, active collaboration, and amplification of information [
Our passively sensed user-driven data were prospectively collected within users’ natural environment and under no influence of perceived experimental manipulation. They also contained clearly divided timeframes, followed by timely clinical surveys, which allowed our model of human behavior during the national lockdown to have high interpretability, which is critical for translating digital phenotyping research to real-life application [
Our analysis was based on observing a small number of patients and should be interpreted with the following limitations. First, the data cannot explain the causal link between app usage and the severity of anxiety. For example, we do not know if decreased engagement in communication apps contributed to the reporting of higher anxiety symptoms, or if the former was a characteristic of the group that developed short-term clinical anxiety symptoms during the crisis (ie, a smaller volume of social support for communication to begin with). Second, besides
To the best of our knowledge, our empirical data are the first to suggest that category-based passive sensing of a shift in smartphone usage patterns can be markers of clinical anxiety symptoms. Further studies, to digitally phenotype short-term reports of anxiety using granular behaviors on social media, are necessary for public health research when in-person psychiatric evaluations are limited during mandated physical isolation.
Supplementary tables and figures for statistical data analysis and the 10-fold cross-validation of our hidden Markov + logistic regression model.
area under the receiver operating characteristics curve
Generalized Anxiety Disorder scale
hidden Markov model
Shapley additive explanation
JR was supported by the American Psychiatric Association 2021 Junior Psychiatrist Research Colloquium (NIDA R-13 grant). ES received funding from the European Union Horizon 2020 research and innovation program (Marie Sklodowska-Curie grant 813533). AA is supported by the Spanish
JR conceived the work, analyzed data, and wrote the manuscript. ES analyzed data, developed the machine learning models, and wrote the manuscript. AN, SL, EB, MMP-R, and AA contributed to the critical review of the draft for important intellectual content. JC acquired and interpreted data for the work. MMP-R mentored JR through the conception of the study, supervised data analysis and interpretation of results, and ensured that questions related to any part of the work’s accuracy or integrity were appropriately investigated and resolved.
AA and JC are cofounders of eB2. MMP-R has received research grant funding from Neurocrine Biosciences, Millennium Pharmaceuticals, Takeda, Merck, and AI Cure; she is an Advisory Board member for Neurocrine Biosciences Inc and a consultant on an American Foundation for Suicide Prevention (grant LSRG-1-005-16, principal investigator: EB-G).