This paper is in the following e-collection/theme issue:

Methods and New Tools in Mental Health Research (454) Posttraumatic Stress Disorder (PTSD) (116) Reviews in Digital Mental Health (243)

Published on in Vol 7, No 7 (2020): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/16654, first published .
Toward a Taxonomy for Analyzing the Heart Rate as a Physiological Indicator of Posttraumatic Stress Disorder: Systematic Review and Development of a Framework

Toward a Taxonomy for Analyzing the Heart Rate as a Physiological Indicator of Posttraumatic Stress Disorder: Systematic Review and Development of a Framework

Toward a Taxonomy for Analyzing the Heart Rate as a Physiological Indicator of Posttraumatic Stress Disorder: Systematic Review and Development of a Framework

Authors of this article:

Mahnoosh Sadeghi1 Author Orcid Image ;   Farzan Sasangohar1, 2 Author Orcid Image ;   Anthony D McDonald1 Author Orcid Image
Article Authors Cited by (9) Tweetations (7) Metrics

    Review

    1Department of Industrial and Systems Engineering, Texas A&M University, College Station, TX, United States

    2Center for Outcomes Research, Houston Methodist Hospital, Houston, TX, United States

    Corresponding Author:

    Farzan Sasangohar, BCS, BA, MASc, SM, PhD

    Department of Industrial and Systems Engineering

    Texas A&M University

    3131 TAMU

    College Station, TX, 77843

    United States

    Phone: 1 9794582337

    Email: sasangohar@tamu.edu


    Background: Posttraumatic stress disorder (PTSD) is a prevalent psychiatric condition that is associated with symptoms such as hyperarousal and overreactions. Treatments for PTSD are limited to medications and in-session therapies. Assessing the way the heart responds to PTSD has shown promise in detecting and understanding the onset of symptoms.

    Objective: This study aimed to extract statistical and mathematical approaches that researchers can use to analyze heart rate (HR) data to understand PTSD.

    Methods: A scoping literature review was conducted to extract HR models. A total of 5 databases including Medical Literature Analysis and Retrieval System Online (Medline) OVID, Medline EBSCO, Cumulative Index to Nursing and Allied Health Literature (CINAHL) EBSCO, Excerpta Medica Database (Embase) Ovid, and Google Scholar were searched. Non–English language studies, as well as studies that did not analyze human data, were excluded. A total of 54 studies that met the inclusion criteria were included in this review.

    Results: We identified 4 categories of models: descriptive time-independent output, descriptive and time-dependent output, predictive and time-independent output, and predictive and time-dependent output. Descriptive and time-independent output models include analysis of variance and first-order exponential; the descriptive time-dependent output model includes a classical time series analysis and mixed regression. Predictive time-independent output models include machine learning methods and analysis of the HR-based fluctuation-dissipation method. Finally, predictive time-dependent output models include the time-variant method and nonlinear dynamic modeling.

    Conclusions: All of the identified modeling categories have relevance in PTSD, although the modeling selection is dependent on the specific goals of the study. Descriptive models are well-founded for the inference of PTSD. However, there is a need for additional studies in this area that explore a broader set of predictive models and other factors (eg, activity level) that have not been analyzed with descriptive models.

    JMIR Ment Health 2020;7(7):e16654

    doi:10.2196/16654

    Keywords

    heart rate; statistics; PTSD; mental health; physiology 


    Background

    Posttraumatic stress disorder (PTSD) is a psychiatric condition that develops as a result of experiencing injury, severe psychological shock, and other trauma [1]. Individuals with PTSD are affected by the recall of traumatic experiences and often develop depression, anxiety, emotional instabilities, and suicidal thoughts [2]. Recent reports suggest that individuals with PTSD are about 5 times more likely to commit suicide than individuals without PTSD [3]. Approximately 10% of American women and 4% of American men experience PTSD in their lifetime [4]. PTSD is an endemic among veterans as well, affecting between 17% and 24% of veterans from recent conflicts [5].

    Although an alarming number of individuals are afflicted with PTSD, there are significant barriers to care delivery [6,7]. These barriers include a shortage of qualified clinicians and understaffed mental health clinics, geographical constraints to accessing mental health facilities, financial obstacles, and cultural factors such as social stigma, and limited capabilities in objective diagnosis (currently limited to self-reported measures such as the PTSD checklist [PCL-5]) [8]. Studies have shown that self-management and factors such as positivity directly affect PTSD symptoms and ease in dealing with them [9]. Mobile health (mHealth) apps have shown promise in facilitating self-management (eg, education, mindfulness, and self-assessment) and have the potential to facilitate direct communication between people who have PTSD and their health care providers [10]. mHealth apps deployed on wearable devices (eg, smartwatches) that are equipped with an array of physiological sensors (eg, heart rate [HR]) may also enable continuous remote monitoring of signs and symptoms of PTSD. Indeed, recent efforts have shown promising applications of watch-based HR sensors to detect the onset of PTSD hyperarousal events [11].

    Objectives

    Despite recent work, the extent of knowledge on the physiological reactions to PTSD and, in particular, HR is limited, and research is needed to better understand the changes in HR associated with PTSD. Few models (eg, analysis of variance [ANOVA], regression analysis) have been developed to relate changes in heart activity to disorder states. In particular, given the opportunity to collect HR data nonintrusively, it is important to use appropriate mathematical and statistical methods to ensure the accumulation of convergent knowledge in this field and to characterize and understand HR in terms of PTSD. In this paper, we document the findings from a review of the current literature on measures and models used in various domains to analyze HR data. In addition to summarizing and synthesizing the HR analysis methods, we provide an evaluation of methods for applications relevant to PTSD detection and diagnosis.


    Search Strategy

    A scoping review was conducted using the strategies outlined in the preferred reporting items for systematic reviews and meta-analyses (PRISMA) methodology [12]. The scoping review approach was selected because it is effective for knowledge evaluation and gap identification [13]. The review spanned 5 main databases: (1) Medical Literature Analysis and Retrieval System Online (Medline) OVID, (2) Medline EBSCO, (3) Cumulative Index to Nursing and Allied Health Literature (CINAHL) EBSCO, (4) Excerpta Medica Database (Embase) Ovid, and (5) Google Scholar. Search terms included heart*,pulse*,heart rate*, model*, heart beat*, and analysis*. All studies published in or after the year 2000 were included. This search was supplemented by a secondary search of cited articles in the results. The search was completed on January 15, 2020.

    Study Selection, Inclusion, and Exclusion Criteria

    Abstracts were reviewed for relevance, and articles that did not discuss HR-related measures in detail and did not provide or use quantitative methods for analysis were excluded. Other exclusion criteria were non–English language articles and articles that assessed non–heart-based physiological measures such as skin conductance and blood pressure. Furthermore, studies that did not analyze human physiology were excluded. The inclusion criteria were all articles that discussed human HR analysis. Our initial search yielded 1905 results. After removing duplicate articles and checking for eligibility using Rayyan QCRI (a web app for assisting literature reviews), 270 articles were further reviewed. Out of the 270, 138 were exclusively about non–heart-based measures reactions, 67 did not focus on human physiology, and 11 had duplicated content. Of these, 54 articles from the search were included in this review based on their relevance to the topic.

    Furthermore, the bibliography of references in each research paper was investigated thoroughly (backward search) to identify pertinent articles, and then Google Scholar searches (forward search) were conducted to find the full text. Figure 1 shows the PRISMA flow chart for the article selection process.

    ‎
    Figure 1. Preferred reporting items for systematic reviews and meta-analyses flow chart for the literature review.

    We listed the articles identified by the search process into 2 categories based on our synthesis: studies of the effects of PTSD on heart physiology and quantitative modeling techniques for heart data. We further partitioned studies of PTSD effects into 2 types: (1) studies that investigate the effect of PTSD on heart rate variability (HRV) and (2) studies that explore the effect of PTSD on HR. The literature on models can be further classified by the model’s focus on describing versus predicting data and the model output. These categories and subdivisions are discussed in the following sections.

    Effects of Posttraumatic Stress Disorder on Heart Rate Variability

    HRV measures variations in heartbeats and is related to the electrical activity of the heart [14]. Common frequency domain analysis metrics for HRV include high frequency power (HF), low frequency power (LF), the ratio of LF to HF, coherence score (COH), root mean square of successive differences between normal heartbeats (RMSSD), and the SD of the interbeat interval of normal sinus beats (SDNN) [15-18]. LF and HF are frequency bands of HRV that tend to correlate with parasympathetic nervous system activity. LF is the frequency activity in the range of 0.04 to 0.15 Hz, and HF is the activity in the range of 0.15 to 0.4 Hz. The quantified relative intensity of these measures is referred to as power [1], and such power is obtained by applying power spectral and frequency domain analyses [19].

    The reviewed articles found that PTSD causes sustained changes in the autonomic nervous system (ANS; the part of the nervous system that is responsible for regulating automated functions in the body, such as heart activity) [20]. The ANS consists of the parasympathetic nervous system (PNS), which regulates blood pressure and breathing rate during rest, and the sympathetic nervous system (SNS), which adjusts blood pressure and HR during activity. Heart activity is representative of the performance of these systems [21]. Various effects of PTSD on ANS have also been documented. Higher HR levels indicate lower HRV and are linked to increased rates of mental stress and physical activity [22,23]. PTSD, as a particular type of anxiety disorder, also disturbs HR and HRV. HRV has been studied widely in the literature to assess PTSD [18,24-26]. Evidence suggests that individuals with PTSD have lower resting HRV than individuals without PTSD when other factors (age, gender, and health level) are controlled [27]. According to the meta-review Nagpal et al [1], HF, a measure for the parasympathetic activity of ANS, is significantly lower in individuals with PTSD than in individuals without PTSD (approximately 0.6 ms2). However, LF, which assesses both the sympathetic and parasympathetic activity of the ANS, is slightly reduced in individuals with PTSD (approximately 0.2 ms2). This results in a significant increase in LF divided by HF of individuals with PTSD [1,28-30].

    RMSSD and SDNN are time-domain measures of HRV. SDNN is an index of SNS activity [24]. SDNN is decreased in individuals with PTSD compared with healthy individuals (approximately 6.7 ms), showing an increase in sympathetic activity [1,31]. In addition, decreased levels of RMSSD was observed among individuals with PTSD (approximately 7.5 ms), suggesting lower vagal activity in this population [1,31].

    Although an HRV analysis is common among studies of anxiety [32], some factors need to be considered when HRV measures are used. First, studies show that HRV is dependent on HR and cannot be analyzed independently to represent ANS activity [32,33]. In addition, previous research has linked high HRV to pathological conditions related to heart deficiencies [32]. For instance, diseases such as atrial fibrillation increase HRV and HR and are associated with higher mortality rates [34]. Hence, higher rates of HRV do not always indicate an abnormal mental state. Ideally, measurements should take into account patient’s comorbidities such as heart deficiencies in addition to subjective (eg, self-reported scales) and objective (eg, HRV, ECG) methods [35]. Gender, health, age, and HR also affect HRV, and they need to be considered as covariates when HRV measures are used [24]. Aging decreases HRV time-domain features such as SDNN [36,37]. HRV time-domain features increase with improved health conditions [38,39]. LF and SDNN are also lower in females than in males; however, the HF parameter of HRV is greater in women than in men [40]. Higher HR levels are also associated with decreased HRV [41] because when the heart beats faster, the beat-to-beat intervals are smaller. Other factors such as climate, job satisfaction, lifestyle, and medications can also affect HRV and should be considered as an influential factor when HRV is analyzed [42].

    Effect of Posttraumatic Stress Disorder on Heart Rate

    HR is the number of heartbeats per 60 seconds. Normal HR differs among individuals based on age and gender, health level, and respiratory activity [43]. Both HR and HRV are modulated by the ANS [44]. As the SNS activates, PNS activity is suppressed; therefore, HR increases and HRV decreases [45]. As a result, there is an inverse relationship between HR and HRV [33].

    PTSD can affect HR in 2 modalities: resting and fluctuation tone [1,46-48]. Studies suggest that resting HR can be between 5 and 6.6 beats higher in individuals with PTSD than in individuals without PTSD depending on the type of population (eg, veteran, civilian) [49-51]. For example, resting HR is roughly higher than 5 beats per minute in civilians with PTSD than in civilians without PTSD, and this number increases to 6.6 beats per minute in the veteran population [51,52]. In the nonresting state, evidence suggests that HR increases with exposure to PTSD stressors [1].

    Another HR measure that has been investigated in terms of PTSD is HR fluctuations (changes in HR levels) in the presence of stimuli [53]. There are conflicting findings on the comparison of this measure between individuals with and without PTSD. Although a study by Roy et al [54] showed that HR changes are higher in people with PTSD than in people without PTSD, a study by Halligan et al [55] claims the opposite.

    Heart Rate Models

    On the basis of our synthesis of the existing literature, we categorized mathematical models of HR into descriptive and predictive models, both of which could provide insight relevant to understanding the psychophysiological responses to PTSD. Descriptive methods can be used to describe and make inferences about a data set, whereas predictive methods can be applied to forecast trends and patterns in the data. Predictive and descriptive models can be further characterized by their type of output—time independent or time dependent (Figure 2). Time-dependent outputs use time as one of the descriptive variables to analyze the dependent variable(s) or output(s). Time-independent output, however, does not depend on time and does not change over time. Although the models reviewed below are summarized and synthesized for relevance to PTSD-related analysis, these methods are not limited to PTSD and anxiety disorder domains.

    ‎
    Figure 2. Taxonomy of heart rate analysis methods. ANOVA: analysis of variance.
    Descriptive Models
    Time-Independent Output
    Analysis of Variance

    Linear regression, and in particular ANOVA, is a statistical model used for the analysis of HR in several articles (Table 1). ANOVA can be used to compare HR trends and group means in experimental studies [56,57]. Studies have used ANOVA to account for the effectiveness of treatments in individuals with PTSD, as measured by HR [58]. Some studies chose ANOVA as their method of analysis to show that resting HR is higher in individuals with PTSD than in individuals without PTSD [57]. For example, the study by Gelpin et al [59] compared the resting HR in individuals pre-and posttreatment to measure the success of therapy sessions. Buckley et al [52] used ANOVA to compare resting HR in patients with PTSD with that of healthy controls, finding that patients with PTSD, in general, have significantly higher resting HR levels (approximately a 6 beats-per-minute difference). Although using ANOVA for the analysis of time-independent HR data is highly common, ANOVA is limited in several respects. ANOVA has strong assumptions and is ill-suited to model-dependent measures with strong temporal correlations. For instance, the independency of observations is one of the main assumptions of ANOVA; however, consecutive HR real time–based data are a highly correlative type of data. Thus, ANOVA should not be used to make time-based HR predictions [60].

    First-Order Exponential Model

    A first-order exponential model provides a function with a sustained growth or decay rate [61]. In terms of HR analysis, first-order exponential models have been used to generate a nonlinear regression model for HR based on heart rate recovery (HRR) [62]. HRR is an indicator of vagal reactivation and SNS deactivation [63].

    Bartels-Ferreira et al [63] used the first-order exponential method to measure postexercise time-independent HRR based on HR decay curves. Recovering from the onset of PTSD symptoms is associated with activation of vagal tone and withdrawal of SNS activity, both of which are correlated with HRR [64]. Although this method shows promise in the assessment of HR fluctuations associated with PTSD, the reviewed literature (Table 1) examined ANS in the context of physical activity, and HR decay after activity was curve fitted by a first-order exponential function [63]. In this case, the goodness of fit was moderate (R2 was approximately 0.65), which warrants additional research. Another limitation associated with this method is that the exponential functions show erroneous patterns for very small (30-second) and very large (600-second) time windows [61]. For instance, Bartels-Ferreira et al [63] found that the least goodness of fit was for the smallest time window, which was 30 seconds (R2=0.42). Conversely, when the length of the window of time was a moderate number (approximately 360 seconds), a relatively better goodness of fit was obtained (approximately 0.69). This shows that the HRR curve fitted by first-order exponential models performs better (higher R2) when windows of times are neither too big nor too small. Table 1 shows a summary of articles that studied descriptive models with time-independent output. In this table, domain is the field of the study. Independent variables are factors that are controlled by researchers, and dependent variables are dependent on them. Independent variables are used to describe or classify dependent variable.

    Table 1. Results of studies that used descriptive models with time-independent output.
    Method and authorsDomainIndependent variablesDependent variables
    ANOVAa

    		Shalev et al [57]PTSDbGender, age, HRc, trauma history, event securityHR

    		Strath et al [65]Physical activityHR, oxygen intake, age, fitnessHR

    		Romero-Ugalde et al [66]Physical activityAccelerometer, energy expenditure, HRHR

    		Khoueiry et al [67]MedicalHR, hospitalization duration, ageHR

    		Tonhajzerova et al [68]PhysiologyResting HR, major depressive disorderHR
    First-order exponential

    		Bartels et al [63]Physical activityHR peak, resting HR, HRRdHR variation

    aANOVA: analysis of variance.

    bPTSD: posttraumatic stress disorder.

    cHR: heart rate.

    dHRR: heart rate recovery.

    Time-Dependent Output

    Classical Time Series Analysis

    Classical time series analysis is a common statistical method that can analyze time-dependent data trends by looking into linear relationships. Classical time series analysis is also a promising method for analyzing HR and HR fluctuations as these measures are time-based [69,70].

    Peng et al [70] applied time series analysis to examine the long-term correlation within HR data and its relation to heart diseases such as cogestive heart failure. Using this method, the authors showed that there is some independency between beat-to-beat HR fluctuations in healthy people that does not exist in patients with cardiovascular disease. The findings further suggest that classical time series analysis is a promising direction for PTSD hyperarousal analysis because similar HR changes have been documented in patients with PTSD compared with healthy people in the presence of stimuli [71].

    Beyond the analogous use case, the classical time series has several benefits compared with ANOVA. As the model explicitly considers autocorrelation, it does not require the assumption of independence of observations [72]. The models also have predictive capability and are well validated for illustrating trends and forecasting [73]. However, 1 drawback of this method is the stationary assumption (constant mean value of the series), which is not always reasonable in HR data (eg, when data are collected before and during exercise).

    Mixed Regression Model

    Mixed regression analysis has been used in the literature to evaluate physiological responses to energy expenditure [74]. This type of modeling can be applied with correlated observations. Thus, it is beneficial for psychophysiology analyses that need to account for individual similarities such as gender [60]. Multiple regression typically proceeds in a stepwise process with a focus on identifying 2 main effects: the population fixed effect and the random effect. The population-fixed effect explains similarities in the dataset (for instance HR), whereas the random effect represents the differences among observations (the error term). For instance, Gee et al [75] used respiration as a random effect to estimate HR and ultimately predict episodes of bradycardia in infants. Using a mixed regression method and accounting for respiration as a covariate, in this case, has increased the accuracy of the measured HR by 11%.

    The ability of mixed regression models to account for individual differences makes them an advantageous choice for modeling PTSD. Several studies have identified significant individual differences in people with PTSD [1,57,76,77]. Specifically, HR and HRV levels are significantly affected by individual differences such as age, general health, and gender [24].

    This type of modeling might produce similar results to ANOVA in many cases. However, in comparison with ANOVA, mixed regression models are more effective for data sets with missing values and multiple random effects [78]. This is important as in real-world and naturalistic studies, data sets with high rates of missing values are common and can be challenging to deal with [79]. Table 2 shows a comparison of time-dependent output methods.

    Table 2. Results from studies that used descriptive models with time-dependent output.
    Method and authorsDomainIndependent variablesDependent variable
    Classical time series

    		Chen et al [69]Health care (patient data)HRa, resting HRHeartbeat

    		Kazmi et al [33]PhysiologyHR, HRVb, timeHR

    		Zakeri et al [80]Physical activityHR, energy expenditure, accelerometer, ageEnergy expenditure

    		Peng et al [70]MedicalHR, heartbeat, timeHR
    Mixed regression

    		Gee et al [75]BiomedicalHR, heartbeat, respiration, timeHR

    		Bonomi et al [81]Physical activityHR, energy expenditure, photoplethysmography, accelerometerHR

    		Xu et al [82]Physical activityHR, energy expenditure, different training paradigms, age, height, weightEnergy expenditure

    aHR: heart rate.

    bHRV: heart rate variability.

    Predictive Models
    Time-Independent Output
    Machine Learning Methods

    Machine learning methods refer to a set of training and predictive algorithms that use data to learn complex trends associated with labels (eg, symptom presence) in a data set. Machine learning analysis is a multiple-step process consisting of dividing a data set into training and testing data (or leveraging resampling techniques such as cross-validation), developing a model from the training data, and evaluating the model on the testing data. This approach is advantageous relative to approaches that use all of the data for training a model (eg, ANOVA) and approximate metrics to evaluate generalizability (eg, adjusted R2). Furthermore, the ability of machine learning algorithms to identify complex patterns in data sets make them a promising approach for analyzing physiological data that are often noisy.

    The success of applying machine learning methods depends on the data used to train and evaluate the algorithm. Machine learning algorithms typically require large training sets—several thousand observations—and they implicitly assume that the data and associated labels are of equal quality. In cases where the data are noisy, or labels are unreliable, machine learning training algorithms may fail to converge to a generalizable solution. Furthermore, if the training data examples are biased (eg, nonrepresentative population samples), the machine learning algorithms trained on the data may also be similarly biased. It is often difficult to identify these issues through standard training and testing processes of machine learning algorithms; thus, machine learning analyses should be accompanied by descriptive analyses to obtain a better understanding of the data and potential errors or bias [83].

    Most of the reviewed studies used HRV, along with machine learning algorithms to predict stress levels in individuals [84-86]. Machine learning studies evaluating HR have primarily focused on energy expenditure [87,88]. An exception is McDonald et al [11] who evaluated several machine learning algorithms—neural networks, decision trees, support vector machines, convolutional neural networks, and random forests—to predict the onset of PTSD symptoms in the veteran population. This study used HR data with a 1 Hz frequency (1 observation per second) as the input of these algorithms. Although the raw 1 Hz data were used to train the neural network–based models, additional feature generation and selection was performed before training the decision tree, support vector machine, and random forest algorithms. This feature generation identified linear trends, Fourier transforms, and change quantiles as relevant features for the detection of the onset of PTSD symptoms. Among all machine learning methods, support vector machines, and random forest algorithms performed best (ie, had the highest area under the receiver operating characteristic curve (ROC) 0.67). Although machine learning shows promise for the inferential analysis of HR data for PTSD research, explaining the purpose of machine learning components may be difficult, and often predictive results have a limited rational explanation [89].

    Fluctuation-Dissipation Theory

    The fluctuation-dissipation theory (FDT) is a common approach in thermodynamics that is used to predict system behavior by breaking the system responses into small forces [90]. This theorem, which follows thermodynamic rules, can model the HRR after stress moments.

    Chen et al [91] used FDT to predict patients’ HR reactions to prespontaneous and postspontaneous breathing trial treatment. They used this method to divide the system (in this case, the treatment process) into different phases, including pretreatment, midtreatment, and posttreatment. After breaking the entire treatment process to these small phases, each phase was modeled separately. The reactions to treatments in each phase were modeled using HRR measures. All models were then combined to create the final comprehensive model. Chen et al [91] found that thermodynamic rules can also model the HR response after stress moments. This is because of the similar effect of stress and spontaneous breathing trials on organs (a common clinical procedure used to assess the ventilation performance of patients). These researchers suggest dividing the system into prestress and poststress moments, modeling each phase, and finally assembling a model for final prediction. They further suggest that the HRR extracted from this type of modeling can be used to personalize care as HR can be remotely monitored through noninvasive hospital devices.

    In terms of mathematical concepts, this type of modeling has a powerful predictive capability by grouping individuals and therefore minimizing the error rate [91]. This approach requires significantly less data than other methods, such as time-variant modeling of HR. Hence, it enables researchers to include more variables in their model. Moreover, Chen et al [91] claim that although models that use Gaussian functions have around 65% error rate to predict patients’ response to spontaneous breathing trial, implementing FDT decreases this error rate by over 10%. Therefore, this approach provides more accurate results than methods that use Gaussian functions, such as some machine learning algorithms (eg, adaptive neuro-fuzzy inference system [ANFIS]). A potential reason for this could be that the system is broken down into smaller pieces, where each part has its own specific and defining features. However, in ANFIS, the system was considered as a whole, and a set of features was defined for the entire system overlooking dissimilarities within the system. In addition, unlike most statistical approaches that make assumptions about the data, this method is assumption-free and is considered more robust to assumptions (eg, normality of residuals, independency of measurements). Despite its promising application to the analysis of HR and the lack of restrictive assumptions, FDT is computationally intense. This means that the model needs a high level of proficiency in understanding the mathematics and statistics behind FDT. Especially, in comparison with approaches such as ANOVA, classical time series, and mixed regression, using this approach requires higher levels of domain knowledge, for example, studies in machine learning and FDT methods (Table 3).

    Table 3. Results from example studies that used predictive models with time-independent output.
    Method and authorsDomainIndependent variablesDependent variable
    Machine learning

    		Kolus et al [87]Biomedical (energy expenditure)HRa, oxygen consumption, work rateWork rate

    		McDonald et al [11]PTSDbHR, subjective stress momentsStress moment

    		Healey et al [86]DrivingHR, HRVc, skin conductance, muscle activity, muscle tension, breathing rateTo detect stress

    		Kolus et al [88]Physical activityHR, maximum HR, oxygen consumption, body type, work rateWork rate

    		Zhang et al [92]Physical activityHR, body attitude information, body movementHR
    Fluctuation-dissipation theory

    		Chen et al [91]Health careHR recovery, blood pressure, instantaneous HRHR

    aHR: heart rate.

    bPTSD: posttraumatic stress disorder.

    cHRV: heart rate variability.

    Time-Dependent Output

    Time-Variant Modeling

    Time-variant modeling is a mathematical approach used to analyze time-dependent data sets and provide a time-dependent output. Time-variant models of HR can generate HRR measures in real time. Some studies suggest that measuring HRR in real time can especially help assess arousals and arousability in different individuals in response to mental stressors [93]. This shows promise for PTSD research given its potential to enable the comparison between the effect of internal stimuli (stressors generated through memory) and external stimuli (stressors generated from the environment) on the arousability of patients with PTSD.

    Although time-variant modeling has been replicated in the literature and has shown promise in analyzing HR data [33,94], it is computationally intense. The process of solving the equations within the model includes defining multiplex matrices for each variable, which is time and space consuming. Moreover, time-variant modeling requires large data sets of HF (eg, 100 Hz) HR data, which is often not feasible for real-time data collection instruments such as wearable devices that record continuous data for large windows of time (eg, more than 30 min).

    Nonlinear Dynamic Modeling

    Nonlinear dynamic modeling of HR consists of depicting HR as the output of a nonlinear dynamic system [95].

    Nonlinear dynamic modeling of HR can be a promising method to assess arousal patterns by measuring SNS activity [96]. Hence, this approach may be useful for analyzing PTSD hyperarousal patterns as they are associated with SNS activity. Despite the advantages of this model, it requires high-frequency HR data (eg, 100 Hz) or even instantaneous HR [96]. Instantaneous HR is an HR measure derived from HRV, which is different from raw HR measured by wearable devices. Instantaneous HR can be extracted by multiplying RR intervals (the time between two consecutive R waves of the HRV signal) by 60 and needs to be measured at an HF (>250 Hz), whereas smartwatches collect HR data with a much lower frequency (<5 Hz) [96].

    This model accounts for the natural nonlinearity and time-dependent features of HR data. In addition, the learnability and predictability of this method can help detect the onset of symptoms in patients with PTSD. A limitation of this method for characterizing PTSD aspects is the assumption of invertibility [97]. This assumption indicates that all the variable matrices used in equations are required to be invertible. In many cases, and mainly in nonlaboratory settings, this assumption cannot be met [97]. Moreover, these methods are relatively slow and more computationally intense compared with other methods such as machine learning (for both training and testing the model) because they involve solving multiple complex mathematical equations [66]. Table 4 shows examples of predictive models with time-dependent output.

    Table 4. Results from studies that used predictive models with time-dependent output.
    Method and authorsDomainIndependent variablesDependent variable
    Time variant

    		Lefever et al [94]Sports science—biomedicalHRa, participants’ input power, road gradient,HRVb

    		Olufsen et al [98]Biology, health careHR, resting HR, blood pressureHR regulations
    Nonlinear dynamic

    		Chen et al [69]Health care (patient data)Resting HR, arterial blood pressure, HR, HRVHeart beat

    		Kazmi et al [33]BiophysicsHuman normal sinus rhythm, human congestive heart rate failureHRV (they look at the correlation)

    aHR: heart rate.

    bHRV: heart rate variability.


    Descriptive Framework Based on the Summary of Findings

    We categorized the methods used to analyze HR data into 2 categories: descriptive and predictive. In the context of PTSD, descriptive models may be used to characterize PTSD triggers and the factors that affect their occurrence, whereas predictive models may be useful to predict PTSD onset to facilitate timely intervention. The extracted models provide methods for evaluating, describing, comparing, interpreting, and understanding patterns in the HR data. However, interpreting the data in a meaningful way depends on the specific objectives of the study. The data at hand can be analyzed with one or many of the reviewed models based on the goal of the study and the assumptions of the models. Each model corresponds to a distinct type of output and different interpretations of the data with different assumptions. On the basis of the process of data collection, the number of observations, and variables in the data, researchers might choose one or a combination of models provided. Table 5 provides a framework for choosing a model based on the limitations, assumptions, and features of each model and the data at hand. Furthermore, Table 5 presents the articles that used a specific method.

    Table 5. Descriptive framework for heart rate–related analysis methods extracted from the literature.
    ModelAssumptionsFeaturesLimitationsCases
    Descriptive, time-independent output

    		ANOVAa
    • Normal distribution of residuals
    • Constant variance of populations
    • Independence and identically distributed observations
    • Capable of comparing groups and looking at trends
    • Computationally simple
    • Restrictive assumptions
    • Type 1 error
    • Just applicable to linear analysis
    		[47,52-54,57-59, 65-68,99-102]

    		A first-order exponential model
    • Continuous observations
    • Observations should be identical (eg, no age, gender difference)
    • Environmental effects are constant
    • Easy to apply and learn
    • Gives higher weights to recent observations
    • Not repeated in studies
    • Higher error rates than classical time series and mixed regression
    • Does not show trends
    • Not accurate for very small and very large windows of time
    		[63]
    Descriptive, time-dependent output

    		Classical time series analysis
    • Stationary observations (constant mean values of series)
    • Advantageous for analyzing time-based trends
    • Does not require independence of data points
    • Used in the literature to analyze cardiovascular disease
    • Includes linear and nonlinear analysis
    • Requires stationary data sets
    		[33,69,70,80]

    		 Mixed regression model
    • Normality of residuals distribution
    • Accounts for differences between individuals (eg, age, gender)
    • Can be used for analyzing repeated measures
    • Can be applied to non-normal data
    • Cannot be used for nonlinear models
    		[50,66,67,75, 80-82,103-107]
    Predictive, time-independent output

    		Machine learning methods
    • Limited dependencies of the observations (each machine learning algorithm has its assumptions that need to be checked)
    • Proactive algorithm (can be used for action-reaction type of data sets)
    • Powerful predictive method
    • Rapid analysis prediction, and processing
    • Simplifies time-intensive computations
    • Can over fit or under fit data
    • Cannot be applied to data sets with highly dependent variables
    • The process has little rational explanation
    		[11,86-88,92,108-110]

    		Fluctuation-dissipation theory
    • Equilibrium system (the system and observations are not changing)
    • Powerful predictive capability
    • Does not have restrictive assumptions such as normality of residuals
    • Significantly less data needed compared with a general data fitting approach
    • Computationally intense
    • Time consuming
    		[70,91,111]
    Predictive, time-dependent output

    		Time-variant modeling
    • Requires big data sets with high-frequency data points (more than 60 Hz)
    • Can be used to describe data as well as forecasting the future
    • Computationally intense
    • Slow process
    		[33,93,94,96, 98,112-116]

    		Nonlinear dynamic modeling
    • Invertible matrices
    • Very accurate
    • Replicated multiple times in studies
    • Computationally intense
    • Slow process
    • Requires invertible matrices that is not always feasible in naturalistic settings
    		[33,66,96,98,104, 112,113,116-121]

    aANOVA: analysis of variance.

    Fit Assessment

    Fit assessment can be conducted to examine the efficiency of each method in modeling a specific dataset. Fit assessment is especially promising for comparing different methods if they are applied to the same data set. However, considering the wide range of applicable fit indices, researchers might struggle to compare them. In the category of descriptive models, R2 and adjusted R2 are the main indices of fit assessment. R2 indicates the degree of variation in the dependent variable caused by the independent variable(s). Adjusted R2 is a revised version of R2 that accounts for the number of independent variables in a model [122]. Generally, adjusted R2 is more promising than R2 as it is more robust to overfitting [122]. In the prediction methods category, a variety of measures other than R2 and adjusted R2 were used to assess the quality of fit. Some of these measures include sensitivity, specificity, accuracy, and area under the ROC curve (AUC)-ROC. Sensitivity is the number of true-positives divided by the total number of observations, and specificity is the number of true-negatives divided by the total number of observations [123]. Accuracy is the number of true predictions divided by the total number of predictions. The error rate is 1 minus the accuracy or the number of wrong detections divided by the total number of observations [124]. Finally, AUC-ROC is a curve that plots the true-positive rate (Y axis) versus the false-positive rate (X axis) to measure the performance of the model. It is important to bear in mind that fit indices are data dependent; therefore, comparisons are the best made by fitting multiple models to the same data set.

    In the statistical analysis of data in the PTSD domain, fit assessments have been used to show the efficiency of the results. For instance, McDonald et al [11] used ROC curves along with accuracy to show that random forest works better than other machine learning methods to predict hyperarousal moments in people with PTSD. Shalev et al [125] used sensitivity and specificity to predict the development of PTSD based on their instant responses to trauma. Bartels et al [63] applied adjusted R2 to assess the goodness of fit for their proposed exponential model. Examples of fit adjustments are summarized in Table 6.

    Table 6. Examples of fit assessment for different methods used in studies.
    StudyMethodVariablesFit measure
    Strath et al [65]ANOVAaHRb, oxygen intake, age, fitnessR2=0.87
    Zakeri et al [80]Classical time seriesHR, energy expenditure, accelerometer, ageR2=0.84
    McDonald et al [11]Machine learningHR, subjective stress momentsArea under receiver operating characteristics curve=0.67
    Healey et al [86]Machine learningHR, HRVc, skin conductance, muscle activity, muscle tension, breathing rateAccuracy=97%
    Chen et al [91]Fluctuation-dissipation theoryHR recovery, blood pressure, instantaneous HRError rate=25%
    Chen et al [66]Nonlinear dynamicResting HR, arterial blood pressure, HR, HRVSensitivity=0.941; predictability=0.988

    aANOVA: analysis of variance.

    bHR: heart rate.

    cHRV: heart rate variability.

    Methodological Considerations for Heart Rate Assessments

    The models identified in this review represent several promising directions for future exploration, but they also illustrate a hidden complexity in the use of HR data as model input. HR is impacted by individual characteristics including age, sex, health, resting HR, respiration, and lifestyle [24]. The maximum HR typically decreases with age. Females have higher HR levels than men [126]. Athletes have lower HR levels than sedentary people [127]. Resting HR is lower in more active people, and lower resting HRs result in lower HR levels [128]. As the respiratory system affects heart activity, studies suggest that incorporating respiration as a factor in HR models improves HR estimation significantly [78]. Lifestyle such as smoking habits affects HR as well; people who smoke have a higher HR than nonsmokers [129].

    Beyond these general characteristics, it is important to consider the type of physical activity in the analysis. Physical activity significantly affects HR [130], where high-intensity activities such as running and cycling affect HR differently from low-intensity activities such as sitting and lying down [99]. Concerns regarding activity were common in the reviewed studies, particularly in the energy expenditure domain [131]. Green et al [131] suggested that body acceleration is a reliable indicator of physical activity and should be included in all analyses as a covariate or constraint. Although activity is directly related to energy expenditure outcomes, it is also relevant for studies investigating stress. Whereas some of the reviewed studies on stress included body acceleration in their analysis [100], many neglected this factor [46,132].

    Heart Rate Assessments in Anxiety Domains

    HR data have been widely investigated in the domains of physical activity and energy expenditure. Although there are some differences between the effects of mental stress on HR and the effects of physical activity on HR, there are many similarities that make these domains connected. Physical activity affects SNS performance in the short term and PNS performance in the long term [133]. As a result, HR increases during physical activities (due to SNS activation), and resting HR is lower in athletes who have higher rates of physical activity (because of PNS performance) [133].

    Similarly, in terms of mental stress, whereas acute stress or immediate response to stressors activates SNS, chronic stress increases vagal and parasympathetic activity [134]. These similarities enable researchers in mental stress domains to employ models and pathways that are extracted in physical activity domains. For instance, one main measure that is used broadly to examine energy expenditure is HRR. This measure is an accepted indicator of SNS deactivation and PNS activation. Recovering from acute stress and arousability is also associated with the withdrawal of SNS and activation of PNS. As a result, HRR can be a proper measure to be considered in studies that examine acute stress.

    Limitations

    This scoping review attempted to include all articles that analyzed HR; however, it is still likely that some were overlooked. Furthermore, the authors categorized the HR models based on their own synthesis of the literature and relevance to PTSD. These models can be listed and categorized in a variety of ways, such as deterministic versus stochastic.

    Another limitation of this review is that although the identified models have been applied across various domains (eg, energy expenditure and general stress prediction), to our knowledge, only 2 papers [11,57] directly applied these methods to data from patients diagnosed with PTSD. In particular, only 1 study [11] used a predictive approach in the PTSD domain. Other studies were primarily limited to linear descriptive statistics such as the t test or ANOVA [60,65–67]. These methods are valid for making inferences about PTSD and comparing their effects on HR among different groups. However, there is a need for additional studies in this area that explore a broader set of predictive models and other factors (eg, activity level) that have not been analyzed with descriptive models.

    Beyond the specific application of these models to PTSD, there are several more general challenges. The reviewed research often proceeded independently, with few links between the various studies. This diversity makes comparisons across studies difficult. Studies have used different data sets with different variables based on individual goals. Furthermore, the reviewed work often focused on testing 1 specific model rather than a broad comparison. Often critical details, such as the model and parameter selection process, were not reported in the articles. Another critical detail often not addressed in the reviewed studies was the mismatch between the model requirements and the sampling rates, which may result in conditions such as overfitting [135].

    Collectively, these limitations suggest a need for substantial additional work in modeling the relationship between HR and PTSD. Future studies should consider comparisons between several models, analyze or explicitly discuss decisions made throughout the modeling process, and comprehensively document their HR data collection. As future studies are conducted that enact these criteria, the utility of the modeling approaches identified here will become clearer, and the path to more effective PTSD treatments will become more attainable.

    Conclusions

    The goals of this review were to identify and characterize quantitative HR models for relevant applications in PTSD. One of the gaps in this area is the absence of a framework that researchers can use before, during, and after their data collection to choose a method to analyze HR data. In this regard, we developed a descriptive framework that can be used to determine the method to apply to HR data to achieve more efficient results. We identified 4 broad categories of methods: descriptive time-independent output, descriptive time-dependent output, predictive time-independent output, and predictive time-dependent output. Descriptive time-independent output models include ANOVA and first-order exponential, whereas descriptive time-dependent output models include classical time series analysis and mixed regression. Predictive time-independent output models include machine learning methods and analysis of HR-based FDT. Finally, predictive time-independent output models include the time-variant method and nonlinear dynamic modeling.

    All of the identified modeling categories have relevance in PTSD, although modeling selection is highly dependent on the specific goals of the modeler. For instance, one might use ANOVA to examine the differences in resting HR in individuals with PTSD versus without PTSD [54].

    Acknowledgments

    The authors would like to acknowledge Ms Margaret Foster, a systematic review expert librarian at the Texas A&M University system, who helped to develop the search strategy.

    Conflicts of Interest

    None declared.

    1. Gleichauf ML. Meta-analysis of heart rate variability as a psychophysiological indicator of posttraumatic stress disorder. J Trauma Treat. 2013;3(1):1000182. [CrossRef]
    2. Spivak B, Segal M, Mester R, Weizman A. Lateral preference in post-traumatic stress disorder. Psychol Med. Jan 1998;28(1):229-232. [CrossRef] [Medline]
    3. LeBouthillier DM, McMillan KA, Thibodeau MA, Asmundson GJ. Types and number of traumas associated with suicidal ideation and suicide attempts in PTSD: findings from a US nationally representative sample. J Trauma Stress. Jun 2015;28(3):183-190. [CrossRef] [Medline]
    4. Resnick HS, Kilpatrick DG, Dansky BS, Saunders BE, Best CL. Prevalence of civilian trauma and posttraumatic stress disorder in a representative national sample of women. J Consult Clin Psychol. Dec 1993;61(6):984-991. [CrossRef] [Medline]
    5. Richardson LK, Frueh BC, Acierno R. Prevalence estimates of combat-related post-traumatic stress disorder: critical review. Aust N Z J Psychiatry. Jan 2010;44(1):4-19. [FREE Full text] [CrossRef] [Medline]
    6. Moon J, Smith A, Sasangohar F, Benzer JK, Kum H. A descriptive model of the current PTSD care system: identifying opportunities for improvement. Proc Int Symp Hum Factors Ergon Health Care. May 15, 2017;6(1):251. [CrossRef]
    7. Reisman M. PTSD treatment for veterans: what's working, what's new, and what's next. Pharm Ther. Oct 2016;41(10):623-634. [FREE Full text] [Medline]
    8. Rodriguez-Paras C, Tippey K, Brown E, Sasangohar F, Creech S, Kum H, et al. Posttraumatic stress disorder and mobile health: app investigation and scoping literature review. JMIR Mhealth Uhealth. Oct 26, 2017;5(10):e156. [FREE Full text] [CrossRef] [Medline]
    9. Khusid MA, Vythilingam M. The emerging role of mindfulness meditation as effective self-management strategy, part 1: clinical implications for depression, post-traumatic stress disorder, and anxiety. Mil Med. Sep 2016;181(9):961-968. [CrossRef] [Medline]
    10. Galea S, Basham K, Culpepper L, Davidson J, Foa E, Kizer K, et al. Treatment for Posttraumatic Stress Disorder in Military and Veteran Populations: Initial Assessment. Washington, DC. National Academies Press; 2012.
    11. McDonald AD, Sasangohar F, Jatav A, Rao AH. Continuous monitoring and detection of post-traumatic stress disorder (PTSD) triggers among veterans: a supervised machine learning approach. IISE Trans Healthc Syst Eng. Jun 11, 2019;9(3):201-211. [CrossRef]
    12. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. Aug 18, 2009;151(4):264-9, W64. [CrossRef] [Medline]
    13. Tricco AC, Lillie E, Zarin W, O'Brien K, Colquhoun H, Kastner M, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. Feb 9, 2016;16(1):15. [FREE Full text] [CrossRef] [Medline]
    14. Acharya UR, Joseph KP, Kannathal N, Lim CM, Suri JS. Heart rate variability: a review. Med Biol Eng Comput. Dec 2006;44(12):1031-1051. [CrossRef] [Medline]
    15. Cohen H, Benjamin J, Geva AB, Matar MA, Kaplan Z, Kotler M. Autonomic dysregulation in panic disorder and in post-traumatic stress disorder: application of power spectrum analysis of heart rate variability at rest and in response to recollection of trauma or panic attacks. Psychiatry Res. Sep 25, 2000;96(1):1-13. [CrossRef] [Medline]
    16. Ginsberg J, Ayers E, Burriss L, Powell D. Discriminative delay Pavlovian eyeblink conditioning in veterans with and without posttraumatic stress disorder. J Anxiety Disord. Jun 2008;22(5):809-823. [CrossRef] [Medline]
    17. Ginzburg K, Ein-Dor T, Solomon Z. Comorbidity of posttraumatic stress disorder, anxiety and depression: a 20-year longitudinal study of war veterans. J Affect Disord. Jun 2010;123(1-3):249-257. [CrossRef] [Medline]
    18. Tan G, Dao TK, Farmer L, Sutherland RJ, Gevirtz R. Heart rate variability (HRV) and posttraumatic stress disorder (PTSD): a pilot study. Appl Psychophysiol Biofeedback. Mar 2011;36(1):27-35. [CrossRef] [Medline]
    19. Kamath M, Fallen E. Power spectral analysis of heart rate variability: a noninvasive signature of cardiac autonomic function. Crit Rev Biomed Eng. 1993;21(3):245-311. [Medline]
    20. Prins A, Kaloupek D, Keane T. Psychophysiological evidence for autonomic arousal and startle in traumatized adult populations. In: Friedman MJ, Charney DS, Deutch AY, editors. Neurobiological and Clinical Consequences of Stress: From Normal Adaptation to Post-traumatic Stress Disorder. Pennsylvania, USA. Lippincott Williams & Wilkins Publishers; 1995:291-314.
    21. Hynynen E, Uusitalo A, Konttinen N, Rusko H. Heart rate variability during night sleep and after awakening in overtrained athletes. Med Sci Sports Exerc. Feb 2006;38(2):313-317. [CrossRef] [Medline]
    22. Buchheit M, Simon C, Charloux A, Doutreleau S, Piquard F, Brandenberger G. Relationship between very high physical activity energy expenditure, heart rate variability and self-estimate of health status in middle-aged individuals. Int J Sports Med. Sep 2006;27(9):697-701. [CrossRef] [Medline]
    23. Weber CS, Thayer JF, Rudat M, Sharma AM, Perschel FH, Buchholz K, et al. Salt-sensitive men show reduced heart rate variability, lower norepinephrine and enhanced cortisol during mental stress. J Hum Hypertens. Jun 2008;22(6):423-431. [CrossRef] [Medline]
    24. Shaffer F, Ginsberg JP. An overview of heart rate variability metrics and norms. Front Public Health. 2017;5:258. [FREE Full text] [CrossRef] [Medline]
    25. Wahbeh H, Oken BS. Peak high-frequency HRV and peak alpha frequency higher in PTSD. Appl Psychophysiol Biofeedback. Mar 2013;38(1):57-69. [FREE Full text] [CrossRef] [Medline]
    26. Pyne JM, Constans JI, Wiederhold MD, Gibson DP, Kimbrell T, Kramer TL, et al. Heart rate variability: pre-deployment predictor of post-deployment PTSD symptoms. Biol Psychol. Dec 2016;121(Pt A):91-98. [FREE Full text] [CrossRef] [Medline]
    27. Gillie BL, Thayer JF. Individual differences in resting heart rate variability and cognitive control in posttraumatic stress disorder. Front Psychol. 2014;5:758. [FREE Full text] [CrossRef] [Medline]
    28. Mellman TA, Pigeon WR, Nowell PD, Nolan B. Relationships between REM sleep findings and PTSD symptoms during the early aftermath of trauma. J Trauma Stress. Oct 2007;20(5):893-901. [CrossRef] [Medline]
    29. Lee SM, Han H, Jang K, Huh S, Huh HJ, Joo J, et al. Heart rate variability associated with posttraumatic stress disorder in victims' families of Sewol ferry disaster. Psychiatry Res. Jan 2018;259:277-282. [CrossRef] [Medline]
    30. Minassian A, Maihofer AX, Baker DG, Nievergelt CM, Geyer MA, Risbrough VB, et al. Marine Resiliency Study Team. Association of predeployment heart rate variability with risk of postdeployment posttraumatic stress disorder in active-duty marines. JAMA Psychiatry. Oct 2015;72(10):979-986. [CrossRef] [Medline]
    31. Park JE, Lee JY, Kang S, Choi JH, Kim TY, So HS, et al. Heart rate variability of chronic posttraumatic stress disorder in the Korean veterans. Psychiatry Res. Sep 2017;255:72-77. [CrossRef] [Medline]
    32. Monfredi O, Lyashkov AE, Johnsen A, Inada S, Schneider H, Wang R, et al. Biophysical characterization of the underappreciated and important relationship between heart rate variability and heart rate. Hypertension. Dec 2014;64(6):1334-1343. [FREE Full text] [CrossRef] [Medline]
    33. Kazmi SZ, Zhang H, Aziz W, Monfredi O, Abbas SA, Shah SA, et al. Inverse correlation between heart rate variability and heart rate demonstrated by linear and nonlinear analysis. PLoS One. 2016;11(6):e0157557. [FREE Full text] [CrossRef] [Medline]
    34. Stein PK, Domitrovich PP, Hui N, Rautaharju P, Gottdiener J. Sometimes higher heart rate variability is not better heart rate variability: results of graphical and nonlinear analyses. J Cardiovasc Electrophysiol. Sep 2005;16(9):954-959. [CrossRef] [Medline]
    35. Centre for Urban Design and Mental Health. URL: https://www.urbandesignmentalhealth.com/how-to-measure-mental-health.html [accessed 2019-11-19]
    36. Bonnemeier H, Richardt G, Potratz J, Wiegand UK, Brandes A, Kluge N, et al. Circadian profile of cardiac autonomic nervous modulation in healthy subjects: differing effects of aging and gender on heart rate variability. J Cardiovasc Electrophysiol. Aug 2003;14(8):791-799. [CrossRef] [Medline]
    37. Almeida-Santos MA, Barreto-Filho JA, Oliveira JL, Reis FP, da Cunha OC, Sousa AC. Aging, heart rate variability and patterns of autonomic regulation of the heart. Arch Gerontol Geriatr. 2016;63:1-8. [CrossRef] [Medline]
    38. Agelink MW, Boz C, Ullrich H, Andrich J. Relationship between major depression and heart rate variability. Clinical consequences and implications for antidepressive treatment. Psychiatry Res. Dec 15, 2002;113(1-2):139-149. [CrossRef] [Medline]
    39. Liao D, Cai J, Brancati FL, Folsom A, Barnes RW, Tyroler HA, et al. Association of vagal tone with serum insulin, glucose, and diabetes mellitus-the ARIC study. Diabetes Res Clin Pract. Dec 1995;30(3):211-221. [CrossRef] [Medline]
    40. Koenig J, Thayer JF. Sex differences in healthy human heart rate variability: a meta-analysis. Neurosci Biobehav Rev. May 2016;64:288-310. [CrossRef] [Medline]
    41. Zhang D, Shen X, Qi X. Resting heart rate and all-cause and cardiovascular mortality in the general population: a meta-analysis. Can Med Assoc J. Feb 16, 2016;188(3):E53-E63. [FREE Full text] [CrossRef] [Medline]
    42. Sammito S, Böckelmann I. Factors influencing heart rate variability. Int Cardiovasc Forum J. May 4, 2016;6:-. [CrossRef]
    43. Umetani K, Singer DH, McCraty R, Atkinson M. Twenty-four hour time domain heart rate variability and heart rate: relations to age and gender over nine decades. J Am Coll Cardiol. Mar 1, 1998;31(3):593-601. [FREE Full text] [CrossRef] [Medline]
    44. Albright TD, Jessell TM, Kandel ER, Posner MI. Neural science. Cell. Feb 2000;100:1-55. [CrossRef]
    45. Mathias C, Bannister R. Autonomic Failure: A Textbook of Clinical Disorders of the Autonomic Nervous System. Oxford, UK. Oxford University Press; 2013.
    46. Shalev AY, Sahar T, Freedman S, Peri T, Glick N, Brandes D, et al. A prospective study of heart rate response following trauma and the subsequent development of posttraumatic stress disorder. Arch Gen Psychiatry. Jun 1998;55(6):553-559. [CrossRef] [Medline]
    47. Pole N, Cumberbatch E, Taylor WM, Metzler TJ, Marmar CR, Neylan TC. Comparisons between high and low peritraumatic dissociators in cardiovascular and emotional activity while remembering trauma. J Trauma Dissociation. 2005;6(4):51-67. [CrossRef] [Medline]
    48. Bryant RA, Creamer M, O'Donnell M, Silove D, McFarlane AC. A multisite study of initial respiration rate and heart rate as predictors of posttraumatic stress disorder. J Clin Psychiatry. Nov 2008;69(11):1694-1701. [CrossRef] [Medline]
    49. Beckham JC, Feldman ME, Barefoot JC, Fairbank JA, Helms MJ, Haney TL, et al. Ambulatory cardiovascular activity in Vietnam combat veterans with and without posttraumatic stress disorder. J Consult Clin Psychol. Apr 2000;68(2):269-276. [CrossRef] [Medline]
    50. Kannel WB, Kannel C, Paffenbarger RS, Cupples L. Heart rate and cardiovascular mortality: the Framingham study. Am Heart J. Jun 1987;113(6):1489-1494. [CrossRef] [Medline]
    51. Woodward SH, Arsenault NJ, Voelker K, Nguyen T, Lynch J, Skultety K, et al. Autonomic activation during sleep in posttraumatic stress disorder and panic: a mattress actigraphic study. Biol Psychiatry. Jul 1, 2009;66(1):41-46. [FREE Full text] [CrossRef] [Medline]
    52. Buckley TC, Holohan D, Greif JL, Bedard M, Suvak M. Twenty-four-hour ambulatory assessment of heart rate and blood pressure in chronic PTSD and non-PTSD veterans. J Trauma Stress. Apr 2004;17(2):163-171. [CrossRef] [Medline]
    53. Orr SP, Pitman RK, Lasko NB, Herz LR. Psychophysiological assessment of posttraumatic stress disorder imagery in world war II and Korean combat veterans. J Abnorm Psychol. Feb 1993;102(1):152-159. [CrossRef] [Medline]
    54. Roy M, Costanzo M, Jovanovic T, Leaman S, Taylor P, Norrholm S, et al. Heart rate response to fear conditioning and virtual reality in subthreshold PTSD. Stud Health Technol Inform. 2013;191:115-119. [Medline]
    55. Halligan SL, Michael T, Wilhelm FH, Clark DM, Ehlers A. Reduced heart rate responding to trauma reliving in trauma survivors with PTSD: correlates and consequences. J Trauma Stress. Oct 2006;19(5):721-734. [CrossRef] [Medline]
    56. Tabachnick BG, Fidell LS. Experimental Designs Using ANOVA. California, USA. Brooks/Cole; 2011.
    57. Shalev AY, Freedman S, Peri T, Brandes D, Sahar T, Orr SP, et al. Prospective study of posttraumatic stress disorder and depression following trauma. Am J Psychiatry. May 1998;155(5):630-637. [CrossRef] [Medline]
    58. Foa EB, Rothbaum BO, Riggs DS, Murdock TB. Treatment of posttraumatic stress disorder in rape victims: a comparison between cognitive-behavioral procedures and counseling. J Consult Clin Psychol. Oct 1991;59(5):715-723. [CrossRef] [Medline]
    59. Gelpin E, Bonne O, Peri T, Brandes D, Shalev A. Treatment of recent trauma survivors with benzodiazepines: a prospective study. J Clin Psychiatry. Sep 1996;57(9):390-394. [Medline]
    60. Cacioppo J, Tassinary L, Berntson G. Handbook of Psychophysiology. Cambridge, USA. Cambridge University Press; 2007.
    61. Hardy GH. Properties of logarithmico-exponential functions. Proc Lond Math Soc. Dec 23, 2016;s2-10(1):54-90. [CrossRef]
    62. Marquardt DW. An algorithm for least-squares estimation of nonlinear parameters. J Soc Ind Appl Math. Jun 1963;11(2):431-441. [CrossRef]
    63. Bartels-Ferreira R, de Sousa ED, Trevizani GA, Silva LP, Nakamura FY, Forjaz CL, et al. Can a first-order exponential decay model fit heart rate recovery after resistance exercise? Clin Physiol Funct Imaging. Mar 2015;35(2):98-103. [CrossRef] [Medline]
    64. Lipov E. Post traumatic stress disorder (PTSD) as an over activation of sympathetic nervous system: an alternative view. J Trauma Treat. 2013;3(1):1222. [CrossRef]
    65. Strath SJ, Swartz AM, Bassett DR, O'Brien WL, King GA, Ainsworth BE. Evaluation of heart rate as a method for assessing moderate intensity physical activity. Med Sci Sports Exerc. Sep 2000;32(9 Suppl):S465-S470. [CrossRef] [Medline]
    66. Romero-Ugalde HM, Garnotel M, Doron M, Jallon P, Charpentier G, Franc S, et al. An original piecewise model for computing energy expenditure from accelerometer and heart rate signals. Physiol Meas. Jul 28, 2017;38(8):1599-1615. [CrossRef] [Medline]
    67. Khoueiry Z, Roubille C, Nagot N, Lattuca B, Piot C, Leclercq F, et al. Could heart rate play a role in pericardial inflammation? Med Hypotheses. Oct 2012;79(4):512-515. [CrossRef] [Medline]
    68. Tonhajzerova I, Ondrejka I, Chladekova L, Farsky I, Visnovcova Z, Calkovska A, et al. Heart rate time irreversibility is impaired in adolescent major depression. Prog Neuropsychopharmacol Biol Psychiatry. Oct 1, 2012;39(1):212-217. [CrossRef] [Medline]
    69. Chen H, Erol Y, Shen E, Russell S. Probabilistic model-based approach for heart beat detection. Physiol Meas. Sep 2016;37(9):1404-1421. [CrossRef] [Medline]
    70. Peng C, Havlin S, Stanley HE, Goldberger AL. Quantification of scaling exponents and crossover phenomena in nonstationary heartbeat time series. Chaos. 1995;5(1):82-87. [CrossRef] [Medline]
    71. Cohen H, Kotler M, Matar MA, Kaplan Z, Loewenthal U, Miodownik H, et al. Analysis of heart rate variability in posttraumatic stress disorder patients in response to a trauma-related reminder. Biol Psychiatry. Nov 15, 1998;44(10):1054-1059. [CrossRef] [Medline]
    72. Kantz H, Schreiber T. Nonlinear Time Series Analysis. Cambridge, USA. Cambridge University Press; 2004.
    73. Montgomery DC, Johnson LA, Gardiner JS. Forecasting And Time Series Analysis. New York, USA. Mcgraw-Hill; 1990.
    74. Galwey NW. Introduction to Mixed Modelling: Beyond Regression and Analysis of Variance. New York, USA. Wiley; 2014.
    75. Gee A, Barbieri R, Paydarfar D, Indic P. Improving heart rate estimation in preterm infants with bivariate point process analysis of heart rate and respiration. Conf Proc IEEE Eng Med Biol Soc. Aug 2016;2016:920-923. [CrossRef] [Medline]
    76. Boscarino JA. A prospective study of PTSD and early-age heart disease mortality among Vietnam veterans: implications for surveillance and prevention. Psychosom Med. Jul 2008;70(6):668-676. [FREE Full text] [CrossRef] [Medline]
    77. Kassam-Adams N, Garcia-España JF, Fein JA, Winston FK. Heart rate and posttraumatic stress in injured children. Arch Gen Psychiatry. Mar 2005;62(3):335-340. [CrossRef] [Medline]
    78. Darlington RB. Regression and Linear Models. New York, USA. Mcgraw-Hill; 1990.
    79. Greenacre M. Correspondence Analysis in Practice. Milton Park, Canada. Chapman & Hall/CRC; 2017.
    80. Zakeri I, Adolph A, Puyau M, Vohra F, Butte N. Cross-sectional time series and multivariate adaptive regression splines models using accelerometry and heart rate predict energy expenditure of preschoolers. J Nutr. Jan 2013;143(1):114-122. [FREE Full text] [CrossRef] [Medline]
    81. Bonomi A, Goldenberg S, Papini G, Kraal J, Stut W, Sartor F, et al. Predicting Energy Expenditure From Photo-Plethysmographic Measurements of Heart Rate Under Beta Blocker Therapy: Data Driven Personalization Strategies Based on Mixed Models. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2015. Presented at: EMBC'15; August 25-29, 2015; Milan, Italy. [CrossRef]
    82. Xu Z, Zong C, Jafari R. Constructing energy expenditure regression model using heart rate with reduced training time. Conf Proc IEEE Eng Med Biol Soc. 2015;2015:6566-6569. [CrossRef] [Medline]
    83. James G, Witten D, Hastie T, Tibshirani R. An Introduction to Statistical Learning: with Applications in R. New York, USA. Springer; 2013.
    84. Sano A, Picard R. Stress Recognition Using Wearable Sensors and Mobile Phones. In: Proceedings of the Humaine Association Conference on Affective Computing and Intelligent Interaction. 2013. Presented at: ACII'13; September 2-5, 2013; Geneva, Switzerland. [CrossRef]
    85. Thayer JF, Ahs F, Fredrikson M, Sollers JJ, Wager TD. A meta-analysis of heart rate variability and neuroimaging studies: implications for heart rate variability as a marker of stress and health. Neurosci Biobehav Rev. Feb 2012;36(2):747-756. [CrossRef] [Medline]
    86. Healey J, Picard R. Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transport Syst. Jun 2005;6(2):156-166. [CrossRef]
    87. Kolus A, Dubé PA, Imbeau D, Labib R, Dubeau D. Estimating oxygen consumption from heart rate using adaptive neuro-fuzzy inference system and analytical approaches. Appl Ergon. Nov 2014;45(6):1475-1483. [CrossRef] [Medline]
    88. Kolus A, Imbeau D, Dubé PA, Dubeau D. Classifying work rate from heart rate measurements using an adaptive neuro-fuzzy inference system. Appl Ergon. May 2016;54:158-168. [CrossRef] [Medline]
    89. Michie D, Spiegelhalter D, Taylor C. Machine Learning, Neural and Statistical Classification. Upper Saddle River, NJ. Prentice Hall; 1994.
    90. Kubo R. The fluctuation-dissipation theorem. Rep Prog Phys. Aug 5, 2002;29(1):255-284. [CrossRef]
    91. Chen M, Niestemski LR, Prevost R, McRae M, Cholleti S, Najarro G, et al. Prediction of heart rate response to conclusion of the spontaneous breathing trial by fluctuation dissipation theory. Phys Biol. Feb 2013;10(1):016006. [FREE Full text] [CrossRef] [Medline]
    92. Zhang Y, Chen W, Su SW, Celler B. Nonlinear modelling and control for heart rate response to exercise. Int J Bioinform Res Appl. 2012;8(5-6):397-416. [CrossRef] [Medline]
    93. Roger D, Jamieson J. Individual differences in delayed heart-rate recovery following stress: the role of extraversion, neuroticism and emotional control. Personal Individ Differ. Jan 1988;9(4):721-726. [CrossRef]
    94. Lefever J, Berckmans D, Aerts J. Time-variant modelling of heart rate responses to exercise intensity during road cycling. Eur J Sport Sci. 2014;14(Suppl 1):S406-S412. [CrossRef] [Medline]
    95. Haber R, Unbehauen H. Structure identification of nonlinear dynamic systems—a survey on input/output approaches. Automatica. Jul 1990;26(4):651-677. [CrossRef]
    96. Valenza G, Lanata A, Scilingo EP. The role of nonlinear dynamics in affective valence and arousal recognition. IEEE Trans Affective Comput. Apr 2012;3(2):237-249. [CrossRef]
    97. Langford J, Salakhutdinov R, Zhang T. Learning Nonlinear Dynamic Models. In: Proceedings of the 26th Annual International Conference on Machine Learning. 2009. Presented at: ICML'09; June 14-18, 2009; Montreal, Canada. [CrossRef]
    98. Olufsen MS, Ottesen JT. A practical approach to parameter estimation applied to model predicting heart rate regulation. J Math Biol. Jul 2013;67(1):39-68. [FREE Full text] [CrossRef] [Medline]
    99. Boulay MR, Simoneau JA, Lortie G, Bouchard C. Monitoring high-intensity endurance exercise with heart rate and thresholds. Med Sci Sports Exerc. Jan 1997;29(1):125-132. [CrossRef] [Medline]
    100. Vrijkotte TG, van Doornen LJ, de Geus EJ. Effects of work stress on ambulatory blood pressure, heart rate, and heart rate variability. Hypertension. Apr 2000;35(4):880-886. [CrossRef] [Medline]
    101. Feng J, Huang Z, Zhou C, Ye X. Study of continuous blood pressure estimation based on pulse transit time, heart rate and photoplethysmography-derived hemodynamic covariates. Australas Phys Eng Sci Med. Jun 2018;41(2):403-413. [CrossRef] [Medline]
    102. Champéroux P, Fesler P, Judé S, Richard S, le Guennec JY, Thireau J. High-frequency autonomic modulation: a new model for analysis of autonomic cardiac control. Br J Pharmacol. Aug 2018;175(15):3131-3143. [FREE Full text] [CrossRef] [Medline]
    103. Diderichsen PM, Cox E, Martin SW, Cleton A, Ribbing J. Predicted heart rate effect of inhaled PF-00610355, a long acting β-adrenoceptor agonist, in volunteers and patients with chronic obstructive pulmonary disease. Br J Clin Pharmacol. Nov 2013;76(5):752-762. [FREE Full text] [CrossRef] [Medline]
    104. Hoyer D, Nowack S, Bauer S, Tetschke F, Rudolph A, Wallwitz U, et al. Fetal development of complex autonomic control evaluated from multiscale heart rate patterns. Am J Physiol Regul Integr Comp Physiol. Mar 1, 2013;304(5):R383-R392. [FREE Full text] [CrossRef] [Medline]
    105. Alrefaie M, Summerskill S, Jackon T. In a heart beat: using driver's physiological changes to determine the quality of a takeover in highly automated vehicles. Accid Anal Prev. Oct 2019;131:180-190. [CrossRef] [Medline]
    106. Oliveira M, Marçôa R, Moutinho J, Oliveira P, Ladeira I, Lima R, et al. Reference equations for the 6-minute walk distance in healthy Portuguese subjects 18-70 years old. Pulmonology. 2019;25(2):83-89. [FREE Full text] [CrossRef] [Medline]
    107. Sartipy U, Savarese G, Dahlström U, Fu M, Lund LH. Association of heart rate with mortality in sinus rhythm and atrial fibrillation in heart failure with preserved ejection fraction. Eur J Heart Fail. Apr 2019;21(4):471-479. [FREE Full text] [CrossRef] [Medline]
    108. Ni J, Muhlstein L, McAuley J. Modeling Heart Rate and Activity Data for Personalized Fitness Recommendation. In: Proceedings of the The World Wide Web Conference. 2019. Presented at: WWW'19; May 13-17, 2019; San Francisco, USA. [CrossRef]
    109. Signorini MG, Pini N, Malovini A, Bellazzi R, Magenes G. Integrating machine learning techniques and physiology based heart rate features for antepartum fetal monitoring. Comput Methods Programs Biomed. Mar 2020;185:105015. [CrossRef] [Medline]
    110. Chaudhuri T, Soh YC, Li H, Xie L. Machine learning driven personal comfort prediction by wearable sensing of pulse rate and skin temperature. Build Environ Elsevier. Mar 2020;170:106615. [CrossRef]
    111. Lu Y, Burykin A, Deem MW, Buchman TG. Predicting clinical physiology: a Markov chain model of heart rate recovery after spontaneous breathing trials in mechanically ventilated patients. J Crit Care. Sep 2009;24(3):347-361. [CrossRef] [Medline]
    112. Valenza G, Citi L, Barbieri R. Estimation of instantaneous complex dynamics through Lyapunov exponents: a study on heartbeat dynamics. PLoS One. 2014;9(8):e105622. [FREE Full text] [CrossRef] [Medline]
    113. Ferrer E, Helm JL. Dynamical systems modeling of physiological coregulation in dyadic interactions. Int J Psychophysiol. Jun 2013;88(3):296-308. [CrossRef] [Medline]
    114. Valenza G, Lanatà A, Scilingo EP. Oscillations of heart rate and respiration synchronize during affective visual stimulation. IEEE Trans Inf Technol Biomed. Jul 2012;16(4):683-690. [CrossRef] [Medline]
    115. Zazula D. Optimization of Heartbeat Detection Based on Clustering and Multimethod Approach. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012. Presented at: EMBC'12; August 28-September 1, 2012; San Diego, CA, USA. [CrossRef]
    116. Echeverría JC, Álvarez-Ramírez J, Peña MA, Rodríguez E, Gaitán MJ, González-Camarena R. Fractal and nonlinear changes in the long-term baseline fluctuations of fetal heart rate. Med Eng Phys. May 2012;34(4):466-471. [CrossRef] [Medline]
    117. Park Y, Ryu K, Shim S, Hoh J, Park M. Comparison of fetal heart rate patterns using nonlinear dynamics in breech versus cephalic presentation at term. Early Hum Dev. Feb 2013;89(2):101-106. [CrossRef] [Medline]
    118. Scalzi S, Tomei P, Verrelli CM. Nonlinear control techniques for the heart rate regulation in treadmill exercises. IEEE Trans Biomed Eng. Mar 2012;59(3):599-603. [CrossRef] [Medline]
    119. Cheng T, Savkin A, Celler B, Su S, Wang L. Nonlinear modeling and control of human heart rate response during exercise with various work load intensities. IEEE Trans Biomed Eng. Nov 2008;55(11):2499-2508. [CrossRef] [Medline]
    120. Mazzoleni MJ, Battaglini CL, Martin KJ, Coffman EM, Mann BP. Modeling and predicting heart rate dynamics across a broad range of transient exercise intensities during cycling. Sports Eng. Jan 19, 2016;19(2):117-127. [CrossRef]
    121. Zakynthinaki MS. Modelling heart rate kinetics. PLoS One. 2015;10(4):e0118263. [FREE Full text] [CrossRef] [Medline]
    122. Gelman A, Hill J. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge, USA. Cambridge University Press; 2006.
    123. Harrington P. Machine Learning In Action. Shelter Island, USA. Manning Publications; 2012.
    124. D'Agostino RB. Goodness-of-Fit-Techniques. New York, USA. CRC Press; 1986.
    125. Shalev AY, Peri T, Canetti L, Schreiber S. Predictors of PTSD in injured trauma survivors: a prospective study. Am J Psychiatry. Feb 1996;153(2):219-225. [CrossRef] [Medline]
    126. Magder SA. The ups and downs of heart rate. Crit Care Med. Jan 2012;40(1):239-245. [CrossRef] [Medline]
    127. Lester M, Sheffield L, Trammell P, Reeves T. The effect of age and athletic training on the maximal heart rate during muscular exercise. Am Heart J. Sep 1968;76(3):370-376. [CrossRef] [Medline]
    128. Sacknoff DM, Gleim GW, Stachenfeld N, Coplan NL. Effect of athletic training on heart rate variability. Am Heart J. May 1994;127(5):1275-1278. [CrossRef] [Medline]
    129. Dietrich DF, Schwartz J, Schindler C, Gaspoz J, Barthélémy JC, Tschopp J, et al. SAPALDIA-Team. Effects of passive smoking on heart rate variability, heart rate and blood pressure: an observational study. Int J Epidemiol. Aug 2007;36(4):834-840. [CrossRef] [Medline]
    130. Freedson PS, Miller K. Objective monitoring of physical activity using motion sensors and heart rate. Res Q Exerc Sport. Jun 2000;71(Suppl 2):21-29. [CrossRef] [Medline]
    131. Green JA, Halsey LG, Wilson RP, Frappell PB. Estimating energy expenditure of animals using the accelerometry technique: activity, inactivity and comparison with the heart-rate technique. J Exp Biol. Feb 2009;212(Pt 4):471-482. [FREE Full text] [CrossRef] [Medline]
    132. Taelman J, Vandeput S, Spaepen A, Van HS. Influence of Mental Stress on Heart Rate and Heart Rate Variability. In: Proceedings of the 4th European Conference of the International Federation for Medical and Biological Engineering. 2008. Presented at: ECIFMBE'18; November 23-27, 2008; Antwerp, Belgium. [CrossRef]
    133. Pagani M, Somers V, Furlan R, Dell'Orto S, Conway J, Baselli G, et al. Changes in autonomic regulation induced by physical training in mild hypertension. Hypertension. Dec 1988;12(6):600-610. [CrossRef] [Medline]
    134. McCarty R, Horwatt K, Konarska M. Chronic stress and sympathetic-adrenal medullary responsiveness. Soc Sci Med. 1988;26(3):333-341. [CrossRef] [Medline]
    135. Chen H, Simpson D, Ying Z. Infill asymptotics for a stochastic process model with measurement error. Stat Sin. 2000;10(1):141-156. [FREE Full text]

    ‎
    ANFIS: adaptive neuro-fuzzy inference system
    ANOVA: analysis of variance
    ANS: autonomic nervous system
    AUC-ROC: area under the receiver operating characteristics curve
    COH: coherence score
    FDT: fluctuation-dissipation theory
    HF: high frequency power
    HR: heart rate
    HRR: heart rate response
    HRV: heart rate variability
    LF: low frequency power
    mHealth: mobile health
    PNS: parasympathetic nervous system
    PRISMA: preferred reporting items for systematic reviews and meta-analyses
    PTSD: posttraumatic stress disorder
    RMSSD: root mean square of successive differences between normal heart beats
    SDNN: SD of the interbeat interval of normal sinus beats
    SNS: sympathetic nervous system

    Edited by J Torous; submitted 10.10.19; peer-reviewed by D Neyens, J Pyne; comments to author 01.11.19; revised version received 11.03.20; accepted 03.04.20; published 22.07.20.

    Copyright

    ©Mahnoosh Sadeghi, Farzan Sasangohar, Anthony D McDonald. Originally published in JMIR Mental Health (http://mental.jmir.org), 22.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.