This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Recognition and timely action around “warning signs” of illness exacerbation is central to the self-management of bipolar disorder. Due to its heterogeneity and fluctuating course, passive and active mobile technologies have been increasingly evaluated as adjunctive or standalone tools to predict and prevent risk of worsening of course in bipolar disorder. As predictive analytics approaches to big data from mobile health (mHealth) applications and ancillary sensors advance, it is likely that early warning systems will increasingly become available to patients. Such systems could reduce the amount of time spent experiencing symptoms and diminish the immense disability experienced by people with bipolar disorder. However, in addition to the challenges in validating such systems, we argue that early warning systems may not be without harms. Probabilistic warnings may be delivered to individuals who may not be able to interpret the warning, have limited information about what behaviors to change, or are unprepared to or cannot feasibly act due to time or logistic constraints. We propose five essential elements for early warning systems and provide a conceptual framework for designing, incorporating stakeholder input, and validating early warning systems for bipolar disorder with a focus on pragmatic considerations.
The potential for technology to facilitate “early warning systems” for bipolar disorder has been described for several decades [
A promise of high frequency data collection agents, such as mobile health (mHealth) technologies, is that potential warning signs and outcomes can be prospectively and concurrently monitored. Moreover, predictions about the near future could be more accurate if based in part on accumulating knowledge about a given patient, rather than upon more static risk factors that frequently fail to predict near-term trajectories for a given individual. A variety of passive and active technologies have been piloted in bipolar disorder to this end [
Within a high frequency self-monitoring framework, an early warning system might gather high frequency data on both predictors (eg, sleep/wake cycle) and outcomes (eg, onset of mood episodes), identify when changes in sleep/wake cycles that previously heralded a mood episode occurrence, and subsequently deliver an alert or intervention targeted to the early warning sign in a timely fashion. Existing applications of early warning systems are already part of daily life (eg, credit card fraud monitoring). Despite the potential of early warning systems, there are a number of challenges to validation and also potential harms. Problems may arise, both if early warning systems produce incorrect predictions and lead to unnecessary distress or resource inefficiency, as well as if predictions are accurate but patients or other stakeholders are unprepared or ill-equipped to act on warnings. This paper proposes a framework for developing, validating, and implementing early warning systems with technologies that collect intensive longitudinal data in bipolar disorder.
We propose a conceptual framework for design elements of an early warning system. The following are the five elements of an early warning system (
Proposed components of an early warning system, selected techniques, and research gaps.
Component | Selected resources | Research gaps |
Platform | Mobile phone apps; text messaging; home-based telehealth; wearables | Best practices for long-term adherence and engagement; effective integration of multiple platforms; user preferences and methods for granular control of transmitted information |
Inputs and outcomes | Patient reports of mood and related risk factors; passive activity/location sensing; passive metadata sensing; serial physiological sensors | Predictive validity of near future and rare events; optimal data capture frequency and duration; interpretability of passive data for warning systems |
Predictive analytics | Linear and non-linear models for intensive longitudinal data; machine learning; within-sample and out of sample validation techniques | Integration of within-person and between-person data to inform predictions; integration of high dimensional variable frequency data; utility of non-linear, complex models in practicable early warning systems; validation metric criteria |
Decision rules | Clinically important thresholds; empirically defined thresholds based on classification models; recursive analyses to define earliest detectable change in risk at threshold | Developing interpretable decision rules based on multiple inputs or interactions; updating decision rules based on accumulating data within patients |
Feedback | Multiple communication platforms with which to alert stakeholders; elements of evidence-based behavioral change content developed for risk factor self-management in bipolar disorder | Optimization of feedback messaging to enhance self-efficacy and health protective behavior; identification of potential patient and other stakeholder’s experience of adverse impact of forewarnings; research methods for quantifying the impact of individual feedback strategies; impact of feedback messaging tailoring by mood state, patient preference, and/or severity of risk |
In considering the elements in
For the purpose of early warning systems, key requirements for platforms would be the availability of online statistical analysis as data is accumulated, which typically and essentially if data from other individual’s are used in prediction models, would involve transmission of data from the device to a server for online analytics. Efficiency of transmission and interoperability of information sources gathered from the device or multiple devices remains a challenge that is especially pressing. Many of the electronic self-monitoring tools for bipolar disorder reviewed by Faurholt-Jepsen et al [
Since data is being transferred, which could include personally identifying information (eg, phone numbers, location) as well as information about symptoms and other potentially sensitive information, mHealth platforms add a risk of loss of confidentiality. A recent review found that 75% of commercially available apps purported to assist with self-managing bipolar disorder did not include a privacy policy [
Potential inputs to early warning sign systems are far ranging and include behaviors (eg, social behaviors, substance use, changes in activity), sleep (eg, number of hours, quality), stressors, medication adherence, and affective states (eg, anxiety, irritability). These inputs measured with the device need to be validated as convergent with gold standard clinical ratings. To date, most validation studies in bipolar disorder using mobile devices have focused on the validity of mood ratings, and thus focus on the validity of measurement of the typical outcome of early warning systems rather than predictors. As reviewed recently, information technology strategies for self-monitoring indicate positive indication of the concurrent validity of aggregated momentary self-ratings of depressive mood states (although less consistently with manic symptoms) captured on mobile devices with clinician-rated data [
There are proof of concept data suggesting that passive sensor data obtained through actigraphy can individuate bipolar disorder from patients with depression and healthy controls [
Taken together, these early stage studies indicate that mobile devices could be feasibly used to monitor at least some of the potential inputs and symptom outcomes over time. Challenges for inputs and outcomes in early warning systems are that mHealth technologies create data that is complex and unique from typical panel-type data, explained in terms of its high volume, velocity, and variety [
Moreover, some critically important outcomes and potential new data sources would seem highly relevant to early warning systems but have received little research. In particular, suicidal ideation and risk of self-harm has been effectively assessed over time in non-bipolar samples with attention to affective states prior to thoughts [
Prediction models link input data and subsequent outcomes. A variety of emerging methods are used to model intensive longitudinal data and predictive analytics applied to these complex data that have varying rates and structures. Techniques such as data mining, machine learning, and probabilistic modeling have been employed to make predictions about the future. While a discussion of these individual techniques is beyond the scope of this paper (see [
Irrespective of the statistical technique applied, validation of an early warning sign system would center on the accuracy and preventative utility of prediction. There are several steps beyond identifying a group-level association between T-1 predictor and T-0 outcome. In terms of raising the clinical utility of predictions, the standard for gauging the usefulness of prediction would additionally include (1) how accurate the prediction is for a given individual; (2) how interpretable the predictions are in the formation of decision rules; and (3) how much lead-time is provided in which alter the course of the predictor, if alterable.
Complex time series models have been applied to understanding the dynamics of mood course in bipolar disorder, focusing on the potential for non-linear chaotic or latent approaches to modeling state shifts in the context of intra-individual noise [
Such models gauge the accuracy of predictions in samples with and without predicted outcomes. However, this conflates potential individual differences in the predictor-outcome relationship. In repeated measures designs, case-crossover analyses can examine the association between predictors and outcomes within patients when outcomes occurred and earlier or later times when outcomes were present [
As an example, Thompson et al [
With validated data and prediction rules, a combination of clinical acumen, patient preferences, and data driven models will be necessary to create decision rules that can guide clinical interventions. While some of these decision rules may be more straightforward, such as a link between cessation of a medication and risk of switch into mania, others rules will be more complex. For example, considering insomnia as a symptom of bipolar disorder, applying the right intervention at the right time and for the right patient will likely be a personal, dynamic, and varied response with no clear-cut point or binary decision. Such decision rules can be developed empirically and a first step will require determining how sensitive and specific the input is in predicting subsequent outcomes, such as by use of a penalized regression technique. Accuracy can be judged by a metric (eg, area under the curve, AUC) generated by leave-one-out validation, and can be compared to baseline estimates of risk (eg, most recent suicidal ideation rating), and/or relative to prior windows of time that did not result in suicidal ideation, as in the case-crossover method. If sensitive and specific to the criteria listed above, sensitivity analyses to determine when prior to the outcome the earliest detectable increase in the outcome occurs (eg, examining the accuracy of prediction by censoring predictors within the span of 1-3 weeks prior to the outcome), based on an assumption is that time-varying predictors increase in accuracy the closer in time to the outcome. The identification of the time of earliest detection would be determined by comparison of accuracy when moving a stable window of information back in time to when the prediction accuracy falls below a clinical metric of accuracy (eg, AUC < 0.80) or a patient-preferred tolerance. To generate cut points, regression tree methods can then be used to identify the levels of the input that would be associated with the best fitting model (ie, with the largest AUC). These cut points can then be used to form decision rules and updated with machine learning algorithms in real-time, and made more precise to the individual weighting information from the individual, relying increasingly on individual data over that from other individuals’ data. Multiple inputs and interactions among inputs may also create even more powerful models, yet with the tradeoff of greater complexity in implementation, and more importantly, diminishing interpretability toward targeted feedback, described next.
The empirical understanding of best practices in the feedback component of early warning systems is in its infancy. Since early warning systems will involve communication around a future probabilistic risk, there is substantial evidence to suggest that people may variably interpret or misinterpret risk communication [
Feedback from early warning systems may be more effective if it extends beyond simply notifying patients, clinicians, and stakeholders of an impending risk. There is substantial evidence that to change behavior in response to a future risk, the arousal of fear or threat is insufficient and possibility counterproductive, particularly among people with low self-efficacy to make changes [
There are several potential approaches to enhancing the effectiveness of warnings derived from the broader health psychology literature [
Designing the feedback element of early warning systems should involve the incorporation of stakeholder perspectives, both patients and clinicians, which can occur at multiple levels. At a broader level of intervention
Within early warning systems, there will be practical challenges to understanding the impact of various feedback strategies. As with all prevention research, it is challenging to determine whether specific elements of early warnings systems have an effect at the individual level, given that it is impossible to know if future threats would have occurred without such a warning. As such, understanding the patient and message variables of effective risk communication using prediction models may be best advanced in experimental studies with proxy measures of target behaviors rather than episodes, at least in initial development stages.
Much work is required to make early warning systems accurate, useful, and safe for people with bipolar disorder and their care teams. Nonetheless, there have been dramatic advances in technology-based data capture, statistical prediction analysis, and risk communication that together form the ingredients of a variety of early warning systems for bipolar disorder. Many such systems have been proposed and are in the proof-of-concept stages of development, and soon will be available to consumers and clinicians. These systems may make it possible for patients to better understand and manage bipolar disorder, avoid or forestall illness exacerbations, and minimize disruptions in social and productive roles associated with illness exacerbations.
We have described a basic framework for designing interventions alongside patients and clinicians, and validating and evaluating such systems. In particular, we caution that early warning systems must empower patients to make changes rather than to simply sound alarms. We hope that this paper stimulates future development in this exciting area.
area under the curve
mobile health
John Torous is editor-in-chief of JMIR Mental Health, but was not involved in the decision making process related to this paper. The peer-review process was handled by a different editor.