Clinical Utility of Wearable Sensors and Patient-Reported Surveys in Patients With Schizophrenia: Noninterventional, Observational Study

Introduction

Background

Psychotic symptom exacerbation and relapse are frequently observed in patients with schizophrenia and can lead to a decline in social functioning, reduced treatment response, and worsening of clinical outcomes [1]. These patients and their caregivers experience an increased burden because of relapse and consequent hospitalization [2-4]. Relapse in schizophrenia may be preceded by early warning signs, including depressed mood; social withdrawal; and changes in physical activities, feelings, emotions, and sleep disturbances [5]. Therefore, identifying warning signs can enable early intervention to avoid subsequent relapse events [6]. Symptom onset can be rapid; however, continuous monitoring may provide an advantage for early intervention [7].

Web-based data capturing technologies such as Information Technology–Aided Relapse Prevention in Schizophrenia have been piloted to recognize warning signs based on patient reporting of prodromal symptoms of relapse [8]. However, the frequency at which it is practical to obtain this information and the subjective nature of patients’ and caregivers’ responses pose challenges. Small, unobtrusive remote sensing devices, along with existing mobile technologies, make it possible to capture real-time data on patients’ activities, sleep patterns, behaviors, and symptoms. More recent studies have indicated the general availability and acceptability of devices for remote assessment and management [9-11]. Smartphones are commonly used and have multiple embedded sensors (eg, accelerometer, microphone, GPS, and camera). These can be leveraged to collect symptom reports through patient-reported outcome (PRO) surveys and to collect passive data to measure changes in behavior [9,12-19]. A consumer wrist-worn smartwatch or fitness band can additionally provide measurements of precise and objective activity patterns spanning sleep-rest and active-awake periods. These devices have an advantage of generating continuous streaming data that are more reproducible and less obtrusive than relying on patient and caregiver reports alone. In addition, changes in device compliance may itself be a signal and indicate a clinically relevant change in behavior [9,17].

Study Objectives

This clinical study was designed to explore the signatures of relapse. However, because patients were mainly recruited from outpatient clinics and followed up for 4 months—a short period to observe relapses in stabilized patients—there were insufficient relapses to perform the primary objective. We subsequently evaluated the feasibility of using wearable devices (singly and in combination) and self-reporting technologies to identify potentially predictive symptom correlates in patients with schizophrenia or schizoaffective disorder who are at increased risk of relapse. Continuous monitoring using wearable devices (eg, fitness bands and smartwatches) and self-reporting via smartphones were used in this study, and predictive modeling was applied to examine the correlations between clinical assessments and aggregated metrics data.

Methods

Overview

This study was conducted at the University of Alabama, Birmingham (UAB), from August 8, 2015, to March 28, 2016. The protocol was approved by the UAB Institutional Review Board, and all patients or legally authorized representatives provided written informed consent and Health Insurance Portability and Accountability Act authorization before the start of the study.

Patients

Men and women (aged ≥19 years) who met the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), criteria for schizophrenia or schizoaffective disorder diagnosed based on the Structured Clinical Interview for DSM-5-Text Revision Axis I Disorders using the Diagnostic Interview for Genetic Studies-4.0 were included in this study. The target population included patients discharged from the inpatient psychiatry unit, emergency department, or outpatient clinics of UAB who were maintained on a stable dose of antipsychotic medication that remained unchanged for 2 weeks before the start of the study.

Patients were excluded if they had physical or clinical disabilities or both, such as hearing, vision, or motor impairment, leading to difficulties in operating a smartphone or responding to prompts (determined using a demonstration smartphone for screening); severe substance use disorder (≥6 symptoms) according to DSM-5 Level 2–Substance Use–Adult scale (adapted from the National Institute on Drug Abuse-Modified Alcohol, Smoking, and Substance Involvement Screening Test); or if they were enrolled or planning to enroll in an interventional study for the treatment or prevention of worsening of symptoms of schizophrenia.

Study Design

This was a noninterventional, observational, exploratory clinical study in which patients were allowed to continue with their usual standard of care and antipsychotic treatment as prescribed by their physician. Patients were screened for eligibility for up to 2 weeks. Enrolled patients were observed for relapse for 4 months (approximately 120 days; observation or study participation period), followed by a 30-day poststudy safety reporting period (Figure 1). Patients were considered to have experienced relapse if they had a rating of moderately severe, very severe, or extremely severe (item score ≥5)—in the previous 2 weeks—for ≥1 item on the Positive and Negative Syndrome Scale (PANSS) positive subscale (items P1-P7) or ≥2 items on the negative subscale (items N1-N7) [20] or if they had symptom exacerbation (increased PANSS total score) that required a change in antipsychotic medication or upward dosage adjustment.

‎View this figure

Wearable remote sensing devices and a smartphone were provided to eligible patients for use during the observation period. All patients had to undergo a training tutorial for the devices and smartphone based on their individual learning needs. Repeated practice was performed until patients were comfortable using these items. The importance of refraining from tampering or attempting to deactivate the devices was conveyed to the patients. The confidence of patients with using the devices was assessed, and retraining was performed, if required. Potential predictors of symptom worsening or relapse (eg, sleep quality) were collected using remote sensing devices, and the results were subsequently compared with standard clinical assessments. Patients did not have access to the activity and sleep data generated by the wearable devices, so that behavior was independent of feedback.

The 30-day poststudy safety reporting period was per Janssen Adverse Event Reporting Requirements for Noninterventional Studies, wherein all adverse events were recorded from the first use of the Janssen products and for 30 days after the last use of these Janssen drug products within the study.

Clinical Assessments

During the observation period, clinical assessments were performed for symptom worsening and relapse identification. PANSS and modified Brief Psychiatric Rating Scale (BPRS) [21] were assessed every 2 weeks (biweekly), whereas Clinical Global Impression-Schizophrenia (CGI-SCH) [22], Calgary Depression Scale (CDS) [23], Young Mania Rating Scale [24], and electronic medical records were assessed every 4 weeks. Physical activity was assessed using the Yale Physical Activity Survey (YPAS) [25], and sleep was monitored using the Pittsburgh Sleep Quality Index (PSQI) [26] every 4 weeks. Physicians completed the CGI-SCH [22] severity scale for each patient, and a trained study coordinator administered the PANSS to patients. Clinical assessments performed at baseline for patient characterization included psychosocial functioning (assessed using the Social Functioning Scale [27]) or quality of life, cognitive functioning, impulsivity, and measure of addiction. Patient self-reports were also used to assess symptom status and relapse.

Device Data Collection and Processing

General Information

The study was monitored according to the sponsor’s current standard operating procedures for the monitoring of clinical trials, and activities were implemented to ensure proper operational study oversight. These activities focused on identifying and resolving operational and quality issues to ensure data integrity, protocol compliance, and safety of the study participants. Written instructions were provided for collecting source documentation, which was reviewed for accuracy and completeness by the sponsor during on-site monitoring visits and underwent internal data reviews throughout the study and at the time of database lock. Discrepancies were resolved with the investigator or designees, as appropriate.

The UAB clinical site captured all clinical assessments using source documents, and these data were entered into the REDCap (Research Electronic Data Capture) system. The nature and location of all source documents were identified to ensure that all sources of original data required to complete data collection were known to the sponsor or investigator and study site personnel.

Smartwatches or fitness bands can monitor activity based on the time frame, duration, and intensity of movement. The unprocessed data collected from wearable devices (eg, accelerometry) may not directly represent variables or metrics that are amenable to patient-relevant interpretation or traditional prespecified statistical measures. The raw data are reduced by the device manufacturers to metrics representing the average state of an individual during specified periods (eg, steps taken, time spent resting or sleeping, and activity intensity levels). These behavioral and lifestyle measures provided a set of metrics that were tested independently and collectively as a pattern toward the specified aims. Analytics can be derived from the patterns that are normal (baseline) for a given patient to detect any relevant changes during clinical follow-up. For sleep, domains that could be assessed through remote sensing devices include onset time, duration, and quality; frequency and pattern of sleep disruptions can be monitored by the number and duration of movements. Similarly for activity, steps per day (mobility) and patterns of daily activity—distribution of high-, medium-, and low-intensity activities—can be assessed. Mobile phone reporting of clinically relevant metrics such as medication compliance; well-being; and the degree of symptom experience, such as seeing or hearing things, could also be assessed.

Device data and PRO responses collected between clinical visits were aggregated to test correlations with the clinical visits and outcomes. Compliance using the devices was also monitored (defined as using or wearing either all devices for 50% of the time in 24 hours or 2 of 4 devices during the 4-month observation period).

Garmin Vivofit Fitness Band

Garmin vivofit is a wristband with an easy-to-read display that was worn at all times by patients to track ambulatory activity (number of steps taken every 15 minutes). A single device was dispensed at visit 1 with instructions for use. At every visit to the clinic, data from the device were downloaded onto the site computer and stored in a secure location. The Garmin vivofit device is a consumer-grade fitness device. Consumer devices have the benefit of higher user acceptance as a social norm. The disadvantage is that the measurements are not clinically validated, and specific firmware versions need to be tracked, as they can impact the results. Despite these challenges, if the intended outcome is detecting relative changes in behavior for individuals rather than cross-sectional studies, they can provide some utility.

Philips Actiwatch Spectrum

The Philips Actiwatch is a wristwatch with an actigraphy system for tracking objective data on off-wrist status, sleep-wake, activity count, and light exposure. Data were collected in 30-second epochs. This actiwatch was worn 24×7, is designed for clinical trials and populations, and is well established for actigraphy-based sleep assessments [28]. A single wristwatch was dispensed to patients at visit 1. At every visit to the clinic, data from the device were downloaded from a proprietary docking station supplied by Philips.

As per the study protocol, patients were requested to wear both Philips Actiwatch and Garmin vivofit devices at all times without additional guidance on the arm preference.

Smartphone

Smartphone apps were used in the study to collect PROs consisting of self-assessment and symptom-tracking questionnaires. One set of questions was given every 2 days (bidaily), and the other set of questions was given weekly throughout the observation period. The Ginger app was used at the beginning of the study for collection, but patients were migrated to the UAB REDCap PRO survey system. The REDCap system collected the same information at the same frequency as the originally deployed Ginger app but had the advantage of direct capture in the clinical database rather than needing to capture the data periodically through a third-party upload. The reason for the switch was that the duration of the trial recruitment period exceeded that of the contracted services with Ginger. A smooth transition was easily developed and implemented, and no data were lost.

Patients who had personal smartphones were asked to download the Ginger app. The site provided a smartphone if the patient did not own one. Patients were restricted to the use of only one personal smartphone with the Ginger app during the course of the study. To prevent the erroneous collection of nonpatient data, the use of the smartphone was restricted to the patient only and was not to be shared with others.

In addition, patients were instructed on the completion of UAB REDCap PRO surveys via an email set up for each patient by the UAB. The email contained a link for access to the survey. Responses to the survey were collected within REDCap through a secure web-based app module to manage and build the web-based surveys.

Both Ginger and REDCap data were recorded on the Health Insurance Portability and Accountability Act–compliant UAB server. At the end of the study, all patients were asked to return their wristwatches. Patients completing the study were allowed to keep the site-issued smartphone, but the monthly plan was terminated. Patients did not have access to the data generated during the course of the study.

Analytical Methods

Statistical Analysis

The sample size for the study was estimated based on previous experience, assuming that 35% to 40% of patients may experience a full significant exacerbation of psychotic symptoms or relapse during the course of the study. The level of relapse anticipated within 120 days for patients treated for a current relapse was based on the experience of UAB in their clinics. For patient characteristics, all continuous variables were summarized using descriptive statistics, and the categorical variables were summarized using frequency measures. As the analyses were exploratory in nature, a two-sided significance level of 5% was used, unless specified otherwise, for all statistical tests. Multiplicity adjustments were not made for the analysis.

Without significant clinical changes to detect or model, we assessed the within-patient stability of the clinical scales using the intraclass correlation coefficient (ICC), a metric commonly used in psychometrics to assess the test-retest and interrater reliability. If the patient-level variance is small for stable patients, it is easier to detect potentially important deviations from individualized norms. The ICC was calculated using a mixed-effect model, in which patient was included as a random effect.

Table 1 summarizes all the metrics obtained from the devices. These metrics were aggregated biweekly to generate the feature sets for predictive modeling. Means and SDs of the device metrics during the 2 weeks immediately before the corresponding clinical assessment were calculated. The summary statistics of all device metrics were combined as the feature set for models predictive of clinical assessments.

Associations among the clinical symptoms and between clinical symptoms and device variables were assessed using mixed-effect models. For each pair, one variable was considered as the response variable and the other variable was considered as the independent variable. Patient was included in the model as a random effect to account for correlations among the repeated measures. Each data variable was scaled so that each had a variance of 1. Testing whether the dependent and independent variables are correlated is equivalent to testing whether the coefficient of the independent variable is 0.

Elastic net [29] was applied to build models using linear regression to predict the clinical assessments and the patient-reported activity and sleep scales using feature sets constructed from the device data. Ten-fold repeated cross-validation with 30 repeats was also applied to train the models and assess the performance of the model. The repeated cross-validation was conducted at the patient level, where data on 10% of patients were held out for validation. The performance of the predictive models was assessed using the root mean square error and the R². The R² was calculated as the difference between the total sum of squares and the sum of squares owing to the error sum of squares divided by the total sum of squares. When the R² is calculated based on cross-validation results and the sum of squares owing to error is calculated based on out-of-sample predictions, the estimated R² may be negative.

Detecting Statistically Relevant Change in Streaming Data

There is evidence that detectable signs of relapse may be specific to an individual [17]. As such, it is important to develop metrics that can detect statistically relevant changes based on a person’s own data rather than what may be normal for individuals. Metrics derived from the devices or PRO can be monitored as a real-time streaming process similar to process control in manufacturing or even the stock market. A stable longitudinal period can be used to establish a baseline norm (mean and variance) for each metric of interest. At any subsequent point, statistical tests can be performed for point outlier status of the daily tracked measures or if the short-term mean and variance of a defined measurement period are statistically different from the baseline measures, indicating that a trend may be occurring. Outliers and trends may have causal explanations other than relapse; however, detected changes create an opportunity to have conversations with the patient about behaviors, with objective data. For example, in Figure 2, examination of sleep onset data revealed a significant trend toward earlier sleep times as the patient approached relapse. The average sleep onset time at baseline was 8 PM. Near relapse, the data trended to a 2-week average around 6:30 PM.

It is relatively easy to detect statistically relevant changes in the streaming metrics. The challenge is assigning causality to the change without an understanding of the context [13,16,17]. Most detected changes are probably not relevant to relapse but could be used in a clinical setting for follow-up or conversations with caregivers based on these objective data.

‎View this figure

Results

Patient Disposition and Baseline Characteristics

A total of 49 patients were screened, and 40 patients were enrolled in the study. In total, 9 participants were screened out because of loss to follow-up after consent (n=4) and after the screening visit (n=1), consent withdrawal (n=1), screen failure—lack of proof of diagnosis (n=1) and no reason (n=1), and unknown—screening visit not being performed (n=1).

Of these 40 patients, 31 (78%) completed the study and 9 (23%) discontinued the study (protocol violations: 18%, 7/40 and withdrawal of consent: 5%, 2/40; Figure 1). Only 8% (3/40) of patients experienced a relapse during the study, of which only 1 patient had sufficient postscreening data to establish a reference baseline. The enrolled patients had a higher proportion of men (25/40; 63%), a median age of 40.3 years, and a mean baseline BMI of 34.9 kg/m². Most patients (38/40;95%) were on at least one antipsychotic medication during the course of the study, and the most common medication was Risperdal Consta (28/40; 70%). All patients continued their prescribed medication throughout the study (Table 2).

Clinical Assessments

For the patient experiencing a relapse, the CGI disease severity scores worsened from 2 to 5 (borderline mentally to markedly ill), and the global improvement changed from 4 to 6 (no change to much worse). The PANSS domains that worsened at relapse included positive subscale items (delusions-P1 and conceptual disorganization-P2), negative subscale items (poor rapport-N3), and general psychopathology scale items (somatic concern-G1, anxiety-G2, tension-G4, depression-G6, uncooperative-G8, lack of judgment-G12, and disturbance of volition-G13). The BPRS scale worsened in the subdomains of somatic concern, anxiety, conceptual disorganization, tension, depressive mood, uncooperative, and disorientation. The Young Mania Rating Scale worsened in the subdomains of sleep, language, thought disorder, and insight, and the CDS worsened in the subdomains of depression and early wakening.

Correlations Among Clinical Symptoms

For the 40 enrolled patients, the within-patient stability of the clinical symptom scales as assessed by ICC showed either excellent or good agreement, suggesting that most of the assessed symptoms were stable throughout the study (Multimedia Appendix 1). When assessing the association among clinical symptoms, the PANSS total score and the BPRS total scores showed a very strong correlation (r=0.97). However, the Clinical Global Impression-Severity scale score did not strongly correlate with the PANSS and BPRS scores (Table 3).

Device-Based Assessments

With 1 patient having a stable baseline before relapse, it is not possible to infer statistical indicators from the data that might have suggested an impending potential relapse. However, a few anecdotal observations have been made. Before relapse, the day-to-day variation in mobility was very high, ranging from 1 to 7 miles per day, as captured by the Garmin vivofit fitness band. Some disrupted sleeping was captured using the Philips Actiwatch. The patient reported a spike in the feeling of suspicion and missed his medication dose at this time.

Data Coverage and Patient Compliance

Across all patients, the range of days each device was in use provided the maximum number of observable days for each patient-device combination. On the basis of the combined patient data, a high percentage of data coverage and compliance was observed for each device. Using the Garmin vivofit, Philips Actiwatch Spectrum, PRO bidaily survey, and PRO weekly survey, the data coverage was 96%, 92%, 80%, and 89%, respectively, and the device compliance was 97%, 94%, 82%, and 88%, respectively.

Correlations Between Clinical Symptoms and Device Metrics

The low variability in clinical symptoms prevented assessments of individual-based changes in device data with clinical status changes. Subsequently, an analysis was performed to directly correlate the aggregated device metrics with related clinical measures. Correlations were observed between the Philips Actigraph sleep features and the PSQI sleep duration component. Features such as sleep duration, time spent sleeping, and time of valid rest significantly correlated with PSQI sleep duration (r=0.36, r=0.36, and r=0.34, respectively). Sleep start time and resting-state time showed more modest but significant correlations with PSQI sleep disturbances (r=−0.26 and r=−0.25, respectively; Table 4).

Correlations were observed between the Philips Actigraph activity features and the YPAS individual indices (Table 5). Garmin activity feature—total steps taken—showed a modest correlation with the BPRS total score (r=−0.23; P=.03). Garmin activity metrics also showed correlations with the YPAS global or individual indices (Tables 6 and 7).

A significant but moderate association between survey data features and psychiatric symptoms was observed. The bidaily survey summary score correlated with the BPRS total score (r=0.23; P=.05) and the CDS total score (r=0.37; P=.01). The weekly survey summary score was also associated with the PANSS total score (r=0.30; P=.03), the BPRS total score (r=0.29; P=.01), and the CDS total score (r=0.37; P=.01). Specific queries such as feel down or depressed, feel confused or have trouble with your thinking, feel stressed or overwhelmed, and have trouble getting things done correlated with the CDS total score (Table 8).

In the elastic net model, survey data (bidaily, weekly, and bidaily+weekly) were observed to be predictive of the PANSS total score, the BPRS total score, and the CDS total score (Table 9). The features derived from the survey data were found to be predictive of the total score, subjective sleep quality component, and sleep disturbance component of the PSQI (Table 4). The features from Philips Actigraph data were also found to be predictive of the sleep latency component of the PSQI. The sleep and activity features derived from the Philips Actigraph and Garmin activity data were observed to be predictors of the sleep duration component of the PSQI (Table 10) and the sitting index of the YPAS (Multimedia Appendix 2).

Discussion

Principal Findings

With the recent increase in the use of mobile technologies (smartphones) and remote sensing wearable devices (smartwatches) with inbuilt sensors, continuous tracking of activity, sleep, and self-reported parameters has been explored widely in various psychiatric disorders [9,12,13,17,19,30-33]. This study demonstrates that parameters that are potentially relevant to the detection of relapse, such as changes in activity and sleep [34], and self-reported symptoms can be collected using smart devices in patients with schizophrenia or schizoaffective disorder.

Several studies have shown that the vast majority of patients with schizophrenia report sleep abnormalities, which often tend to occur before the onset of illness and can predict an acute exacerbation of psychotic symptoms [35]. Individuals with schizophrenia also report significant changes in activity before a relapse episode [36,37]. Therefore, continuous sleep, activity level, and self-reported metrics provide an objective and time-stamped record of changes in behavior that may be clinically relevant and can become a part of the clinical dialog. Although there may be a potential in using remote sensing technologies to track physical activity and sleep behavior to detect relapses, patient adherence and acceptance can limit the effectiveness of these apps. Individuals with schizophrenia may often refuse to use such devices because of a lack of familiarity with the technologies, with using the device, and with the disease state (eg, paranoid behavior). In this study, high data coverage and compliance indicated the overall acceptance of all remote sensing mobile devices used. Our experience in this study suggests that the involvement of patients’ physician, good training, and clear communication related to the research goals and potential benefits and risks of the study are important factors in ensuring good compliance and engagement.

In this study, correlations among the individual clinical symptoms as well as between clinical symptoms and device measures were evaluated. The assessment of within-patient stability of the clinical scales showed that most patients’ symptoms were stable throughout the study. The correlation analysis among clinical symptoms indicated that the PANSS total score and the BPRS total score showed a strong correlation. Patient-reported survey data features were correlated with clinical symptoms.

In this study, an elastic net model was used to predict correlations between clinical assessments and the device metrics. The Philips Actigraph and the Garmin activity data showed some utility in predicting the sitting index of the YPAS. The features derived from the patient-reported survey data were useful in predicting the total score, the subjective sleep quality component, and the sleep disturbance component of the PSQI. Features derived from the Philips Actigraph and Garmin vivofit were predictive of the sleep duration component of the PSQI. The implementation of technology-based measures along with metrics-based prediction methods could be effective in instituting early warning systems for symptom decline and relapse patterns in schizophrenia.

Limitations

With limited relapses or clinical changes, this study was not sufficiently powered to identify predictors of relapse or symptom exacerbation by comparing clinical and technological assessments. Although a higher relapse rate was expected based on historical rates in the clinic, we speculate that there may have been a recruitment bias for this study. Less stable subjects may have been less interested in participation. In addition, the frequent clinical assessments and questionnaires and the presence of the devices would have generated a heightened personal focus on relevant symptoms. Future trial designs should consider tracking recruitment bias in the design. In general, expanding the study to more patients and a longer duration would be required to develop robust models predictive of clinical change. The longitudinal stability of the patients would facilitate, however, the ability to detect relative changes and outliers when they occur. Although detecting change is facilitated by having a long-term stable baseline, patients that are unstable will inherently have more variability. Higher variability can make the detection of event outliers more difficult. Having continuous data, however, allows statistical averaging for longer periods than single-point estimates. Trends and changes in variability can be detected with streaming data, even in unstable subjects, more readily than with point estimates from clinical visits. This study demonstrates that remote sensing devices and mobile technology can be used to monitor metrics that are relevant for patients with schizophrenia. It will be important to capture the context of detected changes in future studies to determine the metrics most predictive of relapse or symptom exacerbation.

Conclusions

In summary, relapse prediction using remote sensing technologies may aid clinicians to be cautioned in advance to detect the approaching exacerbation of symptoms and patterns of relapse in patients with schizophrenia. The operational learnings from this study provide insights to conduct future studies with remote sensing devices in this patient population to devise earlier intervention strategies.