This paper is in the following e-collection/theme issue:

Autism Spectrum Disorder (ASD) (96) Ecological Momentary Assessment (EMA) (293) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1296)

Published on in Vol 6, No 3 (2019): March

Caregiver Daily Reporting of Symptoms in Autism Spectrum Disorder: Observational Study Using Web and Mobile Apps

Caregiver Daily Reporting of Symptoms in Autism Spectrum Disorder: Observational Study Using Web and Mobile Apps

Caregiver Daily Reporting of Symptoms in Autism Spectrum Disorder: Observational Study Using Web and Mobile Apps

Authors of this article:

Abigail Bangerter1 Author Orcid Image ;   Nikolay V. Manyakov2 Author Orcid Image ;   David Lewin3 Author Orcid Image ;   Matthew Boice1 Author Orcid Image ;   Andrew Skalkin4 Author Orcid Image ;   Shyla Jagannatha5 Author Orcid Image ;   Meenakshi Chatterjee6 Author Orcid Image ;   Geraldine Dawson7 Author Orcid Image ;   Matthew S Goodwin8 Author Orcid Image ;   Robert Hendren9 Author Orcid Image ;   Bennett Leventhal9 Author Orcid Image ;   Frederick Shic10 Author Orcid Image ;   Seth Ness1 Author Orcid Image ;   Gahan Pandina1 Author Orcid Image
Article Authors Cited by (18) Tweetations (11) Metrics

    Original Paper

    1Neuroscience Therapeutic Area, Janssen Research & Development, LLC, Titusville, NJ, United States

    2Computational Biology, Discovery Sciences, Janssen Research & Development, Beerse, Belgium

    3Clinical Biostatistics, Janssen Research & Development, LLC, Titusville, NJ, United States

    4Informatics, Janssen Research & Development, LLC, Spring House, PA, United States

    5Statistical Decision Sciences, Janssen Research & Development, LLC, Titusville, NJ, United States

    6Computational Biology, Discovery Sciences, Janssen Research & Development, LLC, Spring House, PA, United States

    7Duke Center for Autism and Brain Development, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, United States

    8Department of Health Sciences, Northeastern University, Boston, MA, United States

    9Department of Psychiatry, School of Medicine, University of California, San Francisco, San Francisco, CA, United States

    10Center for Child Health, Behavior and Development, Seattle Children’s Research Institute, Seattle, WA, United States

    Corresponding Author:

    Abigail Bangerter, DEdPsy

    Neuroscience Therapeutic Area

    Janssen Research & Development, LLC

    1125 Trenton-Harbourton Road

    Titusville, NJ, 08560

    United States

    Phone: 1 609 213 7311

    Fax:1 609 730 2069

    Email: abangert@its.jnj.com


    Background: Currently, no medications are approved to treat core symptoms of autism spectrum disorder (ASD). One barrier to ASD medication development is the lack of validated outcome measures able to detect symptom change. Current ASD interventions are often evaluated using retrospective caregiver reports that describe general clinical presentation but often require recall of specific behaviors weeks after they occur, potentially reducing accuracy of the ratings. My JAKE, a mobile and Web-based mobile health (mHealth) app that is part of the Janssen Autism Knowledge Engine—a dynamically updated clinical research system—was designed to help caregivers of individuals with ASD to continuously log symptoms, record treatments, and track progress, to mitigate difficulties associated with retrospective reporting.

    Objective: My JAKE was deployed in an exploratory, noninterventional clinical trial to evaluate its utility and acceptability to monitor clinical outcomes in ASD. Hypotheses regarding relationships among daily tracking of symptoms, behavior, and retrospective caregiver reports were tested.

    Methods: Caregivers of individuals with ASD aged 6 years to adults (N=144) used the My JAKE app to make daily reports on their child’s sleep quality, affect, and other self-selected specific behaviors across the 8- to 10-week observational study. The results were compared with commonly used paper-and-pencil scales acquired over a concurrent period at regular 4-week intervals.

    Results: Caregiver reporting of behaviors in real time was successfully captured by My JAKE. On average, caregivers made reports 2-3 days per week across the study period. Caregivers were positive about their use of the system, with over 50% indicating that they would like to use My JAKE to track behavior outside of a clinical trial. More positive average daily reporting of overall type of day was correlated with 4 weekly reports of lower caregiver burden made at 4-week intervals (r=–0.27, P=.006, n=88) and with ASD symptoms (r=–0.42, P<.001, n=112).

    Conclusions: My JAKE reporting aligned with retrospective Web-based or paper-and-pencil scales. Use of mHealth apps, such as My JAKE, has the potential to increase the validity and accuracy of caregiver-reported outcomes and could be a useful way of identifying early changes in response to intervention. Such systems may also assist caregivers in tracking symptoms and behavior outside of a clinical trial, help with personalized goal setting, and monitoring of progress, which could collectively improve understanding of and quality of life for individuals with ASD and their families.

    Trial Registration: ClinicalTrials.gov NCT02668991; https://clinicaltrials.gov/ct2/show/NCT02668991 

    JMIR Ment Health 2019;6(3):e11365

    doi:10.2196/11365

    Keywords

    autism spectrum disorder; ecological momentary assessment; symptom assessment; mobile app; mHealth; affect; patient reported outcome measures 


    Background

    Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder with a prevalence rate of approximately 1% that is characterized by social communication impairments and restricted, repetitive patterns of behavior [1,2]. There are no approved medications for the core symptoms of ASD, which is in part related to a lack of sensitive outcome measures available for use in clinical trials evaluating potential medications [3,4]. For example, Loth et al 2016 report the failure to detect efficacy in compounds that have shown preclinical promise. The heterogeneity of the ASD population and difficulties with detection of short-term changes have led to recent attempts to develop more sensitive measures including biomarkers [5-7] and novel caregiver rating scales [8,9].

    ASD interventions are frequently evaluated using caregiver-reported measures administered during study visits [10]. Two common caregiver measurements of behavior change in ASD are the Social Responsiveness Scale-2 (SRS-2), which measures symptoms associated with ASD [11], and the Aberrant Behavior Checklist (ABC) [3], which measures general behaviors. These surveys require recall of specific behaviors over periods of time, up to 6 months in the case of the SRS-2, which can reduce accuracy of ratings. For example, it was found that caregivers remembered ASD symptoms as being worse in the past compared with current reporting [12]. In addition to problems with retrospective recall, caregivers’ reports can be subject to bias on the basis of context [13]. For example, completing a scale in a clinic context may yield different responses than those provided in settings where the behavior more typically occurs [14]. A respondent’s mood during recall can also influence responses [15]. Finally, measures completed retrospectively are commonly administered before and after an intervention, not during the intervention period, which may limit the ability to assess stability or dynamic change over time and impact detection of potential causal relationships [8,9,16]. Shorter recall periods on symptom reporting scales, for example, the Autism Impact Measure [9] (weeks) and the Autism Behavior Inventory (ABI) [8] (1 week) address this to a certain degree, but these scales could still be influenced by most recent behaviors not necessarily representative of the reporting period.

    The advent of mobile health (mHealth) technologies has enabled the development of viable alternatives to retrospective reporting that can facilitate naturalistic logging and recording of behaviors in real or near real time [17,18]. Monitoring of ASD symptoms in a naturalistic setting has the potential to improve reporting validity, leading to increased insight into the condition. Moreover, reporting on symptoms in real time holds the potential to improve accuracy and reliability by reducing episodic memory delays and recall bias [19]. In addition, related to the development of biosensor technologies, real-time observation and reports captured by mHealth have the ability to be used in combination with biometric data to enhance the understanding of the relationship of measures such as actigraphy to classify patterns, symptoms, and behaviors of ASD [20-25].

    mHealth approaches could also prove useful in recording transitory factors that reflect or impact mental or behavioral states, such as sleep [26] and caregiver affect [27], items that are more likely to change in the short term. In particular, individuals with ASD have a higher level of reported sleep [28] and psychiatric symptoms, primarily anxiety and mood disorders [29], than the general population; increased anxiety [30-32] and sleep problems [33] are associated with an increase in the severity of the core symptoms of ASD. In addition, a bidirectional relationship has been reported between sleep and internalizing and externalizing behaviors in ASD, as well as a relationship between child sleep and caregiver stress [34-37]. Dynamic measures of these changing states in ASD could lead to better understanding of their effects on core ASD symptoms and help identify more personalized and efficacious interventions.

    Paper diaries have been used in clinical trials to obtain information from participants on a daily basis. However, 1 major issue with these approaches, which is prevented with electronic reporting systems, is the reliability of the timing of the report. In 2002, Hufford et al compared the use of paper diaries with electronic reporting and found that although the completion rate was the same at 90%, only 11% of the paper diaries were completed in real time despite what participants reported about their compliance [38].

    There is also a need for more idiographic approaches than those afforded by generic scales when considering outcomes that might improve quality of life for patients and caregivers [39,40]. Specific symptoms of most concern to caregivers may not be obvious within change scores of multi-symptom scales, and caregiver-reported responsiveness to treatment may not be detected with these measures [41-43]. Assessments can be enriched by focusing on identifying and monitoring change in problems that caregivers identify as important [44]. As mHealth apps offer the potential for deep personalization, they are well-placed to provide opportunities for tracking real-world, meaningful outcomes.

    Although the use of apps to monitor and track health data is expanding [45-47], adoption of e-tech into clinical trials has been slow [48]. Within the field of ASD, there are limited publications regarding user experiences, including opinions about a wide range of features and technologies that mobile apps are capable of monitoring and tracking [49]. Although there are examples of mHealth apps for ecological momentary assessment (EMA) self-reporting in high functioning adolescents with ASD [50-52], to our knowledge, no studies of regular daily or real-time reporting by caregivers of individuals with ASD have been reported. Caregiver reporting may be particularly important for understanding concordance between self-report and caregiver report in ASD and for tracking behavior in younger children or individuals with greater intellectual impairment that might limit the ability to self-report.

    Aims

    To address ASD symptom monitoring and tracking needs, the Janssen Autism Knowledge Engine (JAKE), a dynamically-updated clinical research system, was created to provide quantifiable and reproducible measures for use in assessing treatment outcomes, potentially including detection of change in ASD symptoms and behaviors and ASD subgroup identification.

    A comprehensive pilot version of JAKE was developed that considered caregiver feedback from focus groups, review of symptoms and symptom descriptions from expert autism clinicians and beta testing [7]. My JAKE is 1 component of the JAKE system and is a Web and mobile app for use by caregivers and clinicians to log symptoms, record treatments, track progress, and gather detailed medical information. Included in My JAKE is the ABI, a Web-based, caregiver-rated scale for assessing ASD core diagnostic symptoms and associated behavior, designed specifically to provide robust and sensitive outcome measures for ASD clinical trials and other interventional studies [8].

    One aim of this study was to test the utility of My JAKE as used by caregivers of individuals with ASD who participated in a noninterventional, observational study for a duration of 8 to 10 weeks. In addition, daily ratings of behaviors were compared with the ABI core and associated symptoms of ASD and periodic, retrospectively-collected paper rating scales. Though there was an exploratory component of the study, prespecified hypotheses were created, and they are shown in Table 1.

    Table 1. Correlation hypotheses of overall type of day, sleep, and mood with autism spectrum disorder (ASD) symptoms and behaviors.
    MeasureHypothesis
    Overall type of day
    • Higher average reports of “overall type of day” will correlate negatively with increased symptoms and behaviors: Core ASD symptoms, challenging behavior, mental health, measured on the ABIa
    • Higher average reports of “overall type of day” will correlate negatively with increased reported caregiver burden on the Zarit Burden Inventory
    Sleep
    • Daily reported sleep quality will correlate negatively with sleep problems reported on the ABI
    Mood
    • Average valence reported in the mood report will correlate negatively with symptoms and behaviors reported by caregivers: ABI and ABCb; Average arousal reported in the mood report will correlate negatively with symptoms and behaviors reported by caregivers: ABI and ABC
    • Percentage of reports in each quadrant during the course of the study will correlate with symptoms and behaviors: Quadrant 1 (positive valence, positive arousal) negative correlation with behavior; Quadrant 2 (negative valence, positive arousal) positive correlation with behavior; Quadrant 3 (negative valence, negative arousal) positive correlation with behavior; Quadrant 4 (positive valence, negative arousal) no prediction made

    aABI: Autism Behavior Inventory.

    bABC: Aberrant Behavior Checklist.


    Materials

    My JAKE

    My JAKE is a Web and mobile app comprising various modules to help caregivers of individuals with ASD and their health care providers log symptoms, record participant activities, medical information and treatments, and track progress and change in behaviors. Self-report by study participants was not supported. My JAKE is accessible through most Web browsers, as well as an app for mobile devices, and it was used throughout the study. Caregivers were encouraged to use their own computer and mobile device to access My JAKE. My JAKE includes the following elements.

    Home Page

    A dashboard on the home page provides an overview of the participant’s appointments, progress, and notifications of My JAKE sections due to be completed (Figure 1). When caregivers log in to the app or the Web version of My JAKE, they will see the “things to do” list, with reminders of what needs to be completed that day.

    An external pop up reminder appears outside the app at 8 pm every evening. Clicking on the pop up will take the caregivers to the home page from where they can complete the tasks that had not already been completed for that day. The progress in completing the daily tasks is recorded in the green bar (Figure 1).

    Medical/Developmental History

    A comprehensive medical and developmental history was completed by caregivers during the screening phase of the study. Certain sections, such as developmental milestones, could be completed throughout the study.

    The Autism Behavior Inventory

    The ABI included a series of 73 items related to core and associated symptoms of ASD (Figure 2) [39]. The ABI was completed by the primary caregiver at baseline, midpoint (week 4), and endpoint. Specifically, caregivers are asked to report on the behavior observed in the past week.

    The Daily Tracker

    The Daily Tracker was used to measure sleep and overall type of day (Figure 3). It comprised several questions answered by the caregivers of ASD participants. In this study, all caregivers were asked in the morning “How was (participant name)’s sleep last night?” (Sleep) and in the evening “How was (participant name)’s day?” (Overall type of day). Caregivers could choose up to 3 additional behaviors to track. Questions could be answered by dragging a card along an 8-point scale ranging from “troubling” to “encouraging,” either in the My JAKE mobile app or through a Web browser. Picture representation of “good weather” and “bad weather” depicted extremes of the scale, and these icons were selected following extensive user testing with caregivers [7]. Text labels “troubling or encouraging” appear when the user begins to move the card on the screen (shown on right hand side image, Figure 3).

    ‎
    Figure 1. Example of the “to do list” and progress completion apps.
    ‎
    Figure 2. Sample Web-based Aberrant Behavior Checklist to capture autism spectrum disorder symptoms.
    ‎
    Figure 3. My JAKE Daily Tracker mobile interface: Sleep and overall type of day.

    Responses to these questions were tracked and displayed over a 2-week period on the website home page to provide feedback to caregivers and increase engagement in the study.

    Mood Report

    The circumplex model of affect has a long and widespread history measuring affective or emotional states, including in those with ASD [53,54]. A digital, square version of the classical affective circumplex was created for the My JAKE mobile app for this study (Figure 4). The horizontal axis represents valence, termed “mood,” and the vertical axis represents arousal, termed “activity.” The model was divided into “Quadrants” of activity and mood relationships. Caregivers were instructed to move the icon representing the participant to a location on the screen that best captured the participant mood at that moment. Caregivers were asked to do this twice a day but could use as many times as desired.

    Journal and Event Tracker

    Caregivers were able to provide ad hoc free-text entries when tracking both positive and negative events as they occurred, and they were able to track all items of interest, including sleep and diet, on a daily basis. A list of common events for ASD, includes repetitive behaviors and tantrums, allowed real-time and retrospective reporting of specific behavior, which would receive a time stamp, enabling comparison with actigraphy and other biosensor data.

    Therapy Tracker

    This module allowed tracking of participants’ medical treatments or therapies using a calendar-like interface. Caregivers could also optionally export created appointments to a calendaring system of their own choice.

    ‎
    Figure 4. Revised “circumplex” model of affect for the My JAKE mobile app.
    Connection to Microsoft Health Vault

    Microsoft HealthVault [55]—a publicly available, Class 1 electronic personal health record system—was used as the storage mechanism for all data created and accessed by My JAKE. All caregivers were required to create an account in Health Vault to enable registration with My JAKE. The use of HealthVault permitted caregivers to own and control their dependent’s study data, even after the study ended. In addition, caregivers controlled who, as well as which apps, could access their HealthVault account at any time. As an optional service to reduce caregiver burden, Zweena Health, an external company, entered select medical and developmental history data on behalf of participants into the Microsoft HealthVault account, both directly and through My JAKE.

    Standard Instruments for Measuring Change

    Caregiver reported rating scales were used to assess ASD symptom change over time. Scales were selected on the basis of their previous use in clinical trials for ASD or recommendations in reviews of scales for use in measuring change in ASD core and associated behavior [56-58].

    The ABC, a 58-item behavior rating scale, was used to measure behavior problems across the following 5 subscales: irritability, lethargy (social withdrawal), stereotypy, hyperactivity, and inappropriate speech. Items are rated on a 4-point Likert scale—ranging from 0 (not at all a problem) to 3 (the problem is severe in degree)—with higher scores indicating more severe problems. The ABC has recently been validated for use in ASD [59,60].

    The Zarit Burden Interview—Short Version (ZBI) [61]—is a scale with 22 items designed to assess psychological burden experienced by caregivers. Items ask how the caregivers feel, and responses range from 0 to 4 (never to nearly always). The ZBI has been used to assess burden among caregivers of individuals with ASD [62-68]. Using the ZBI, caregivers of individuals with ASD have been found to experience greater burden of care than other caregivers [63] and burden was related to unmet need in young people with ASD, for example, depression, anxiety, and inappropriate behavior [61,69].

    The Social Responsiveness Scale 2

    Social Responsiveness Scale 2 (SRS-2) identifies presence and severity of social impairment because of ASD. It contains 65 items intended to assess social communication and restricted and repetitive behaviors. A total of 3 forms are available, dependent on the age of the individual with ASD [11].

    Participants

    The study enrolled males and females aged ≥6 years, between 6.0 and 54.0 years of age, with a confirmed diagnosis of ASD on the basis of Diagnostic and Statistical Manual of Mental Disorders criteria. In addition, participants were required to meet research-determined cut-offs identifying a potential diagnosis of ASD according to the Autism Diagnostic Observation Schedule (ADOS-2). Participants were permitted to receive behavioral and/or pharmacologic interventions for ASD and comorbid disorders during the study, but this was not required. However, study inclusion did require participants to live with a caregiver or if not, to spend at least 3 hours a day for at least 4 days each week or at least 3 weekends a month with a caregiver.

    Institutional Review Boards approved the study protocol and its amendments. The study was conducted in accordance with the ethical principles that have their origin in the Declaration of Helsinki, consistent with Good Clinical Practices and applicable regulatory requirements. Participants, their caregivers (for participants <18 years old) or legally authorized representatives provided written informed consent before participating in the study. The study is registered at ClinicalTrials.gov (NCT02668991).

    Study Design

    This prospective, noninterventional study was conducted from July 06, 2015 to October 14, 2016 at 9 study sites in the United States. The study comprised a 14-day screening phase followed by an 8- to 10-week data-collection phase. Throughout the study, data were collected via My JAKE and additional caregiver report scales were completed on paper.

    Procedure

    Individuals with ASD and their caregivers were recruited by sites to participate in the study, which involved a screening visit and 3 additional site visits. Participants received a stipend of US $80 to $100 for attendance at each site visit, which lasted around 3 hours. No additional incentives were given for use of My JAKE between visits.

    Caregivers were registered to use My JAKE during the baseline visit at the site. The site staff demonstrated the My JAKE app and described requirements for study completion. Caregivers were also provided with a “quick guide” explaining the system and when and how to use it.

    The My JAKE home page included an interactive “to do list,” which presented and linked required components (Figure 2). Once a required assessment or measure within My JAKE was completed, it was “grayed out” on the list, and progress toward daily completion was updated. A pop-up reminder outside the app appeared daily at 8 pm. After first completing the ABI at the baseline visit, caregivers could select 3 specific behaviors from the ABI item list for regular tracking through the duration of the study. These formed a part of the daily tracker, along with the additional “overall type of day” question. These questions were available for the caregiver to respond from 6 pm until 12 am. The sleep tracker appeared at 5 am each day and a request to complete the mood tracker appeared twice during the day on the “to do list.” Caregivers were free to use other components of the app, for example, journal, events tracker, in whichever way they wanted.

    Site staff members were able to monitor the caregiver’s usage of the app through a clinical dashboard and they had 2 follow up calls (week 1 and week 6) with participants and caregivers at the site visit. Reminders for caregivers to use the app were given at the 4-week midpoint visit, as necessary.

    Exit Interview

    Caregivers were asked to provide final feedback on their experience with My JAKE in an exit interview and by completing a 36-question Web-based survey about system functionality. Examples of questions included “Which JAKE components would you like to use outside of a clinical trial?” and “How easy were the following tasks to complete?”

    Data Analyses

    Correlations between My JAKE daily report measures—for example, 0 to 7 (0=troubling, 7=encouraging) rating on overall type of day, mood report—with ABI and scale domains were calculated using Spearman rank correlations controlling for age and gender.

    Features of the mood report analyzed were percentage of time in a given Quadrant—1 to 4 (1=top right, 2=top left, 3=bottom left, 4=bottom right; Figure 4)—percentage of positive valence or arousal, average valence or arousal, and variability of valence or arousal.

    All features were averaged over the entire duration of the study and compared with the baseline ABI Core ASD symptoms, associated behaviors, mental health (MH), self-regulation, challenging behavior, and the ZBI. Data were included in these analyses when there were 12 or more reports available.

    In addition, the features reported the week before midpoint and the week before endpoint were averaged and compared with scales completed for the corresponding time period. Data were included in analyses for weekly features when there were 3 or more reports available.


    Study Population

    Participants

    A total of 144 participants with a diagnosis of ASD were enrolled; 136 (94.4%) participants completed the study and 8 (5.6%) participants discontinued. The most common reason for discontinuation was self-withdrawal from the study (6; 4.2%). The majority of the ASD study population sample was male (77.8%), consistent with higher male:female ratio in ASD [49]. Mean (SD) age of participants was 14.6 (7.8) years. Mean (SD) ADOS total score of the participants was 7.6 (1.7), and intelligence quotient (IQ) was 99.2 (19.6).

    Respondents

    A majority of caregivers (41%) were between the ages of 41 to 50 years. Mothers made up 79.9% (115/144) of all caregivers. Most caregivers (32.6%; 47/144) had a bachelor’s degree (Table 2).

    Table 2. Caregiver demographics (N=144).
    Characteristicn (%)
    Age group (years)

    		Missing10 (6.9)

    		20-305 (3.5)

    		31-4041 (28.5)

    		41-5059 (41.0)

    		>5029 (20.1)
    Relationship

    		Missing10 (6.9)

    		Father12 (8.3)

    		Grandparent1 (0.7)

    		Mother115 (79.9)

    		Other caregiver (nonfamily member)1 (0.7)

    		Other family member5 (3.5)
    Education level

    		Missing14 (9.7)

    		Doctorate degree3 (2.1)

    		Professional degree6 (4.2)

    		Master’s degree23 (16.0)

    		Bachelor’s degree47 (32.6)

    		Some college, no degree41 (28.5)

    		High School graduate9 (6.3)

    		Less than high school1 (0.7)
    Data Collected

    Caregivers were asked about overall type of day and sleep once per day and asked to complete the mood reports twice per day (Table 3). The average number of caregiver reports per caregiver, completed over the 8-week study, was highest for mood. A total of 12 caregivers (8%) elected not to use My JAKE during the study. More reports were given for younger participants (<13 years of age) than for older participants (≥13 years of age) over the study period (Table 3). The frequency of caregiver reports obtained by week across the course of the study for overall type of day, sleep, and mood is shown in Figure 5. A linear mixed-effect model with random intercept, having participant as random factor and time (week) as fixed covariate, was used for modeling of weekly number of reports per participant to test whether there were changes in number of weekly reports as a function of time in the study. The number of weekly “overall day” reports per participant significantly decreases at rate 0.025 reports/week (P<.001, 95% CI–0.20 to –0.11), and the number of weekly “mood” reports per participant significantly decreases at rate 0.23 reports/week (P<.001, 95% CI–0.32 to –0.14). A decrease in reporting of sleep was not significant.

    Caregiver ratings were obtained from the exit interview on ease of use and mood report. Most caregivers reported their comfort level using the mobile app as “easy”—37.0 % (40/108)—or “very easy”—37.0% (40/108; Figure 6). Over half of the caregivers responded that they were either likely or very likely to use the mood report outside of clinical trials.

    Table 3. Average number of caregiver reports given for age subgroups (N=132).
    VariableMean (SD)
    Mood report

    		All participants61.2 (35.62)

    		<13 yearsa67.5 (38.86)

    		≥13 yearsb53.5 (30.54)
    Overall day

    		All participants29.8 (35.62)

    		<13 yearsa31.3 (17.62)

    		≥13 yearsb27.9 (16.97)
    Sleep

    		All participants28.4 (35.62)

    		<13 yearsa32.1 (19.07)

    		≥13 yearsb23.9 (18.24)

    an=72 for <13 years of age.

    bn=60 for ≥13 years of age.

    ‎
    Figure 5. Frequency of caregiver reports over the course of the study, shown as average number of weekly reports with correspondent standard deviations: overall day, sleep, and mood.
    ‎
    Figure 6. Caregiver rating on the use of the mobile app and the likelihood of using the mood report outside of a clinical trial.
    Caregiver Selected Behaviors

    Caregivers used the behavior selection component of My JAKE to choose 3 behaviors from the ABI that they wanted to track during the study. The most commonly selected behavior was “sleep problems” (Table 4).

    Sleep Tracker

    Caregivers’ daily reports of sleep displayed a significant negative correlation (P<.001) with retrospective reporting on the ABI sleep domain for average rating for the duration of the study and for averaged ratings in the 7 days before midpoint and endpoint visits (Table 5). A negative correlation was expected because a higher score on the ABI sleep symptoms indicates worse symptoms and vice versa.

    Overall Type of Day

    Overall type of day rating scores was averaged across all days in the study and then compared with ABI baseline scale scores for core ASD symptoms, challenging behavior, and MH domains a week after baseline, a week before the week 4 visit, and endpoint. Caregiver rating of overall type of day showed correlations with retrospective caregiver report measures (Table 6). As with sleep measures, a negative correlation was expected with ABI core symptoms, challenging behavior, and MH. In addition, the rating of overall type of day across the duration of the study negatively correlated with caregiver reported burden using the ZBI (n=114, r=–0.24, P=.005).

    Mood Report
    Average Valence

    Average valence for the study duration and for the week before midpoint and endpoint was compared with retrospective caregiver reports (Table 7). Average valence was negatively correlated with symptoms and behaviors. In particular, reported negative valence was associated with increased ASD core symptoms (ABI) and increased challenging behaviors across all time points.

    Table 4. Summary of most commonly selected Autism Behavior Inventory (ABI) daily tracking items.
    RankFrequencyaABI item selected for tracking
    124Has sleep problems
    220Acts without thinking
    319Has difficulty being flexible
    418Is fixated on certain topics or activities and unable to move on
    418Worries about things
    418Is tense or anxious
    716Is irritable and whiny
    814Shows inappropriate affection toward unfamiliar people
    913Has sensitivities to certain food textures
    913Reacts with aggression when he or she is upset or stressed
    1111Is able to take turns in conversation
    1111Is anxious in social situations
    1111Is excessively active
    1410Directs facial expression toward other people to communicate feelings
    1410Insists on saying words and phrases over and over

    aNumber of caregivers selecting the item to track.

    Table 5. Correlation coefficients of daily sleep reports with retrospective reports.
    Daily tracker sleepABIa sleep
    Duration (n=104)–0.62
    7 days before midpoint (n=78)–0.67
    7 days before endpoint (n=78)–0.63

    aABI: Autism Behavior Inventory.

    Table 6. Overall type of day correlations with Autism Behavior Inventory (ABI) core and other ABI domains, and Zarit Burden Interview (ZBI).
    Overall type of dayABI core symptomsChallenging behaviorMental healthZBI caregiver burden
    Study duration (n=112); ZBI (n=114)–0.42–0.35–0.38–0.24
    Midpoint (n=82)–0.34–0.42–0.29–0.27
    Endpoint (n=80)–0.32–0.45–0.26–0.27
    Table 7. Correlations of average valence with Autism Behavior Inventory (ABI) core and other domains and autism spectrum disorder (ASD) behaviors.
    TimepointABI core ASD symptom scale scoreABI mental healthABI self-regulationABI challenging behaviorABCa hyperactivity noncomplianceABC lethargy social withdrawal
    Duration (n=111)–0.26b–0.32c–0.25b–0.29c–0.16–0.29c
    Week 4 (n=95)–0.34c–0.35c–0.21b–0.32c–0.15–0.25b
    Endpoint (n=93)–0.26b–0.12–0.16–0.25b–0.06–0.09

    aABC: Aberrant Behavior Checklist.

    bSignificant relationship(P<.05) among average valence and ABI core and other domains and ASD behaviors.

    cSignificant relationship (P<.001) among average valence and ABI core and other domains and ASD behaviors.

    Table 8. Correlations of quadrants of activity and mood relationships with Autism Behavior Inventory (ABI) core and other domains, and autism spectrum disorder (ASD) behaviors.
    QuadrantABI core ASD symptom scale scoreABI mental healthABI self-regulationABI challenging behaviorABCa hyperactivity noncomplianceABC lethargy social withdrawal
    1–0.12–0.30–0.17–0.25–0.07–0.07
    20.180.260.270.310.210.21
    30.190.290.180.230.170.17
    4–0.010.10–0.010.05–0.04–0.04

    aABC: Aberrant Behavior Checklist.

    Average Arousal

    Average arousal was also compared for the same time points and no significant correlations were found.

    Quadrant Reports

    The percentage of reports in each quadrant of the mood report across the duration of the study was compared with caregiver reported scales at study baseline (Table 8). Negative correlations were expected between reports in quadrant 1 and increased severity of behavior (Table 1). Those with the fewest reports in quadrant 1 reported challenging behaviors (r=–0.25, P=.01) and MH (r=–0.30, P=.001). In contrast, positive correlations were expected among reports in other quadrants. In quadrant 2 (low valence, high arousal), significant positive correlations were found with MH (r=0.26, P=.006), challenging behavior (r=0.31, P=.06), and self-regulation (r=0.27, P=.005). In quadrant 3 (low valence, low arousal) significant positive correlations were found with MH (r=0.29, P=.003), challenging behavior (r=0.23, P=.02) and core ASD symptoms (r=0.19, P=.45). There were no significant correlations for quadrant 4 (high valence, low arousal).


    Principal Findings

    To our knowledge, this 8- to 10-week prospective, observational study including caregiver reporting using an mHealth app appears to be the first study of its kind and size in ASD. Caregivers who were using the app, reporting on 4 to 5 days of the week, were able to use My JAKE successfully to report mood, overall type of day, and sleep. There was some decline in reporting over time possibly as caregivers were requested, not mandated, to provide daily reports, but a large proportion of caregivers continued to provide regular reports on their child’s behavior over the course of the 8-week study. Reminders within the system and native to the phone app were limited to once per day, and missed assessments could not be retroactively completed, which meant that daily reports were guaranteed within real time. A few studies have addressed feasibility and compliance with these types of measures, but in a 2-week smoking cessation EMA study, Shiffman et al reported that responsiveness dropped in half of the participants by day 14 [16]. In comparison, a larger proportion of respondents in this study continued to provide daily reports at midpoint (4 weeks) and endpoint. Our data suggest that those caregivers who were still providing reports were doing so at the same rate as when the study began. It was also not unexpected that caregivers of younger children reported more often than caregivers of adolescents and young adults. Due to the exploratory nature of this study, other than the built-in pop-up reminders and midpoint visit, there were no other specific or consistent attempts to increase adherence and participation. However, clinical teams at sites had access to a dashboard, which enabled real-time viewing of data entry points. It would be possible for reports identified as key outcome measures to be more closely monitored and participation to be reinforced through telephone or text reminders by increased initial training on the use of the app. The current feasibility study had many components, including the use of biosensors in the clinic and at home, meaning potentially less focus was on the reporting in the app. In addition, it is expected that in the absence of an intervention, caregiver motivation to monitor and report behaviors will be less. We are currently implementing My JAKE in a 12-week interventional study. This version has some enhanced features for reminders and reduced, more specific requirements for caregivers. It also has an additional training session for parents on the use of the app, with increased support in selecting behaviors of interest to track. Furthermore, in response to parent feedback, interactive features for monitoring and viewing daily reports have been added to the app (whereas they were previously available on the Web). We will observe the influence that these enhancements have on parent reporting.

    The use of daily tracking of symptoms by caregivers could be a useful way of identifying early change in response to intervention. Sleep problems were most commonly prioritized by caregivers to track, and 3 items relating to anxiety also appeared in the top 15 items selected by caregivers, confirming research on the prevalence of these cooccurring problems in ASD and the importance of these behaviors to caregivers [70,71]. Other items were consistent with those commonly described by caregivers in clinical settings such as hyperactivity or impulsivity, rigidity, aggression, and problems with social interaction or interpretation [72,73].

    Daily measures of behavior showed significant correlations with the ABI and other scales, suggesting that daily electronic tracking of symptoms by caregivers could be a useful way of identifying early change in response to intervention. In particular, the single question regarding “overall type of day” correlated with caregiver report of core symptom severity. Overall type of day also correlated negatively with caregiver report of burden (ZBI), consistent with research on the impact of ASD symptoms and behavior on caregiver strain [74,75], and it has a possibility, with further validation, for use as a global index for caregiver-reported change. It is difficult to address the validity of a new measure, especially in relation to an existing measure [16]. Some concordance is expected, and it was seen in this study, indicating that similar constructs and concepts were being reported by caregivers in both real time and retrospectively. Further triangulation of data is important, for example, use of biosensor actigraphy information alongside daily sleep reports, and consideration of the relationship between physiological changes, detected by biosensors, and parent daily report as well as retrospective reports. This will be a focus of data analysis in our next clinical trial using My JAKE. In addition, we will be investigating the relationship of daily reports with measures of symptom and behavior change from other informants, such as the Clinician Global Impression of Severity [76].

    Caregivers reported that they found My JAKE easy-to-use, which was a likely contributor to their willingness and expressions of interest in using the system outside of a clinical trial. In particular, caregivers made use of the mood report, which allowed them quick and easy reporting on 2 dimensions of valence and arousal with a single action. Although valence correlated with ABI domains, arousal did not. The use of quadrants that combined the 2 features is a potentially useful way to measure and report mood, as it takes into account the second dimension of arousal. Further investigation into the relationship of quadrant reports and behavior will be carried out to establish the value of the 2-dimensional mood report for measuring mood in ASD. In addition, variability in mood or caregiver affect as interpreted by a caregiver or observer is a feature of particular interest when caregivers report regularly over a period of time. It should be noted that no anchor words were provided for reference on any part of the mood report, that is, caregivers were left to provide their own definitions while rating valence and arousal. Thus, the scale may be partly self-referential, and may require other analytic methods to objectively characterize scale performance (ie, operationally defining clinically meaningful change).

    Study Limitations

    The caregiver burden in this study was considerable. In addition to either filling in or curating a lengthy medical and developmental history, caregivers were expected to use My JAKE to report on their child’s behavior several times per day and this led to missing data. A better sampling strategy, including focus on only a few elements, is currently being implemented in an interventional study to prevent data loss and could contribute to keeping caregiver burden low and quality and utility of data high.

    We also observed a differential pattern of reporting that included fewer assessments completed for caregivers of adolescents and adults compared with children. This may be because of a lack of awareness of behaviors, less frequent contact or observation opportunity, or other factors, and it should be taken into consideration when selecting completion requirements in future studies.

    Although this was a relatively heterogeneous group of individuals with ASD, it would be helpful to obtain data for participants across the spectrum of severity and IQ to determine whether there are meaningful patterns of differences. This would necessarily require a much larger sample than allowed by this initial study.

    Data were generally provided by the primary caregiver. It may be important to have data from other collateral sources such as school or program staff or self-reported data from individuals with ASD who are capable of providing feedback on their own behavior.

    Though My JAKE was not used by caregivers for EMA in this study, it does have the capacity to do so with a minor modification. However, the burden of more frequent prompts and reminders for caregivers would need to be considered, in contrast with the need for consistency of many repeated measures over fluctuating periods of time. In addition, caregivers are not always with the child to report behaviors in real time (or the nature of a distracting behavior itself may make reporting difficult). The ability of My JAKE to be used for real-time and retrospective reporting of events increases its flexibility to be used to collect the type of data most relevant to outcomes of interest for a particular intervention while considering what is least burdensome for caregivers. We plan to use the event reporting from our current intervention study to correlate with biosensor data obtained from an actigraphy watch that participants with ASD will wear over the 12-week period. The use of biosensor data will also improve our understanding of the content validity of daily assessments, providing an opportunity for comparison with objective measures alongside the retrospective parent reports.

    Next Steps

    My JAKE is currently employed in an interventional clinical study in ASD. Learnings from this study relating to frequency and burden of caregiver reporting, specific instructions to caregivers, and ability of study sites to train and support caregivers in regular use of the app have been used to ensure maintenance of quality and useful data for measuring outcomes. The current version of My JAKE is also suitable for multiple raters, including teachers and other health care professionals, enabling the possibility of additional reporting across a broader range of contexts, settings, and perspectives. Finally, with some adaptations, the app has the potential to allow for self-reporting in more cognitively-able and older individuals with ASD. This would add an important dimension to the assessment of change in symptoms over time.

    Conclusions

    To our knowledge, this study represents the first reported implementation of an mHealth app in a large number of caregivers for electronically tracking behaviors in ASD. Caregivers were successful in using My JAKE to report behaviors, and reported behaviors largely agreed with other more traditional retrospective reports. The study findings will aid in selecting specific outcome measures, leading to fewer and more specific reporting requirements from caregivers, with the expected result of reduced caregiver burden, improved data quality, and potentially more sensitive and meaningful outcomes in monitoring behavior of individuals with ASD.

    Acknowledgments

    The authors thank the study participants and the following investigators for their participation in this study. Arizona: Christopher J Smith, PhD. California: Bennett Leventhal, MD; Robert Hendren, DO. Connecticut (at the time of study conduct): Frederick Shic, PhD. Massachusetts: Jean Frazier, MD; Matthew Goodwin, PhD. New Jersey: Yvette Janvier, MD. New York: Russell Tobe, MD. North Carolina: Geraldine Dawson, PhD. Pennsylvania: Judith S Miller, PhD. Washington: Bryan King, MD.

    Stacey E Shehin, PhD, an employee of PRA Health Sciences provided medical writing assistance, which was funded by Janssen Research & Development, LLC. Ellen Baum, PhD (Janssen Global Services) provided editorial assistance. Portions of this study have been previously presented at the International Meeting for Autism Research, May 10 to 13, 2017, San Francisco, USA. This study was funded by Janssen Research & Development, LLC.

    Authors' Contributions

    AB, NVM, MB, SJ, AS, MC, MSG, GD, BL, RH, FS, SN, and GP were involved in study design, data collection, analysis or interpretation of the results, drafting or revising the manuscript, and final approval of the manuscript. DL was responsible for the statistical analyses. All authors had full access to all study data, and they take responsibility for the integrity of the data and the accuracy of the data analysis. All authors have approved the final manuscript.

    Conflicts of Interest

    AB, JM, NVM, DL, MB, AS, SJ, MC, SN, and GP are employees of Janssen Research & Development, LLC and may hold company stocks or stock options. GD is on the Scientific Advisory Boards of Janssen Research and Development and Akili, Inc, a consultant to Roche, has received grant funding from Janssen Research and Development, LLC and Perkin Elmer, and receives royalties from Guildford Press and Oxford University Press. MSG has received research and consulting funding from Janssen Research & Development, LLC. RH received reimbursement for consultation from Janssen Research & Development, LLC. BL has received research grant funding from the NIH, is a consultant to Janssen Research & Development, LLC and the Illinois Children’s Healthcare Foundation, and is a board member of the Brain Research Foundation. FS is on the Scientific Advisory Board of and is a consultant to Janssen Research and Development, LLC, and has received grant funding from Janssen Research and Development LLC, and Roche.

    1. Centers for Disease Control and Prevention. 2017. URL: https://www.cdc.gov/ncbddd/autism/data.html [accessed 2018-05-10] [WebCite Cache]
    2. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Arlington, TX. American Psychiatric Publishing; 2013.
    3. Aman MG, Novotny S, Samango-Sprouse C, Lecavalier L, Leonard E, Gadow KD, et al. Outcome measures for clinical drug trials in autism. CNS Spectr. Jan 2004;9(1):36-47. [FREE Full text] [Medline]
    4. Ghosh A, Michalon A, Lindemann L, Fontoura P, Santarelli L. Drug discovery for autism spectrum disorder: challenges and opportunities. Nat Rev Drug Discov. Oct 2013;12(10):777-790. [CrossRef] [Medline]
    5. Loth E, Charman T, Mason L, Tillmann J, Jones EJ, Wooldridge C, et al. The EU-AIMS Longitudinal European Autism Project (LEAP): design and methodologies to identify and validate stratification biomarkers for autism spectrum disorders. Mol Autism. Jun 2017;8:24. [FREE Full text] [CrossRef] [Medline]
    6. Loth E, Spooren W, Ham LM, Isaac MB, Auriche-Benichou C, Banaschewski T, et al. Identification and validation of biomarkers for autism spectrum disorders. Nat Rev Drug Discov. Jan 2016;15(1):70-73. [CrossRef] [Medline]
    7. Ness SL, Manyakov NV, Bangerter A, Lewin D, Jagannatha S, Boice M, et al. JAKE® Multimodal Data Capture System: insights from an Observational Study of Autism Spectrum Disorder. Front Neurosci. 2017;11:517. [FREE Full text] [CrossRef] [Medline]
    8. Bangerter A, Ness S, Aman MG, Esbensen AJ, Goodwin MS, Dawson G, et al. Autism Behavior Inventory: a novel tool for assessing core and associated symptoms of autism spectrum disordera. J Child Adolesc Psychopharmacol. Nov 2017;27(9):814-822. [FREE Full text] [CrossRef] [Medline]
    9. Kanne SM, Mazurek MO, Sikora D, Bellando J, Branum-Martin L, Handen B, et al. The Autism Impact Measure (AIM): initial development of a new tool for treatment outcome measurement. J Autism Dev Disord. Jan 2014;44(1):168-179. [CrossRef] [Medline]
    10. Lord C. Methods and measures of behavior in the diagnosis of autism and related disorders. Psychiatr Clin North Am. Mar 1991;14(1):69-80. [Medline]
    11. Constantino JN, Davis SA, Todd RD, Schindler MK, Gross MM, Brophy SL, et al. Validation of a brief quantitative measure of autistic traits: comparison of the social responsiveness scale with the autism diagnostic interview-revised. J Autism Dev Disord. Aug 2003;33(4):427-433. [CrossRef] [Medline]
    12. Jones RM, Risi S, Wexler D, Anderson D, Corsello C, Pickles A, et al. How interview questions are placed in time influences caregiver description of social communication symptoms on the ADI-R. J Child Psychol Psychiatry. May 2015;56(5):577-585. [FREE Full text] [CrossRef] [Medline]
    13. Coyne JC, Gottlieb BH. The mismeasure of coping by checklist. J Pers. Dec 1996;64(4):959-991. [CrossRef] [Medline]
    14. Rutherford C, Costa D, Mercieca-Bebber R, Rice H, Gabb L, King M. Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis. Qual Life Res. Mar 2016;25(3):559-574. [CrossRef] [Medline]
    15. Heide M, Gronhaug K. Respondents' Moods As a Biasing Factor in Surveys: an Experimental Study, in NA - Advances in Consumer Research. Provo, UT. Association for Consumer Research; 1991. URL: http://www.acrwebsite.org/volumes/7218/volumes/v18/NA-18 [WebCite Cache]
    16. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4:1-32. [Medline]
    17. Stone AA, Shiffman S. Capturing momentary, self-report data: a proposal for reporting guidelines. Ann Behav Med. 2002;24(3):236-243. [Medline]
    18. Woodard CR, Goodwin MS, Zelazo PR, Aube D, Scrimgeour M, Ostholthoff T, et al. A comparison of autonomic, behavioral, and parent-report measures of sensory sensitivity in young children with autism. Research in Autism Spectrum Disorders. Jul 2012;6(3):1234-1246. [CrossRef]
    19. Damiano CR, Mazefsky CA, White SW, Dichter GS. Future directions for research in autism spectrum disorders. J Clin Child Adolesc Psychol. 2014;43(5):828-843. [FREE Full text] [CrossRef] [Medline]
    20. W Adams Z, McClure EA, Gray KM, Danielson CK, Treiber FA, Ruggiero KJ. Mobile devices for the remote acquisition of physiological and behavioral biomarkers in psychiatric clinical research. J Psychiatr Res. Dec 2017;85:1-14. [FREE Full text] [CrossRef] [Medline]
    21. Goodwin MS, Intille SS, Albinali F, Velicer WF. Automated detection of stereotypical motor movements. J Autism Dev Disord. Jun 2011;41(6):770-782. [CrossRef] [Medline]
    22. Großekathöfer U, Manyakov NV, Mihajlović V, Pandina G, Skalkin A, Ness S, et al. Automated detection of stereotypical motor movements in autism spectrum disorder using recurrence quantification analysis. Front Neuroinform. 2017;11:9. [FREE Full text] [CrossRef] [Medline]
    23. Hossain SM, Ali AA, Rahman M, Ertin E, Epstein D, Kennedy A, et al. Identifying drug (cocaine) intake events from acute physiological response in the presence of free-living physical activity. IPSN. 2014;2014:71-82. [FREE Full text] [Medline]
    24. Kratz A, Braley T, Whibley D, Murphy S. Are nights of poor sleep related to greater next-day fatigue and pain in older adults with symptomatic osteoarthritis? An actigraphy and ecological momentary assessment study. J Pain. 2018;19(3):S7-S8. [CrossRef]
    25. Staples P, Torous J, Barnett I, Carlson K, Sandoval L, Keshavan M, et al. A comparison of passive and active estimates of sleep in a cohort with schizophrenia. NPJ Schizophr. Oct 16, 2017;3(1):37. [FREE Full text] [CrossRef] [Medline]
    26. Libman E, Fichten C, Bailes S, Amsel R. Sleep questionnaire versus sleep diary: which measure is better? Int J Rehabil Health. 2000;5(3):205-209. [CrossRef]
    27. Wenze SJ, Miller IW. Use of ecological momentary assessment in mood disorders research. Clin Psychol Rev. Aug 2010;30(6):794-804. [CrossRef] [Medline]
    28. Malow BA, Connolly HV, Weiss SK, Halbower A, Goldman S, Hyman SL, et al. The Pediatric Sleep Clinical Global Impressions Scale-a new tool to measure pediatric insomnia in autism spectrum disorders. J Dev Behav Pediatr. Jun 2016;37(5):370-376. [CrossRef] [Medline]
    29. Russell AJ, Murphy CM, Wilson E, Gillan N, Brown C, Robertson DM, et al. The mental health of individuals referred for assessment of autism spectrum disorder in adulthood: a clinic report. Autism. Dec 2016;20(5):623-627. [CrossRef] [Medline]
    30. Lidstone J, Uljarevi M, Sullivan J, Rodgers J, McConachie H, Freeston M. Relations among restricted and repetitive behaviors, anxiety and sensory features in children with autism spectrum disorders. Res Autism Spectr Disord. 2014;8(2):82-92. [CrossRef]
    31. Miller A, Vernon T, Wu V, Russo K. Social skill group interventions for adolescents with autism spectrum disorders: a systematic review. Rev J Autism Dev Disord. Jun 5, 2014;1(4):254-265. [CrossRef]
    32. Spiker MA, Lin CE, Van Dyke M, Wood JJ. Restricted interests and anxiety in children with autism. Autism. May 2012;16(3):306-320. [CrossRef] [Medline]
    33. Veatch OJ, Sutcliffe JS, Warren ZE, Keenan BT, Potter MH, Malow BA. Shorter sleep duration is associated with social impairment and comorbidities in ASD. Autism Res. Jul 2017;10(7):1221-1238. [CrossRef] [Medline]
    34. Adams HL, Matson JL, Cervantes PE, Goldin RL. The relationship between autism symptom severity and sleep problems: Should bidirectionality be considered? Res Autism Spectr Disord. 2014;8(3):193-199. [FREE Full text] [CrossRef]
    35. Hollway J, Aman M, Butter E. Correlates and risk markers for sleep disturbance in participants of the Autism Treatment Network. J Autism Dev Disord. Dec 2013;43(12):2830-2843. [Medline]
    36. Mayes S, Calhoun S, Murray M, Zahid J. Variables associated with anxiety and depression in children with autism. J Dev Phys Disabil. 2011;23(4):325-337. [CrossRef]
    37. Meltzer LJ, Mindell JA. Relationship between child sleep disturbances and maternal sleep, mood, and parenting stress: a pilot study. J Fam Psychol. Mar 2007;21(1):67-73. [CrossRef] [Medline]
    38. Hufford M, Stone A, Shiffman S, Schwartz J, Broderick J. Paper vs electronic diaries. Appl Clin Trials. 2002;11(8):38-43. https://www.researchgate.net/publication/284666042_Paper_vs_electronic_diaries_Compliance_and_subject_evaluations.
    39. Barlow DH, Nock MK. Why can't we be more idiographic in our research? Perspect Psychol Sci. 2009;4(1):19-21. [CrossRef]
    40. McConachie H, Parr JR, Glod M, Hanratty J, Livingstone N, Oono IP. National Institute for Health Research. Jun 2015. URL: https://pure.qub.ac.uk/portal/files/39022827/HTA_MEASURE_Review.pdf [accessed 2018-12-03] [WebCite Cache]
    41. Arnold LE, Huestis RD, Smeltzer DJ, Scheib J, Wemmer D, Colner G. Levoamphetamine vs dextroamphetamine in minimal brain dysfunction. Replication, time response, and differential effect by diagnostic group and family rating. Arch Gen Psychiatry. Mar 1976;33(3):292-301. [Medline]
    42. Arnold LE, Vitiello B, McDougle C, Scahill L, Shah B, Gonzalez NM, et al. Parent-defined target symptoms respond to risperidone in RUPP autism study: customer approach to clinical trials. J Am Acad Child Adolesc Psychiatry. Dec 2003;42(12):1443-1450. [CrossRef] [Medline]
    43. Arnold LE, Wender PH, McCloskey K, Snyder SH. Levoamphetamine and dextroamphetamine: comparative efficacy in the hyperkinetic syndrome. Assessment by target symptoms. Arch Gen Psychiatry. Dec 1972;27(6):816-822. [Medline]
    44. Weisz JR, Chorpita BF, Palinkas LA, Schoenwald SK, Miranda J, Bearman SK, et al. Research Network on Youth Mental Health. Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial. Arch Gen Psychiatry. Mar 2012;69(3):274-282. [CrossRef] [Medline]
    45. Fox S, Duggan M. Pew Research Center's Internet & American Life Project. 2013. URL: http://www.pewinternet.org/2013/01/28/tracking-for-health/ [accessed 2019-03-08] [WebCite Cache]
    46. Free C, Phillips G, Watson L, Galli L, Felix L, Edwards P, et al. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med. Jan 2013;10(1):e1001363. [FREE Full text] [CrossRef] [Medline]
    47. Woodward B. m-Health Care Models and Applications. In: m-Health: Fundamentals and Applications. Hoboken, New Jersey. Wiley-IEEE Press; 2016.
    48. Rosa C, Campbell AN, Miele GM, Brunner M, Winstanley EL. Using e-technologies in clinical trials. Contemp Clin Trials. Nov 2015;45(Pt A):41-54. [CrossRef] [Medline]
    49. Dennison L, Morrison L, Conway G, Yardley L. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. J Med Internet Res. 2013;15(4):e86. [FREE Full text] [CrossRef] [Medline]
    50. Chen RY, Feltes JR, Tzeng WS, Lu ZY, Pan M, Zhao N, et al. Phone-based interventions in adolescent psychiatry: a perspective and proof of concept pilot study with a focus on depression and autism. JMIR Res Protoc. Jun 16, 2017;6(6):e114. [FREE Full text] [CrossRef] [Medline]
    51. Khor AS, Gray KM, Reid SC, Melvin GA. Feasibility and validity of ecological momentary assessment in adolescents with high-functioning autism and Asperger's disorder. J Adolesc. Jan 2014;37(1):37-46. [CrossRef] [Medline]
    52. Kovac M, Mosner M, Miller S, Hanna EK, Dichter GS. Experience sampling of positive affect in adolescents with autism: Feasibility and preliminary findings. Res Autism Spectr Disord. 2016;29-30:57-65. [FREE Full text] [CrossRef] [Medline]
    53. Kring AM, Barrett LF, Gard DE. On the broad applicability of the affective circumplex: representations of affective knowledge among schizophrenia patients. Psychol Sci. May 2003;14(3):207-214. [CrossRef] [Medline]
    54. Tseng A, Bansal R, Liu J, Gerber AJ, Goh S, Posner J, et al. Using the circumplex model of affect to study valence and arousal ratings of emotional faces by children and adults with autism spectrum disorders. J Autism Dev Disord. Jun 2014;44(6):1332-1346. [FREE Full text] [CrossRef] [Medline]
    55. Microsoft. International HealthVault. 2018. URL: https://international.healthvault.com/us/en [accessed 2018-05-10] [WebCite Cache]
    56. Anagnostou E, Jones N, Huerta M, Halladay AK, Wang P, Scahill L, et al. Measuring social communication behaviors as a treatment endpoint in individuals with autism spectrum disorder. Autism. Jul 2015;19(5):622-636. [CrossRef] [Medline]
    57. Lecavalier L, Wood JJ, Halladay AK, Jones NE, Aman MG, Cook EH, et al. Measuring anxiety as a treatment endpoint in youth with autism spectrum disorder. J Autism Dev Disord. May 2014;44(5):1128-1143. [FREE Full text] [CrossRef] [Medline]
    58. Scahill L, Aman MG, Lecavalier L, Halladay AK, Bishop SL, Bodfish JW, et al. Measuring repetitive behaviors as a treatment endpoint in youth with autism spectrum disorder. Autism. Jan 2015;19(1):38-52. [CrossRef] [Medline]
    59. Aman MG, Singh NN. Aberrant Behavior Checklist Manual. East Aurora, NY. Slosson Educational Publications, Inc; 2017.
    60. Kaat AJ, Lecavalier L, Aman MG. Validity of the aberrant behavior checklist in children with autism spectrum disorder. J Autism Dev Disord. May 2014;44(5):1103-1116. [CrossRef] [Medline]
    61. Zarit SH, Reever KE, Bach-Peterson J. Relatives of the impaired elderly: correlates of feelings of burden. Gerontologist. Dec 1980;20(6):649-655. [Medline]
    62. Cetinbakis G, Bastug G, Ozel-Kizil E. Factors contributing to higher caregiving burden in Turkish mothers of children with autism spectrum disorders. Int J Develop Disabilities. Jun 05, 2018:1-8. [CrossRef]
    63. Ezzat O, Samarkandi OA, Bayoumi M. Quality of life and subjective burden on family caregiver of children with autism. Asian J Neurosurg. 2017;6(1):33. [CrossRef]
    64. Hartley SL, Barker ET, Seltzer MM, Greenberg JS, Floyd FJ. Marital satisfaction and parenting experiences of mothers and fathers of adolescents and adults with autism. Am J Intellect Dev Disabil. Jan 2011;116(1):81-95. [FREE Full text] [CrossRef] [Medline]
    65. Javalkar K, Rak E, Phillips A, Haberman C, Ferris M, Van Tilburg M. Predictors of caregiver burden among mothers of children with chronic conditions. Children. May 16, 2017;4(5):39. [CrossRef]
    66. Macedo EC, da Silva LR, Paiva MS, Ramos MN. Burden and quality of life of mothers of children and adolescents with chronic illnesses: an integrative review. Rev Lat Am Enfermagem. 2015;23(4):769-777. [FREE Full text] [CrossRef] [Medline]
    67. Orsmond GI, Seltzer MM, Greenberg JS, Krauss MW. Mother-child relationship quality among adolescents and adults with autism. Am J Ment Retard. Mar 2006;111(2):121-137. [CrossRef] [Medline]
    68. Pandey S, Sharma C. Perceived burden in caregivers of children with autism spectrum disorder. J Nepal Health Res Counc. Jul 03, 2018;16(2):184-189. [Medline]
    69. Tomasello M, Carpenter M, Call J, Behne T, Moll H. Understanding and sharing intentions: the origins of cultural cognition. Behav Brain Sci. Oct 2005;28(5):675-91; discussion 691. [CrossRef] [Medline]
    70. Goldman SE, Richdale AL, Clemons T, Malow BA. Parental sleep concerns in autism spectrum disorders: variations from childhood to adolescence. J Autism Dev Disord. Apr 2012;42(4):531-538. [CrossRef] [Medline]
    71. Kerns CM, Kendall PC, Berry L, Souders MC, Franklin ME, Schultz RT, et al. Traditional and atypical presentations of anxiety in youth with autism spectrum disorder. J Autism Dev Disord. Nov 2014;44(11):2851-2861. [FREE Full text] [CrossRef] [Medline]
    72. Kanne SM, Mazurek MO. Aggression in children and adolescents with ASD: prevalence and risk factors. J Autism Dev Disord. Jul 2011;41(7):926-937. [CrossRef] [Medline]
    73. Leyfer OT, Folstein SE, Bacalman S, Davis NO, Dinh E, Morgan J, et al. Comorbid psychiatric disorders in children with autism: interview development and rates of disorders. J Autism Dev Disord. Oct 2006;36(7):849-861. [CrossRef] [Medline]
    74. Al-Farsi OA, Al-Farsi YM, Al-Sharbati MM, Al-Adawi S. Stress, anxiety, and depression among parents of children with autism spectrum disorder in Oman: a case-control study. Neuropsychiatr Dis Treat. 2016;12:1943-1951. [FREE Full text] [CrossRef] [Medline]
    75. Schieve LA, Blumberg SJ, Rice C, Visser SN, Boyle C. The relationship between autism and parenting stress. Pediatrics. Feb 2007;119(Suppl 1):S114-S121. [CrossRef] [Medline]
    76. Arnold LE, Aman MG, Martin A, Collier-Crespin A, Vitiello B, Tierney E, et al. Assessment in multisite randomized clinical trials of patients with autistic disorder: the Autism RUPP Network. Research Units on Pediatric Psychopharmacology. J Autism Dev Disord. Apr 2000;30(2):99-111. [Medline]

    ‎
    ABC: Aberrant Behavior Checklist
    ABI: Autism Behavior Inventory
    ADOS: Autism Diagnostic Observation Schedule
    ASD: autism spectrum disorder
    EMA: ecological momentary assessment
    IQ: intelligence quotient
    JAKE: Janssen Autism Knowledge Engine
    MH: mental health
    mHealth: mobile health
    SRS-2: Social Responsiveness Scale 2
    ZBI: Zarit Burden Interview

    Edited by G Eysenbach; submitted 28.06.18; peer-reviewed by Z Warren, G Abowd; comments to author 06.10.18; revised version received 05.12.18; accepted 31.12.18; published 26.03.19.

    Copyright

    ©Abigail Bangerter, Nikolay V. Manyakov, David Lewin, Matthew Boice, Andrew Skalkin, Shyla Jagannatha, Meenakshi Chatterjee, Geraldine Dawson, Matthew S Goodwin, Robert Hendren, Bennett Leventhal, Frederick Shic, Seth Ness, Gahan Pandina. Originally published in JMIR Mental Health (http://mental.jmir.org), 26.03.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.