This paper is in the following e-collection/theme issue:

Special Issue on Video Games for Mental Health (8) Extended Reality, Virtual Reality and Virtual Worlds (560) Serious Games for Health and Medicine (683) Anxiety and Stress Disorders (1053) Attention Deficit Disorder (ADD/ADHD) (60) Autism Spectrum Disorder (ASD) (109) Mental Health Games (80) Mental Health Issues in Adolescence (442)

Published on in Vol 7, No 3 (2020): March

Efficacy of a Virtual Reality Biofeedback Game (DEEP) to Reduce Anxiety and Disruptive Classroom Behavior: Single-Case Study

Efficacy of a Virtual Reality Biofeedback Game (DEEP) to Reduce Anxiety and Disruptive Classroom Behavior: Single-Case Study

Efficacy of a Virtual Reality Biofeedback Game (DEEP) to Reduce Anxiety and Disruptive Classroom Behavior: Single-Case Study

Authors of this article:

Rineke Bossenbroek1 Author Orcid Image ;   Aniek Wols1 Author Orcid Image ;   Joanneke Weerdmeester1 Author Orcid Image ;   Anna Lichtwarck-Aschoff1 Author Orcid Image ;   Isabela Granic1 Author Orcid Image ;   Marieke M J W van Rooij1 Author Orcid Image
Article Authors Cited by (56) Tweetations (19) Metrics

    Original Paper

    Behavioural Science Institute, Radboud University, Nijmegen, Netherlands

    Corresponding Author:

    Rineke Bossenbroek, MSc

    Behavioural Science Institute

    Radboud University

    PO Box 9104

    Nijmegen, 6500 HE

    Netherlands

    Phone: 31 243612638

    Email: r.bossenbroek@pwo.ru.nl


    Background: Many adolescents in special education are affected by anxiety in addition to their behavioral problems. Anxiety leads to substantial long-term problems and may underlie disruptive behaviors in the classroom as a result of the individual’s inability to tolerate anxiety-provoking situations. Thus, interventions in special needs schools that help adolescents cope with anxiety and, in turn, diminish disruptive classroom behaviors are needed.

    Objective: This study aimed to evaluate the effect of a virtual reality biofeedback game, DEEP, on daily levels of state-anxiety and disruptive classroom behavior in a clinical sample. In addition, the study also aimed to examine the duration of the calm or relaxed state after playing DEEP.

    Methods: A total of 8 adolescents attending a special secondary school for students with behavioral and psychiatric problems participated in a single-case experimental ABAB study. Over a 4-week period, participants completed 6 DEEP sessions. In addition, momentary assessments (ie, 3 times a day) of self-reported state-anxiety and teacher-reported classroom behavior were collected throughout all A and B phases.

    Results: From analyzing the individual profiles, it was found that 6 participants showed reductions in anxiety, and 5 participants showed reductions in disruptive classroom behaviors after the introduction of DEEP. On a group level, results showed a small but significant reduction of anxiety (d=–0.29) and a small, nonsignificant reduction of disruptive classroom behavior (d=−0.16) on days when participants played DEEP. Moreover, it was found that the calm or relaxed state of participants after playing DEEP lasted for about 2 hours on average.

    Conclusions: This study demonstrates the potential of the game, DEEP, as an intervention for anxiety and disruptive classroom behavior in a special school setting. Future research is needed to fully optimize and personalize DEEP as an intervention for the heterogeneous special school population.

    JMIR Ment Health 2020;7(3):e16066

    doi:10.2196/16066

    Keywords

    anxiety; disruptive behavior; single-case study; applied game; serious games; special education; attention-deficit/hyperactivity disorder (ADHD); autism spectrum disorder (ASD); adolescents 


    Background

    Adolescents attending schools for special education in the Netherlands (called “cluster 4 schools”) are characterized by profound behavioral and psychiatric problems [1], such as autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), and oppositional defiant disorder. In addition, 27% to 32% of the Dutch heterogeneous special school population exhibits clinical levels of anxiety [2], indicated by high comorbidity rates between anxiety and ASD (40%) [3] and ADHD (25%) [4]. Anxiety symptoms significantly interfere with adolescents’ mental health, academic achievement, and social functioning [5]. Moreover, anxiety may underlie disruptive behaviors in the classroom, such as acting out spontaneously or expressing worries frequently, as a result of the individual’s inability to tolerate anxiety-provoking situations [6]. Individuals suffering from anxiety often have trouble regulating their behaviors in those situations because of underdeveloped self-regulation skills [7]. Clearly, interventions in special needs schools that help adolescents cope with anxiety and, in turn, diminish disruptive classroom behaviors are needed.

    Conventional school-based interventions for anxiety are usually based on cognitive behavioral therapy (CBT) [8]. However, these conventional approaches pose limitations with regard to the special school population, as they often do not take the presence of comorbid disorders into consideration. For example, children with ASD may find it difficult to learn CBT skills such as cognitive restructuring because of their cognitive and social impairments [9]. Children with ADHD might also feel uncomfortable in talk therapies because higher-order abstractions may be challenging for them [10]. Moreover, conventional approaches entail challenges in their form of delivery because children are often not motivated to attend the didactic-based sessions nor to complete the homework assignments [11]. Within the school setting in particular, clinicians often fail to match interventions to the child’s readiness or motivation for change and hence struggle to keep students engaged [12,13]. Another limitation of CBT includes the use of decontextualized exercises that do not fully represent the authentic emotional and physical experiences associated with anxiety [14], limiting the transfer of the learned skills to other contexts (eg, the classroom setting). Finally, school clinicians have limited time to successfully deliver CBT [13,15]. As a result of these limitations and challenges, conventional school-based programs for anxiety often yield disappointing outcomes, with small-to-moderate effect sizes and intervention effects that do not sustain over time (see Mychailyszyn et al [16] and Werner-Seidler et al [17] for meta-analyses). Taking these limitations together, alternative interventions for anxiety need to be considered.

    A promising alternative approach to enhance mental health among children and adolescents is the use of video games. A recent review showed that video games provide youths with immersive emotional experiences, teaching them new forms of emotional, cognitive, and behavioral strategies [14]. Video games may have the potential to address each of the limitations discussed. First, video games may be suitable for the heterogeneous special school population as they provide the player with visual aids and structured sensory information, an important prerequisite in treating anxiety in children with ASD [9,18]. Moreover, video games may impose less load on working memory capacities compared with the verbal reasoning required in CBT, which is an important requirement to treat anxiety in children with ADHD [19]. Second, video games hold great potential to intrinsically motivate and engage children, thereby addressing one of the most challenging tasks faced by (school) clinicians [20]. Third, video games provide youths with an opportunity to practice anxiety regulation skills until they are automatized and can be transferred to situations outside of the game [14]. Finally, a potential benefit of video games is their relatively low cost compared with traditionally delivered mental health interventions [21], thereby addressing the limited resources of clinicians in school settings. Overall, video games hold great promise as a novel intervention approach for adolescents with anxiety in the special school setting.

    DEEP

    Recently, a virtual reality biofeedback game (DEEP) was developed as a potential intervention to reduce anxiety in youths [22]. In DEEP, players explore an underwater fantasy world by using their own breathing to control their movement. Players wear a belt with a stretch sensor around their waist just below their diaphragm. This sensor measures the expansion of the diaphragm associated with breathing, providing input to control in-game movement: the slower and deeper players breathe through their diaphragm, the better they can move around in the underwater world. The game aims to provide a relaxing and immersive experience for the player; there are no in-game goals to attain. DEEP targets a fundamental causal mechanism that contributes to the development and maintenance of anxiety symptoms: physiological reactivity. Anxious youths tend to experience hyperarousal (eg, rapid breathing or increased heart rate) in response to stressors [23]. The body’s physiological response interacts with negative cognitive biases (eg, focus on threat-relevant information [24]) to produce the negative affective states that characterize anxiety [25,26]. Thus, the ability to regulate physiological arousal may modify the interaction with negative cognitive biases, resulting in a positive change in the affective state.

    The mechanism through which DEEP teaches players how to regulate physiological responses is diaphragmatic breathing [27]. Diaphragmatic breathing is the act of initiating breath into the lungs from contraction of the diaphragm muscle rather than the rib cage, which facilitates breathing efficiency and efficient exhalation [28,29]. Diaphragmatic breathing at a slow pace has been shown to evoke a relaxation response of the autonomic nervous system [30]. Hence, diaphragmatic breathing is commonly used as a relaxation technique in evidence-based treatments for anxiety [29,31].

    The breathing exercises incorporated in DEEP are based on biofeedback, which is defined as the process of feeding information back to the individual about one’s physiological state to gain awareness and control over physiological processes [32]. Biofeedback is an effective evidence-based therapeutic technique for regulating anxiety (see Schoenberg et al [33] and Weerdmeester et al [34] for recent reviews). In DEEP, biofeedback mechanics are applied in various ways. First, players receive feedback about their stage of breathing by their way of movement: when players inhale, an upward (when close to the ground) or a forward force is applied; when players exhale, these forces are strengthened. Second, players receive feedback about their breathing through visual cues. A circle is shown in the players’ visual field that expands with inhalation and contracts with exhalation (Figure 1). Finally, players’ breathing is mirrored by elements in the environment (eg, plants) that change in color, size, or movement accordingly. See Multimedia Appendix 1 for a short video of DEEP.

    ‎
    Figure 1. Visual circle that is depicted in the players’ visual field corresponding to an inhalation (left) and exhalation peak (right).

    Previous Research and Remaining Questions

    Preliminary evidence for the efficacy of DEEP as an intervention for anxiety has been demonstrated in a recent pilot study [22]. In a sample of 86 typically developing children, DEEP reduced levels of state-anxiety directly after playing the game for 7 min. However, knowledge about the duration of the calm or relaxed state after playing DEEP is lacking, and it is also unknown if playing DEEP can reduce levels of state-anxiety in a clinical sample.

    Next to the potential of DEEP as an intervention for anxiety, an additional yet uncertain beneficial effect of DEEP could be an improvement in the ability to regulate disruptive classroom behaviors. Previous research has shown that traditionally delivered deep-breathing exercises reduced rates of disruptive classroom behaviors (assessed by blind independent observers) among youths attending a special school for students with behavioral problems [35]. Moreover, a game-based intervention incorporating deep-breathing techniques and biofeedback has been found to significantly decrease self-reported externalizing behaviors of adolescents in residential youth care [36,37]. There may be two ways in which DEEP could potentially affect adolescents’ behaviors. First, playing DEEP may reduce disruptive classroom behaviors through its effect on anxiety levels. Disruptive behaviors of anxious youths are often driven by the individual’s inability to escape or avoid anxiety-provoking situations [6,38]. Not being able to withdraw from or avoid those situations (ie, blocking of the individual’s goals) may increase levels of frustration, which could result in subsequent externalizing behaviors [38,39]. Thus, if adolescents feel more relaxed and less anxious after playing DEEP, they may handle any subsequent anxiety-provoking situation better and feel less frustrated about being unable to escape. The diminished frustration may in turn make them less prone to show disruptive behaviors in the classroom.

    Second, playing DEEP could potentially reduce disruptive behaviors through its effect on interoceptive awareness: the ability to recognize internal physiological states [40]. Playing DEEP might bring about increased interoceptive awareness as players are continuously informed about their breathing while playing. Heightened interoception might in turn enable adolescents to mobilize self-regulation resources as individuals who are able to focus on their body’s response to anxiety might recognize more quickly that control is needed [41]. Next to an efficient mobilization of self-regulation resources, previous research has shown that a better perception of one’s bodily signals may facilitate the use of adaptive emotion regulation strategies [42]. Those skills might empower adolescents to control their disruptive behaviors in the classroom.

    Design and Hypotheses

    Our primary aim in this study was to investigate the effect of DEEP on daily levels of state-anxiety and disruptive classroom behavior in a clinical sample. It was expected that playing DEEP would reduce both participants’ state-anxiety and disruptive classroom behavior. Second, we explored the duration of the calm or relaxed state of participants after playing DEEP. No specific hypotheses were formed as this is the first study examining the duration of the effect of playing DEEP. To meet the research aims, we conducted a single-case experimental design (SCED) study in a special school setting. The SCED methodology was particularly appropriate because it allowed us to evaluate the efficacy of an intervention for individuals with heterogeneous characteristics (as is the case in special education [43,44]). This study followed an ABAB design, in which participants’ state-anxiety and disruptive classroom behavior in the absence of the intervention (A or baseline-withdrawal phase) were compared with participants’ state-anxiety and disruptive behavior during the intervention (B or intervention phase).

    Disruptive classroom behavior outcomes were personalized per adolescent as difficulties to regulate oneself may manifest differently in each individual. This idiographic assessment approach may provide insight into whether the intervention is impacting the problems that teachers frequently observe and consider most important [45]. The Single-Case Reporting Guideline in Behavioural Interventions [46] was used to report this study.


    Participants

    Participants were 8 adolescents (mean age 14.67, SD 1.83 years) attending a secondary special school for students with behavioral and psychiatric problems in the northern part of the Netherlands. Adolescents were considered eligible for the study if teachers had the impression that the adolescent showed symptoms of anxiety, displayed disruptive behaviors in the classroom, and could handle the burden of completing momentary questionnaires (ie, 3 times a day). In total, teachers put forward 10 eligible adolescents. A school clinician contacted the parents of those 10 adolescents by phone to inform them about the study goals and to invite their child for participation. A total of 8 adolescents and their parents expressed their interest to participate in the study. The research team contacted these parents by phone to provide detailed information about the study’s procedure. All adolescents and their parents provided initial verbal consent and received an information and consent letter. Parental written informed consent was sent by mail. Individual demographic characteristics including participants’ age, gender, educational level, pretest trait-anxiety score, diagnoses, current medication, and treatment are provided in Table 1. All participants were of Dutch descent.

    Table 1. Individualized description of participants’ demographics at pretest.
    ParticipantAge (years)GenderEducational levelTrait-anxiety scoresaDiagnosesbCurrent medicationCurrent treatment
    112.94MaleLower secondary vocational education44ADHDc and ASDdMethylphenidateNone
    212.91MaleLower secondary vocational education31ADHD and ASDAtomoxetineNone
    313.90MaleLower and higher secondary vocation37ADHD and ASDMethylphenidatePsychotherapy
    413.03MaleLower and higher secondary vocation22RADe, ODDf, and ADHDNonePsychomotor therapy
    516.48MaleMiddle level vocational education47ASDNoneNone
    617.34MaleLower secondary education36ASDNoneAmbulatory care offered by the school clinicians
    716.52MaleLower secondary education36ADHD and ASDMethylphenidategPsychotherapy
    814.22FemaleLower secondary education27ASD, PDh, SADi, and EDjNoneNone

    aTo gain a general impression of adolescents’ predisposition toward anxiety, the trait-scale of the State-Trait Anxiety Inventory for Children [47] was assessed at pretest. Trait-anxiety scores can range from 20 to 60, with higher scores reflecting higher levels of trait-anxiety. Previous research suggests that children with subclinical and clinical levels of anxiety score on average 32.8 (SD 7.2) and 35.8 (SD 8.1) on the trait-scale, respectively [48,49].

    bDiagnoses were derived from the electronic school database by the school clinician.

    cADHD: attention-deficit/hyperactivity disorder.

    dASD: autism spectrum disorder.

    eRAD: reactive attachment disorder.

    fODD: oppositional defiant disorder.

    gParticipant 7 stopped taking medication on day 18, 19, and 20 of the study.

    hPD: personality disorder.

    iSAD: social anxiety disorder.

    jED: eating disorder.

    Design

    This study followed an ABAB withdrawal or reversal design, with baseline (A0), intervention (B), and withdrawal or no intervention (A1) phases. A and B phases were alternated several times over a period of 4 weeks (ie, 20 schooldays in total). All participants started with the A0 baseline phase, which lasted for 5 or 6 days. After the baseline phase, the first intervention period (ie, B phase) began. The B phases usually lasted for 1 day, in which participants completed 1 DEEP session in the morning. However, some participants played DEEP on 2 subsequent days because the planned DEEP sessions did not synchronize with the participants’ schedule. The intervention days (ie, B phases) were alternated with withdrawal days, in which participants did not play DEEP (ie, A1 phases). All participants completed 6 DEEP sessions throughout the study period, except for participant 8 who completed 5 DEEP sessions because of illness. Another exception to this design was participant 5, who was doing an internship outside of the school for 2 days a week. Therefore, he participated in the study 3 days a week for 8 subsequent weeks (ie, 24 schooldays in total).

    Throughout all phases, state-anxiety and disruptive classroom behavior were assessed with paper-and-pencil questionnaires 3 times per day. Adolescents reported their state-anxiety around 10:00 AM, 12:00 PM, and 2:00 PM. However, during B phases, participants filled in the first state-anxiety questionnaire directly after the DEEP session (between 8:45 AM and 10:15 AM). Teachers reported what they had observed about participants’ disruptive classroom behavior in the past 2 hours around 10:20 AM, 12:25 PM, and 2:30 PM. During B phases, teachers reported what they had observed about participants’ behavior after participants came back from the DEEP session. Therefore, 4 assessments around 10:20 AM were missing because, in those cases, the participant finished the DEEP session around 10:15 AM.

    Participant 4 was observed by 3 teachers: 1 teacher observed for 4 days a week, 1 teacher observed for 1 day a week, and 1 teacher observed for 2 days in total when one of the other 2 teachers was ill. Participants 6 and 7 were classmates and were also observed by 2 teachers: 1 teacher observed for 4 days a week, and the other observed for 1 day a week. The remaining participants were observed by a single teacher. Participants 1 and 2 were classmates, so their behavior was observed by the same teacher.

    Procedure

    At the start of the study, interviews with the teachers were held to discuss specific disruptive classroom behaviors of each participant. On the basis of their input, a personalized questionnaire about each participant’s disruptive behavior was developed. Afterward, participants filled in questionnaires regarding their demographics and trait-anxiety at pretest. Then, participants completed the ABAB study that lasted for 4 weeks. The study procedure was followed twice; in the first block, participants 1 to 4 participated, and in the second block, participants 6 to 8 participated. Participant 5 participated in both blocks. After participation, participants and their teachers both received a monetary compensation of €25.00. A total of 2 teachers received a monetary compensation of €37.50 because they had observed 2 adolescents for 4 or 5 days a week. Ethical approval for this study was obtained from the Radboud University Ethics Committee Social Sciences (ECSW-2017-038R1).

    Measures

    State-Anxiety

    Adolescents’ state-anxiety was assessed with the 6-item short form of the state scale of the Dutch State-Trait Anxiety Inventory (STAI) [50,51]. The 6 items reflect how calm, tense, upset, relaxed, content, or worried one feels at the moment. To ensure that the questionnaire would match the cognitive abilities of the sample, the questionnaire was adapted to a visual analog scale, consisting of a line of 10.0 cm. State-anxiety scores could therefore range between 0.0 (eg, not tense) and 10.0 (very tense). Item 1, 4, and 5 were recoded, and a mean score across all items was calculated, with higher values indicating more anxious feelings. The state scale of the STAI has shown good reliability and acceptable validity among various populations, including adolescents ([52,53]; for short form: [51,54]), and internal consistency was good (Cronbach alpha=.78) in this study.

    Disruptive Classroom Behavior

    On the basis of the interviews with the teachers, 2 or 3 disruptive behaviors were defined per participant (Table 2). During the ABAB study, teachers indicated how often those behaviors occurred in the past 2 hours by rating the items on a 6-point Likert scale: 0=never, 1=rarely, 2=sometimes, 3=often, 4=very often, and 5=almost always. Items 1 and 2 from participant 8 were recoded as the desired effect for this participant would be an increase in behavior rather than a decrease.

    Table 2. Disruptive classroom behaviors of each participant, indicated by their teacher.
    ParticipantBehavior 1Behavior 2Behavior 3
    1Clears throat or sniffs noseAsks for confirmationaGets off the chair and walks out of the classroom
    2Talks (loudly) out of turnAsks for confirmationaGets off the chair and walks around the classroom
    3Taps fingers on the table or chair legPlays with or pulls hairGets off the chair and walks around or out of the classroom
    4Talks out of turnGets off the chair and walks around the classroomAsks for confirmationa
    5Looks around during an independent work hourTalks or laughs with classmates during an independent work hourAsks for confirmationa
    6Shouts or talks loudlyTalks out of turnN/Ab
    7Looks around during an independent work hourAsks for confirmationaN/A
    8Asks a questioncMakes contact with classmatescN/A

    aFor example: “Am I doing this right?” and “What are we going to do now?”.

    bN/A: not applicable, because the teachers mentioned only 2 disruptive classroom behaviors for these participants.

    cThe desired effect for participant 8 was an increase in behavior rather than a decrease.

    Intervention

    Participants completed DEEP sessions in a separate room at school. Upon arrival, participants first sat in a turnaround desk chair after which the DEEP breathing belt was placed around the abdomen. The DEEP belt contains an Arduino-compatible FLORA wearable electronic platform [55] that opens Arduino software [56] to keep track of real-time belt values. Afterward, the HTC Vive VR headset was adjusted to fit each player, and headphones were plugged in for the games’ music. Before starting the game, participants were instructed that they could move in the game by breathing deeply through their belly and by turning around in their chair. Finally, DEEP was started on a laptop, and a timer was started to keep track of time. Participants could play DEEP for 15 min but were allowed to stop earlier if they wanted to. On average, participants played DEEP for 12.41 (SD 3.76) min.

    Analyses

    Means and standard deviations of A0, B, and A1 phase assessments of state-anxiety were calculated for each participant. In addition, the percentage of data points exceeding the median (PEM [57]) was computed for each participant. These scores reflect the percentage of data points in the B phases below the median of the A0 baseline phase. Afterward, a recent review [58] was used to choose the most appropriate analyses for the state-anxiety data.

    To gain a general impression of the data, a visual analysis was performed involving the examination of trend (for the A0 baseline phase only), variability, and level [59]. Visual analysis was necessary to assess if the data pattern corresponded to what is expected from a baseline (ie, no stable upward or downward trend) and effective intervention (ie, less variability in B phases compared with the A0 baseline phase; reduction of the level of anxiety over the course of the intervention [58]). To assess if baseline data showed no stable upward or downward trend, trend stability envelopes [60] were calculated for each participant via a Web-based calculator [61]. Lane and Gast [60] proposed that stable trends are present when at least 80% of the data points fall within the envelope defined by the split-middle trend line plus or minus 25% of the baseline median. To compare variability in the B phases with variability in the A0 baseline phase, the median absolute deviation (MAD) was computed for participants’ A0 baseline phase and the B phases. To calculate the MAD of the B phases, data points of all B phases were merged together for each participant. The MAD was calculated on the 6 STAI items instead of participants’ mean anxiety to optimally represent participants’ variability.

    To assess changes in level, a visual analysis usually focuses on mean score differences between phases [59]. However, this strategy has some limitations. First, mean score differences between phases do not take the presence of potential carryover effects into consideration, which refers to the continued impact of an intervention on subsequent (A1) phases [62]. Especially when adolescents have played DEEP several times, the intervention may have an irreversible effect on adolescents’ breathing skills. Second, the ABAB design consisted of several relatively short phases that also differed in length, making mean score comparisons between phases questionable. As such, it was decided to examine if participants’ level of anxiety decreased over time, regardless of phase, thereby keeping the temporal structure of the data intact. Levels of anxiety were classified with recursive partitioning, which is a data-driven approach that uses regression trees to identify segments of a time-series (of at least two data points) that have a stable mean value [63]. Recursive partitioning was performed with the ts_levels function of the casnet package (version 0.1.3, developed by Hasselman [64]) in R [65].

    After the visual analysis, nonoverlap of all pairs (NAP [66]) was calculated, which reflects the proportion of NAP comparisons across A and B phases. NAP was calculated as the area under the curve percentage from a receiver operating characteristic analysis in IBM SPSS Statistics 21. NAP ranges of 0 to 0.65 are considered as weak effects, 0.66 to 0.92 as medium effects, and greater than 0.92 as strong effects [66]. The first NAP analysis included all A and B phases. To account for potential carryover effects, a second NAP analysis was conducted comparing the data points in the A0 baseline phase only with the data points in the B phases.

    Finally, the between-case standardized mean difference (BC-SMD [67]) was computed to give an overall quantitative summary of the magnitude of change from A to B phases on a group level. The BC-SMD is an effect size designed specifically for SCED studies and can be compared with effect sizes for between-group designs. Moreover, the BC-SMD accounts for repeated intraindividual assessments and between- and within-case variances and includes a correction for a small sample bias [68,69]. The BC-SMD assumes that the outcome measurement is normally distributed and that there are no clear baseline trends [70]. A total of 4 separate BC-SMD quantifications were obtained. First, an overall change in level from all A to B phases was computed. Afterward, to investigate the duration of the effect of each DEEP session, we obtained overall changes in level from A to B phases measured around 10:00 AM, 12:00 PM, and 2:00 PM separately. The BC-SMD was calculated via a Web-based calculator, developed by Pustejovsky (version 0.3.1 [71]). Restricted maximum likelihood estimation was used to generate the design-comparable model effect size as it is the most flexible estimation model [72]. A random effect for treatment level was specified, which permits the treatment effect to vary across cases. Effect sizes of 0.20, 0.50, or 0.80 were interpreted as small, medium, or large effect sizes, respectively [73].

    The same procedure, as described earlier, was repeated for classroom behavior. As NAP scores revealed the same pattern of results for all observed behaviors within the majority of participants, it was decided to only report the results of behavior 1 of each participant. However, for participants 2 and 3, behavior 2 was reported, and for participant 5, behavior 3 was reported because these were the only behaviors that showed medium effects for these participants; all other behaviors of these participants showed no effect.

    In terms of missingness, 133 (27.0%) out of the 492 state-anxiety assessments were missing for reasons such as practical lessons, national holidays, or illness. Furthermore, 161 (32.7%) out of the 492 disruptive classroom behavior assessments were missing because of the absence of the adolescent (eg, practical lessons in front of another teacher) or lack of time. We decided not to use imputation strategies, as these percentages of missing data may not affect the quality of statistical inferences [74]. Moreover, the BC-SMD is a robust analysis strategy as it accounts for missing observations [70].


    Anxiety

    Descriptive Statistics

    Table 3 presents means and SDs on A0, B, and A1 phase assessments of anxiety and PEM scores for each participant. Overall, participants reported a mean anxiety of 3.32 (SD 1.81) in the A0 baseline phase, 2.35 (SD 1.48) in the B phases, and 2.30 (SD 1.31) in the A1 phases. Thus, participants’ mean anxiety scores fell within one-third of the full range of the scale in all phases. The findings indicate that participants’ anxiety scores may have declined over the course of the intervention. The minor difference between the participant’s anxiety in A1 and B phases suggests that carryover effects may have occurred. PEM scores ranged between 61% (11/18) and 100% (17/17), indicating that most assessments of anxiety were lower on the days of DEEP sessions compared with the A0 baseline phase.

    Table 3. Means and standard deviations of anxiety by phase and percentage of data points exceeding the median scores for each participant.
    ParticipantA0 baseline phase, mean (SD)B phases, mean (SD)A1 phases, mean (SD)Data points exceeding the median, n (%)
    15.49 (2.24)3.32 (1.79)3.55 (1.47)11 (92)
    21.95 (1.59)1.19 (1.60)0.91 (1.07)12 (80)
    33.17 (1.68)1.52 (0.90)1.79 (1.02)17 (100)
    41.98 (0.56)1.68 (0.33)2.21 (0.71)14 (82)
    54.11 (1.45)2.75 (1.78)1.95 (0.51)12 (86)
    62.91 (1.49)2.52 (1.16)3.45 (1.13)11 (61)
    73.30 (1.29)2.62 (1.27)2.40 (0.99)14 (78)
    83.73 (0.99)3.58 (1.21)4.00 (1.19)10 (67)
    Visual Analysis

    Each participant’s mean anxiety score is represented graphically in Figure 2. Trend stability envelopes revealed no stable upward or downward baseline trend for all participants (Table 4). The MADs showed that the variability in the B phases compared with the A0 baseline phase seemed to decrease for half of the participants (ie, participants 2, 3, 4, and 7; Table 4). In contrast, the data patterns of the other half (ie, participants 1, 5, 6, and 8) did not correspond to what is expected from an effective intervention as the variability in anxiety during the B phases compared with the A0 baseline phase increased or remained the same.

    Finally, the changes in levels of anxiety that were identified by recursive partitioning are represented in Figure 2. Anxiety scores seemed to decrease for half of the participants (ie, participants 1, 3, 5, and 7) over the course of the intervention. However, scores of participant 7 should be interpreted cautiously as the decrease in anxiety seemed to correspond with the days he stopped taking his medication. A potential floor effect was detected for participant 2. This floor effect was considered as a desired effect as the levels of anxiety stabilized over the course of the intervention for this participant. No changes in anxiety level were identified for participants 4, 6, and 8 (Table 4).

    ‎
    Figure 2. The effect of DEEP on anxiety for 8 adolescents. Every 3 data points represent 1 day (measured around 10:00 AM, 12:00 PM, and 2:00 PM). The dashed and stepped lines represent the relatively stable anxiety levels that were identified using recursive partitioning.
    Table 4. Results from visual analysis (baseline trend, variability, and change in level) and nonoverlap of all pairs on anxiety.
    ParticipantA0 baseline trendVariabilityChange in levelComparison: A vs BComparison: A0 baseline vs B

    		Slope (% of data points within envelope)MADa A0 phaseMAD B phases
    		NAPb95% CIcNAP95% CIc
    1−0.11 (67)3.413.85Seems to decrease0.72d0.55-0.890.82d0.66-0.98
    20.04 (20)0.590.30Potential floor effect0.630.45-0.820.74d0.55-0.93
    3−0.16 (47)2.821.04Seems to decrease0.66d0.51-0.810.80d0.64-0.96
    40.03 (63)2.971.63No change in level0.75d0.61-0.880.68d0.49-0.87
    5−0.21 (38)1.481.48Seems to decrease0.580.39-0.770.81d0.63-0.98
    6−0.11 (33)1.852.00No change in level0.650.48-0.830.590.37-0.80
    7−0.35 (43)2.001.63Partly decreases0.550.38-0.720.67d0.48-0.86
    8−0.02 (60)2.523.56No change in level0.580.39-0.770.560.35-0.77

    aMAD: median absolute deviation.

    bNAP: nonoverlap of all pairs.

    cConfidence intervals are asymptotic.

    dMedium effect.

    Nonoverlap of All Pairs

    NAP scores including all A and B phases are presented in Table 4. Participants 1, 3, and 4 showed changes in the medium range, indicating that their anxiety scores were lower during the B phases compared with the A phases. Regarding NAP scores including A0 baseline and B phases only participants 1, 2, 3, 4, 5, and 7 showed changes in the medium range. Carryover effects may have occurred for participants 2, 5, and 7 as they showed medium effects in the A0 baseline vs B phase comparison but no effect in the A vs B comparison.

    Between-Case Standardized Mean Difference

    The assumptions of normality and absence of clear baseline trends were tested and met. Therefore, the BC-SMD analysis was deemed appropriate. The overall A vs B comparison yielded an effect size of d=−0.29 (SE 0.11; 95% CI −0.51 to −0.08). This d statistic indicates a small but significant reduction of anxiety with the introduction of DEEP [73]. The A vs B comparison yielded a value of d=−0.43 (SE 0.15; 95% CI −0.74 to −0.15) for the data measured around 10:00 AM, a value of d=−0.34 (SE 0.17; 95% CI −0.68 to −0.02) for the data measured around 12:00 PM, and a value of d=−0.02 (SE 0.17; 95% CI −0.34 to 0.30) for the data measured around 2:00 PM. These d statistics indicate a medium significant reduction of anxiety directly after gameplay, a small but significant reduction 2 hours after gameplay, and no reduction 4 hours after gameplay (see Multimedia Appendix 2 for a graphical representation). Thus, the calm or relaxed state of participants after a DEEP session lasted for 2 hours on average.

    Disruptive Classroom Behavior

    Descriptive Statistics

    Table 5 presents means and SDs on A0, B, and A1 phase assessments of disruptive classroom behavior and PEM scores for each participant. Overall, participants reported a mean disruptive classroom behavior score of 2.60 (SD 1.72) in the A0 baseline phase, 2.11 (SD 1.53) in the B phases, and 2.21 (SD 1.49) in the A1 phases. These findings indicate a small decrease in participants’ disruptive classroom behavior during the intervention. The minor difference between participants’ classroom behavior in A1 and B phases suggests that carryover effects may have occurred. PEM scores ranged between 0% (0/11) and 77% (10/13), indicating that the effect of DEEP on classroom behavior varied between participants.

    Table 5. Means and standard deviations of disruptive classroom behavior by phase and percentage of data points exceeding the median scores for each participant.
    ParticipantA0 baseline phase, mean (SD)B phases, mean (SD)A1 phases, mean (SD)Data points exceeding the median, n (%)
    1 - behavior 10.46 (0.66)0.91 (1.14)1.17 (1.64)0 (0)
    2 - behavior 22.23 (1.30)1.38 (0.65)1.06 (0.68)7 (54)
    3 - behavior 22.93 (0.83)2.27 (0.96)2.23 (0.75)8 (53)
    4 - behavior 13.00 (1.10)1.31 (1.25)2.50 (1.58)10 (77)
    5 - behavior 31.93 (0.62)1.21 (0.80)1.63 (0.50)8 (57)
    6 - behavior 12.00 (1.26)2.38 (0.89)2.14 (0.86)2 (13)
    7 - behavior 12.22 (1.20)2.40 (1.30)2.64 (0.50)3 (20)
    8 - behavior 16.00 (0.00)5.27 (0.47)5.78 (0.44)8 (73)
    Visual Analysis

    Each participants’ classroom behavior score is represented graphically in Figure 3. Trend stability envelopes revealed no stable upward or downward baseline trend for all participants (Table 6). The MADs showed that the variability in the B phases compared with the A0 baseline phase decreased only for participant 6. However, this finding should be interpreted cautiously as the level of disruptive behavior for this participant seemed to increase over the course of the intervention. The data pattern of all other participants did not correspond to what is expected from an effective intervention as variability in disruptive behaviors in the B phases compared with the A0 baseline phase increased or remained the same (Table 6).

    Finally, the changes in relatively stable levels of disruptive classroom behavior that were identified by recursive partitioning are represented in Figure 3. Desired effects were found for half of the participants (ie, participants 2, 4, 5, and 8) as their disruptive behavior seemed to decrease over the course of the intervention. All other participants did not show any desirable effects in terms of their behavior (Table 6).

    ‎
    Figure 3. The effect of DEEP on disruptive classroom behavior for 8 adolescents. Every 3 data points represent 1 day (measured around 10:20 AM, 12:25 PM, and 2:30 PM). The dashed and stepped lines represent the relatively stable disruptive classroom behavior levels that were identified using recursive partitioning.
    Table 6. Results from visual analysis (baseline trend, variability, and change in level) and nonoverlap of all pairs on disruptive classroom behavior.
    ParticipantA0 baseline trendVariabilityChange in levelComparison: A vs BComparison: A0 baseline vs B

    		Slope (% of data points within envelope)MADa A0 phaseMAD B phases
    		NAPb95% CIcNAP95% CIc
    1 - behavior 1−0.07 (0)0.001.48Partly increases0.450.24-0.650.400.16-0.63
    2 - behavior 20.07 (38)1.481.48Seems to decrease0.500.31-0.680.68d0.47-0.89
    3 - behavior 20.00 (57)0.001.48No change in level0.570.40-0.750.69d0.50-0.89
    4 - behavior 10.12 (44)1.481.48Partly decreases0.77d0.62-0.910.83d0.68-0.98
    5 - behavior 30.00 (64)0.001.48Partly decreases0.68d0.50-0.860.73d0.55-0.92
    6 - behavior 10.00 (36)1.480.00Seems to increase0.410.23-0.590.400.17-0.63
    7 - behavior 10.30 (67)1.481.48No change in level0.520.31-0.730.450.21-0.69
    8 - behavior 10.00 (100)0.000.00Seems to decrease0.82d0.64-0.990.86d0.70-1.00

    aMAD: median absolute deviation.

    bNAP: nonoverlap of all pairs.

    cConfidence intervals are asymptotic.

    dMedium effect.

    Nonoverlap of All Pairs

    NAP scores including all A and B phases are presented in Table 6. Participants 4, 5, and 8 showed changes in the medium range, indicating that their disruptive classroom behavior scores were lower during the B phases compared with the A phases. Regarding NAP scores including A0 baseline and B phases only, participants 2, 3, 4, 5, and 8 showed changes in the medium range. Carryover effects in terms of participants’ classroom behavior may have occurred for participants 2 and 3, as they showed medium effects in the A0 baseline vs B comparison but no effect in the A vs B comparison.

    Between-Case Standardized Mean Difference

    The assumptions of normality and absence of clear baseline trends were tested and met. The overall A vs B comparison of the BC-SMD analysis yielded a value of d=−0.16 (SE 0.12; 95% CI −0.39 to 0.05), indicative of a small, nonsignificant reduction of participants’ disruptive behavior with the introduction of DEEP [73].


    Principal Findings

    This SCED study evaluated the efficacy of the virtual reality biofeedback game, DEEP, as an intervention to reduce anxiety and disruptive classroom behaviors in adolescents in a special school setting. The primary aim of the study was to test the effect of playing DEEP on daily levels of state-anxiety and disruptive classroom behavior. On a group level, results indicated a small-sized reduction of state-anxiety after the introduction of DEEP. On the individual level, strong evidence was found for 5 out of 8 participants as their NAP scores indicated a medium-sized reduction and their level of anxiety seemed to decrease over the course of the intervention. Moderate evidence was found for 1 participant, as NAP scores indicated a medium-sized reduction, but no change was found in the level of anxiety. In terms of disruptive classroom behavior, a small, nonsignificant reduction was found on the group level. On the individual level, strong evidence was found for 4 out of 8 participants as their NAP scores indicated a medium-sized reduction and their level of disruptive behavior seemed to decrease. Moderate evidence was found for 1 participant as the NAP score indicated a medium change, but no change was found in the level of disruptive behavior. The secondary aim of our study was to investigate the duration of the calm or relaxed state of participants after playing DEEP. Results indicated that, on average, the effect of playing DEEP lasted for 2 hours.

    The Effect of DEEP on State-Anxiety

    In line with our hypothesis, the current findings indicate that, on a group level, DEEP reduces daily levels of state-anxiety. These results corroborate the previous pilot study conducted by Van Rooij et al [22], who found that playing DEEP for 7 min reduced levels of state-anxiety directly after the game. This study adds that DEEP also holds potential as an intervention for anxiety in adolescents with various disorders such as ADHD or ASD. Even though the direct effects of the evidence-based clinical techniques (ie, diaphragmatic breathing and biofeedback) incorporated in DEEP require further investigation, the techniques may have enabled individuals to gain awareness about and control over their diaphragmatic breathing, enabling them to regulate or dampen high arousal levels [27].

    This study is the first study examining the duration of the effect of DEEP on state-anxiety and found that, on average, the calm or relaxed state of participants after playing DEEP lasted for 2 hours. This duration indicates that the effect of playing DEEP does not persist through the whole school day but may be particularly valuable to use in specific anxiety-provoking situations in class, such as during exams or when giving a speech. Although this study provides insight into the duration of the effect of DEEP on a group level, insight into individual variability in the duration of the effect is lacking. The NAP and visual analysis strategies used in this study are not suitable to analyze the duration of an effect on an individual level. It is possible that some individuals mainly reported reduced levels of state-anxiety directly after gameplay, whereas others still felt calmer or more relaxed at the end of a day playing DEEP. In addition, it is unknown if there are differences in the duration of the effect of DEEP between the 6 sessions. Participant 6, for example, showed a steep increase in anxiety during the day after the first and second DEEP session but seemed to feel calm or relaxed for a longer period after the last couple of DEEP sessions (Figure 2). Future studies could use more assessments per day and multilevel modeling techniques such as the N-of-1 randomized controlled trial methodology [75] to identify differences in the duration of the effect of DEEP between individuals and to investigate if the duration of the effect of DEEP might change over the course of the sessions within each individual.

    The Effect of DEEP on Disruptive Classroom Behavior

    Analyses yielded mixed results for the efficacy of DEEP to reduce levels of disruptive classroom behavior. The results partly confirmed our hypothesis as 5 out of 8 individuals showed a reduction of disruptive classroom behavior after the introduction of DEEP. DEEP may have affected participants’ behavior in various ways. First, as high levels of anxiety may underlie escape-driven disruptive behaviors [6,38], participants in our study may have felt less anxious after playing DEEP and, as a result, were less prone to show disruptive behaviors in the classroom. Participant 5, for example, reported reduced levels of state-anxiety and was less likely to ask his teacher for constant confirmation on the days of DEEP sessions. Although findings need to be interpreted cautiously, participant 5 may have felt more relaxed after playing DEEP, enabling him to allow worrisome thoughts to pass through his mind (as hypothesized by Hayes et al [76]), which in turn may have led to a decreased need for constant confirmation from his teacher.

    Second, DEEP may have reduced participants’ disruptive behavior through its effect on interoceptive awareness. Although empirical evidence is yet lacking, it is likely that participants’ interoceptive awareness improved over the course of the intervention because participants were continuously informed about their stage of breathing while playing DEEP. It has been theorized that increased interoceptive awareness may enable individuals to mobilize self-regulation resources [41], which are needed to regulate one’s behavior. Participants that benefited from DEEP in terms of their behavior may have been increasingly aware of their bodies and physiological responses, which may have enabled them to regulate their emotions and behavior in the classroom. To illustrate, after the intervention period, participant 8 noted that she learned to “...pay attention to my breathing more often, for example when I stress about an exam. It happened once, for an English exam. I tried to breathe in and out more deeply.” Supposedly, this participant became more aware of her bodily response to stress and tried to regulate heightened arousal levels by breathing. This self-regulation strategy may have helped her to cope with stressful situations in class, resulting in an increase in her participation (eg, asking questions to the teacher) in the classroom. Nevertheless, these explanations remain speculative; future studies are warranted to examine the underlying mechanisms (eg, interoceptive awareness and self-regulation) by which DEEP might affect disruptive behaviors.

    Contrary to our hypothesis, we found a small, nonsignificant effect of DEEP on disruptive behavior on a group level. The 3 participants that did not seem to benefit from DEEP in terms of their behavior may have cancelled out the effect of the participants that did, leading to an unobservable effect on a group level. There are several possible explanations why DEEP did not affect the behaviors of those participants. First, anxiety may not have been the cause of disruptive behaviors in the classroom for these individuals. Rather, neurological deficits in individuals with ADHD or ASD that are associated with attention-related problems [77-79] may have caused the disruptive behaviors of the individuals that did not improve. Hence, DEEP may not have been targeting the underlying mechanisms responsible for the disruptive behaviors of these adolescents. Second, there may have been individual differences in the extent to which participants were actually engaging with the diaphragmatic breathing mechanic while playing DEEP. Some individuals may not have developed the skill to breathe through their diaphragm, limiting them in their ability to regulate their behaviors in the classroom. Future studies are encouraged to investigate if individual profiles in learning diaphragmatic breathing skills (as measured by in-game diaphragm expansions) might underlie changes in disruptive behavior [22].

    Strengths, Limitations, and Future Directions

    A clear strength of this investigation is that the study was conducted in a special school setting, thereby addressing ecological validity issues relevant to the school setting. Another strength is that we used an SCED, which is a suitable design to test interventions in the heterogeneous special school population [43,44]. In addition, the study followed all external validity recommendations for SCED studies as described in the Risk of Bias in N-of-1 Trials (RoBiNT [80]).

    Regarding the RoBiNt recommendations for internal validity, not all recommendations were carried out. First of all, we did not randomize the beginning of the study phases. We explicitly chose not to randomize the conditions because of constraints of the school setting (eg, scheduling conflicts) and characteristics of the target group (eg, in need of structure). In terms of sampling of the target behaviors, the required minimum of 3 data points in each study phase [80] was not always met because of missing data. Another limitation was that neither participants nor teachers were blinded to the study phases as the intervention required that participants leave the classroom to play DEEP in a separate room. The lack of blinding may have caused bias in teacher and child reports as they knew DEEP was supposed to reduce anxiety and disruptive classroom behaviors. In addition, inter-rater reliability measures of the target behaviors were not computed as no blinded assessors were used. The study was also limited because the behaviors of participants 4, 6, and 7 were assessed by multiple teachers, thereby reducing the reliability of those observations. Future studies may benefit from blinded assessors to reduce bias, and it is also recommended to tailor the units of measurement to the specific disruptive behaviors as some of the disruptive behaviors identified in this study would be better suited to rate or frequency measures rather than duration.

    Although this study provided initial insight into the efficacy of DEEP in a special school setting, future research is needed to fully optimize DEEP as an intervention for this heterogeneous population. The intervention effects of DEEP could be optimized using a parametric analysis of the optimal amount of play time for a given individual [81]. Moreover, the active intervention elements of DEEP (eg, diaphragmatic breathing and biofeedback) that are necessary to produce behavior change should be determined in future studies. Although there are problems in terms of specificity when using physiological measures, adding physiological measures to detect changes in arousal during and after playing DEEP may also help uncover the potential working mechanism of DEEP. In addition, the conditions under which DEEP may be most successful for certain individuals could be established by replicating the current single-case study in new settings. Some individuals may benefit most from playing DEEP right before (eg, exams) or after (eg, quarrels) stressful events, whereas others may show optimal effects when they play DEEP at fixed times. Finally, larger group studies and replicated single-case research may yield variables that moderate intervention outcomes such as trait-anxiety scores, comorbid disorders, age, and gender, which could be used to personalize DEEP.

    Implications

    The aforementioned limitations notwithstanding, this study has important implications for clinical practice. Although it should be noted that findings need to be interpreted cautiously in a study of this type, this study demonstrated the potential of DEEP as an intervention to reduce daily levels of state-anxiety and disruptive classroom behavior in a special school setting. The results implicate the clinical techniques of diaphragmatic breathing and biofeedback in treating individuals with anxiety. Moreover, the results demonstrate that an applied game incorporating those techniques can be used either in isolation or as an add-on to existing interventions in a clinical sample. On overage, our results showed that the calm or relaxed state of participants after playing DEEP lasted for 2 hours. Therefore, school clinicians are recommended to tailor implementation strategies of DEEP to the individual with different needs early or later in the day, considering individual variation in the duration of the effect of DEEP. The implementation of game-based interventions might be a promising avenue for the special school setting, as video games can be tailored to the diverse needs and learning paces of the heterogeneous special school population.

    This study also has strong implications for future research on behavioral health interventions. This study demonstrated that there are individual differences in the extent to which an applied game to enhance mental health is effective. Therefore, we need to tailor behavioral interventions to the personal needs of different individuals. Although the randomized controlled trial may be the golden standard in behavioral health intervention research, this costly and time-invasive method is limited because of a lack of attention paid to individual differences [82,83]. We advocate the use of SCED research to optimize health interventions through ongoing tailoring and testing [81]. Single-case designs provide an excellent opportunity to dynamically and efficiently assess the most promising intervention elements through systematic replication of single-case experiments [81]. Moreover, single-case designs may enhance our understanding of the conditions under which interventions are most successful by replicating single-case studies in new settings, thereby also establishing its generality. Knowledge derived from a body of SCED research about certain demographic and diagnostic variables, generality, and the conditions that may influence intervention outcomes may eventually enable (school) clinicians to personalize behavioral health interventions to the unique characteristics of the individual.

    Acknowledgments

    The authors would like to gratefully acknowledge the developers of DEEP, Owen Harris and Niki Smit. They would also like to thank the participating school, school clinicians, teachers, and adolescents. This work was supported by funding from the Netherlands Organisation for Scientific Research (NWO, grant number 406-16-524), the Netherlands Organisation for Health Research and Development (ZonMw, grant number 912-15-207), and by funding from the Radboud University, Behavioural Science Institute. The funding sources had no role in the design of the study, data collection, analyses, interpretation of data, writing the manuscript, and in the decision to submit the study for publication.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    DEEP trailer.

    MP4 File (MP4 Video), 185651 KB
    Multimedia Appendix 2

    Participants’ mean anxiety scores measured around 10:00 AM, 12:00 PM, or 2:00 PM, split out by phases.

    PNG File , 10 KB
    1. van Rijswijk CM. Indicatiestelling speciaal onderwijs 2003/2004. De indicatiecommissies van cluster vier [Needs assessment special education 2003/2004. The needs assessment committee of cluster four]. Den Haag, The Netherlands. Landelijke Commissie Toezicht Indicatiestelling; 2004.
    2. Vonck E, Scholte E. Angstig depressieve kinderen in het speciaal onderwijs [Anxious depressed children in special education]. Amsterdam, The Netherlands. SWP; 2008.
    3. van Steensel FJA, Bögels SM, Perrin S. Anxiety disorders in children and adolescents with autistic spectrum disorders: a meta-analysis. Clin Child Fam Psychol Rev. Sep 2011;14(3):302-317. [FREE Full text] [CrossRef] [Medline]
    4. Schatz DB, Rostain AL. ADHD with comorbid anxiety: a review of the current literature. J Atten Disord. Nov 2006;10(2):141-149. [CrossRef] [Medline]
    5. Woodward LJ, Fergusson DM. Life course outcomes of young people with anxiety disorders in adolescence. J Am Acad Child Adolesc Psychiatry. Sep 2001;40(9):1086-1093. [CrossRef] [Medline]
    6. Minahan BJ, Rappaport N. Anxiety in students a hidden culprit in behavior issues. Phi Delta Kappan. 2012;94(4):34-39. [CrossRef]
    7. Carthy T, Horesh N, Apter A, Gross JJ. Patterns of emotional reactivity and regulation in children with anxiety disorders. J Psychopathol Behav Assess. 2010;32(1):23-36. [CrossRef]
    8. Neil AL, Christensen H. Efficacy and effectiveness of school-based prevention and early intervention programs for anxiety. Clin Psychol Rev. Apr 2009;29(3):208-215. [CrossRef] [Medline]
    9. Silver M, Oakes P. Evaluation of a new computer intervention to teach people with autism or Asperger syndrome to recognize and predict emotions in others. Autism. Sep 2001;5(3):299-316. [CrossRef] [Medline]
    10. Weiss M, Safren SA, Solanto MV, Hechtman L, Rostain AL, Ramsay JR, et al. Research forum on psychological treatment of adults with ADHD. J Atten Disord. May 2008;11(6):642-651. [CrossRef] [Medline]
    11. Salloum A, Johnco C, Lewin AB, McBride NM, Storch EA. Barriers to access and participation in community mental health treatment for anxious children. J Affect Disord. May 15, 2016;196:54-61. [CrossRef] [Medline]
    12. Mennuti RB, Christner RW. An introduction to cognitive-behavioral therapy with youth. In: Mennuti RB, Christner RW, Freeman A, editors. Cognitive-Behavioral Interventions in Educational Settings: A Handbook for Practice. Second Edition. New York. Routledge; 2012:13-30.
    13. Beidas RS, Mychailyszyn MP, Edmunds JM, Khanna MS, Downey MM, Kendall PC. Training school mental health providers to deliver cognitive-behavioral therapy. School Ment Health. Dec 1, 2012;4(4):197-206. [FREE Full text] [CrossRef] [Medline]
    14. Granic I, Lobel A, Engels RC. The benefits of playing video games. Am Psychol. Jan 2014;69(1):66-78. [CrossRef] [Medline]
    15. Langley AK, Nadeem E, Kataoka SH, Stein BD, Jaycox LH. Evidence-based mental health programs in schools: barriers and facilitators of successful implementation. School Ment Health. Sep 2010;2(3):105-113. [FREE Full text] [CrossRef] [Medline]
    16. Mychailyszyn MP, Brodman DM, Read KL, Kendall PC. Cognitive-behavioral school-based interventions for anxious and depressed youth: a meta-analysis of outcomes. Clin Psychol Sci Pract. 2012;19(2):129-153. [CrossRef]
    17. Werner-Seidler A, Perry Y, Calear AL, Newby JM, Christensen H. School-based depression and anxiety prevention programs for young people: a systematic review and meta-analysis. Clin Psychol Rev. Feb 2017;51:30-47. [FREE Full text] [CrossRef] [Medline]
    18. Chalfant AM, Rapee R, Carroll L. Treating anxiety disorders in children with high functioning autism spectrum disorders: a controlled trial. J Autism Dev Disord. Nov 2007;37(10):1842-1857. [CrossRef] [Medline]
    19. Manassis K, Tannock R, Young A, Francis-John S. Cognition in anxious children with attention deficit hyperactivity disorder: a comparison with clinical and normal children. Behav Brain Funct. Jan 15, 2007;3:4. [FREE Full text] [CrossRef] [Medline]
    20. Crenshaw DA. Therapeutic Engagement of Children and Adolescents: Play, Symbol, Drawing, and Storytelling Strategies. Lanham, MD. Rowman & Littlefield; 2008.
    21. Kazdin AE. Technology-based interventions and reducing the burdens of mental illness: perspectives and comments on the special series. Cogn Behav Pract. 2015;22(3):359-366. [CrossRef]
    22. van Rooij M, Lobel A, Harris O, Smit N, Granic I. DEEP: A Biofeedback Virtual Reality Game for Children At-risk for Anxiety. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2016. Presented at: CHI EA'16; May 7 - 12, 2016:1989-1997; California, San Jose, USA.
    23. Weems CF, Zakem AH, Costa NM, Cannon MF, Watts SE. Physiological response and childhood anxiety: association with symptoms of anxiety disorders and cognitive bias. J Clin Child Adolesc Psychol. Dec 2005;34(4):712-723. [CrossRef] [Medline]
    24. Beard C. Cognitive bias modification for anxiety: current evidence and future directions. Expert Rev Neurother. Feb 2011;11(2):299-311. [FREE Full text] [CrossRef] [Medline]
    25. Vasey MW, Dadds MR. The Developmental Psychopathology of Anxiety. London. Oxford University Press; 2001.
    26. Alfano CA, Beidel DC, Turner SM. Cognition in childhood anxiety: conceptual, methodological, and developmental issues. Clin Psychol Rev. Nov 2002;22(8):1209-1238. [CrossRef] [Medline]
    27. Dworkin BR. Learning and Physiological Regulation. Chicago, IL, US. University Of Chicago Press; 1993.
    28. Martarelli D, Cocchioni M, Scuri S, Pompei P. Diaphragmatic breathing reduces exercise-induced oxidative stress. Evid Based Complement Alternat Med. 2011;2011:932430. [FREE Full text] [CrossRef] [Medline]
    29. Chen YF, Huang XY, Chien CH, Cheng JF. The effectiveness of diaphragmatic breathing relaxation training for reducing anxiety. Perspect Psychiatr Care. Oct 2017;53(4):329-336. [CrossRef] [Medline]
    30. Subbalakshmi NK, Adhikari P, Jeganathan PS. Comparative study on cardiac autonomic modulation during deep breathing test and diaphragmatic breathing in type 2 diabetes and healthy subjects. J Diabetes Investig. Jul 2014;5(4):456-463. [FREE Full text] [CrossRef] [Medline]
    31. Han JN, Stegen K, de Valck C, Clément J, van de Woestijne KP. Influence of breathing therapy on complaints, anxiety and breathing pattern in patients with hyperventilation syndrome and anxiety disorders. J Psychosom Res. Nov 1996;41(5):481-493. [CrossRef] [Medline]
    32. Gilbert C, Moss D. Biofeedback and biological monitoring. In: Moss DP, McGrady AV, Davies TC, Wickramasekera I, editors. Handbook of Mind-Body Medicine for Primary Care. Thousand Oaks, CA. Sage; 2003:109-122.
    33. Schoenberg PL, David AS. Biofeedback for psychiatric disorders: a systematic review. Appl Psychophysiol Biofeedback. Jun 2014;39(2):109-135. [CrossRef] [Medline]
    34. Weerdmeester J, van Rooij MM, Engels CM, Granic I. An integrative model for the effectiveness of biofeedback interventions for anxiety regulation. J Med Internet Res Prepints. 2019. [FREE Full text] [CrossRef]
    35. Jensen PS. Observations of students with disruptive behaviour in yoga classes and in the classroom. Int Sci Yoga J SENSE. 2014;4(4):21-44. [FREE Full text]
    36. Schuurmans AA, Nijhof KS, Engels RCM, Granic I. Using a videogame intervention to reduce anxiety and externalizing problems among youths in residential care: an initial randomized controlled trial. J Psychopathol Behav Assess. 2018;40(2):344-354. [FREE Full text] [CrossRef] [Medline]
    37. Schuurmans AA, Nijhof KS, Vermaes IP, Engels RC, Granic I. A pilot study evaluating 'Dojo,' a videogame intervention for youths with externalizing and anxiety problems. Games Health J. Oct 2015;4(5):401-408. [CrossRef] [Medline]
    38. Bubier JL, Drabick DA. Co-occurring anxiety and disruptive behavior disorders: the roles of anxious symptoms, reactive aggression, and shared risk processes. Clin Psychol Rev. Nov 2009;29(7):658-669. [FREE Full text] [CrossRef] [Medline]
    39. Polman H, de Castro BO, Koops W, van Boxtel HW, Merk WW. A meta-analysis of the distinction between reactive and proactive aggression in children and adolescents. J Abnorm Child Psychol. Aug 2007;35(4):522-535. [CrossRef] [Medline]
    40. Mehling WE, Price C, Daubenmier JJ, Acree M, Bartmess E, Stewart A. The Multidimensional Assessment of Interoceptive Awareness (MAIA). PLoS One. 2012;7(11):e48230. [FREE Full text] [CrossRef] [Medline]
    41. Teper R, Segal ZV, Inzlicht M. Inside the mindful mind: how mindfulness enhances emotion regulation through improvements in executive control. Curr Dir Psychol Sci. 2013;22(6):449-454. [CrossRef]
    42. Kever A, Pollatos O, Vermeulen N, Grynberg D. Interoceptive sensitivity facilitates both antecedent- and response-focused emotion regulation strategies. Pers Individ Differ. 2015;87:20-23. [CrossRef]
    43. Alnahdi GH. Single-subject designs in special education: advantages and limitations. J Res Spec Educ Needs. 2013;15(4):257-265. [CrossRef]
    44. Horner RH, Carr EG, Halle J, McGee G, Odom S, Wolery M. The use of single-subject research to identify evidence-based practice in special education. Except Child. 2005;71(2):165-179. [CrossRef]
    45. Weisz JR, Chorpita BF, Frye A, Ng MY, Lau N, Bearman SK, et al. Research Network on Youth Mental Health. Youth Top Problems: using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. J Consult Clin Psychol. Jun 2011;79(3):369-380. [CrossRef] [Medline]
    46. Tate RL, Perdices M, Rosenkoetter U, Shadish W, Vohra S, Barlow DH, et al. The Single-Case Reporting Guideline In Behavioural Interventions (SCRIBE) 2016 statement. Can J Occup Ther. Jun 2016;83(3):184-195. [CrossRef] [Medline]
    47. Spielberger CD. Manual for the State-Trait Anxiety Inventory for Children. Palo Alto, CA. Consulting Psychologists Press; 1973.
    48. Bell-Dolan DJ, Last CG, Strauss CC. Symptoms of anxiety disorders in normal children. J Am Acad Child Adolesc Psychiatry. Sep 1990;29(5):759-765. [CrossRef] [Medline]
    49. Perrin S, Last CG. Do childhood anxiety measures measure anxiety? J Abnorm Child Psychol. Dec 1992;20(6):567-578. [CrossRef] [Medline]
    50. Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI). Br J Clin Psychol. Sep 1992;31(3):301-306. [CrossRef] [Medline]
    51. van der Bij AK, de Weerd S, Cikot RJ, Steegers EA, Braspenning JC. Validation of the dutch short form of the state scale of the Spielberger State-Trait Anxiety Inventory: considerations for usage in screening outcomes. Community Genet. 2003;6(2):84-87. [CrossRef] [Medline]
    52. Hishinuma ES, Miyamoto RH, Nishimura ST, Nahulu LB, Andrade NN, Makini GK, et al. Psychometric properties of the state-trait anxiety inventory for Asian/Pacific-islander adolescents. Assessment. Mar 2000;7(1):17-36. [CrossRef] [Medline]
    53. Cao Y, Liu Z. Factor structure and factorial invariance of the State-Trait Anxiety Inventory for Chinese children and adolescents. Psych J. Jun 2015;4(2):74-87. [CrossRef] [Medline]
    54. Nilsson S, Buchholz M, Thunberg G. Assessing children's anxiety using the Modified Short State-trait Anxiety Inventory and Talking Mats: a pilot study. Nurs Res Pract. 2012;2012:932570. [FREE Full text] [CrossRef] [Medline]
    55. Adafruit Industries. URL: https://www.adafruit.com/product/659 [accessed 2020-01-10]
    56. Arduino. 2016. URL: https://www.arduino.cc/en/Main/Software [accessed 2020-01-10]
    57. Ma H. An alternative method for quantitative synthesis of single-subject researches: percentage of data points exceeding the median. Behav Modif. Sep 2006;30(5):598-617. [CrossRef] [Medline]
    58. Manolov R, Moeyaert M. Recommendations for choosing single-case data analytical techniques. Behav Ther. Jan 2017;48(1):97-114. [CrossRef] [Medline]
    59. Kratochwill TR, Hitchcock J, Horner RH, Levin JR, Odom SL, Rindskopf DM, et al. Better Evaluation. 2010. URL: https://www.betterevaluation.org/en/resources/guide/single-case_design_technical_documentation [accessed 2019-12-17]
    60. Lane JD, Gast DL. Visual analysis in single case experimental design studies: brief review and guidelines. Neuropsychol Rehabil. 2014;24(3-4):445-463. [CrossRef] [Medline]
    61. Manolov R. 2014. URL: https://manolov.shinyapps.io/Overlap/ [accessed 2020-01-10]
    62. Rizvi SL, Nock MK. Single-case experimental designs for the evaluation of treatments for self-injurious and suicidal behaviors. Suicide Life Threat Behav. Oct 2008;38(5):498-510. [CrossRef] [Medline]
    63. Lewis PA, Stevens JG. Nonlinear modeling of time series using Multivariate Adaptive Regression Splines (MARS). J Am Stat Assoc. 1991;86(416):864-877. [CrossRef]
    64. Hasselman F. GitHub. 2015. URL: https://github.com/FredHasselman/casnet [accessed 2020-01-10]
    65. RStudio Team. RStudio: Integrated Development Environment for R. Boston, MA. RStudio URL: http://www.rstudio.com/ [accessed 2020-01-10]
    66. Parker RI, Vannest K. An improved effect size for single-case research: nonoverlap of all pairs. Behav Ther. Dec 2009;40(4):357-367. [CrossRef] [Medline]
    67. Hedges LV, Pustejovsky JE, Shadish WR. A standardized mean difference effect size for single case designs. Res Synth Methods. Sep 2012;3(3):224-239. [CrossRef] [Medline]
    68. Pustejovsky JE, Hedges LV, Shadish WR. Design-comparable effect sizes in multiple baseline designs. J Educ Behav Stat. 2014;39(5):368-393. [CrossRef]
    69. Shadish WR, Hedges LV, Pustejovsky JE. Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: a primer and applications. J Sch Psychol. Apr 2014;52(2):123-147. [CrossRef] [Medline]
    70. Hedges LV, Pustejovsky JE, Shadish WR. A standardized mean difference effect size for multiple baseline designs across individuals. Res Synth Methods. Dec 2013;4(4):324-341. [CrossRef] [Medline]
    71. Pustejovsky JE. scdhlm: A web-based calculator for between-case standardized mean differences (Version 0.3.1). 2016. URL: https://jepusto.shinyapps.io/scdhlm [accessed 2020-01-10]
    72. Valentine JC, Tanner‐Smith EE, Pustejovsky JE, Lau TS. Between‐case standardized mean difference effect sizes for single‐case designs: a primer and tutorial using the scdhlm web application. Campbell Syst Rev. 2016;12(1):1-31. [FREE Full text] [CrossRef]
    73. Cohen J. A power primer. Psychol Bull. Jul 1992;112(1):155-159. [CrossRef] [Medline]
    74. Dong Y, Peng CJ. Principled missing data methods for researchers. Springerplus. Dec 2013;2(1):222. [FREE Full text] [CrossRef] [Medline]
    75. Cushing CC, Walters RW, Hoffman L. Aggregated N-of-1 randomized controlled trials: modern data analytics applied to a clinically valid method of intervention effectiveness. J Pediatr Psychol. Mar 2014;39(2):138-150. [CrossRef] [Medline]
    76. Hayes-Skelton SA, Roemer L, Orsillo SM, Borkovec TD. A contemporary view of applied relaxation for generalized anxiety disorder. Cogn Behav Ther. 2013;42(4):292-302. [FREE Full text] [CrossRef] [Medline]
    77. Jones EJ, Gliga T, Bedford R, Charman T, Johnson MH. Developmental pathways to autism: a review of prospective studies of infants at risk. Neurosci Biobehav Rev. Feb 2014;39:1-33. [FREE Full text] [CrossRef] [Medline]
    78. Visser JC, Rommelse NN, Greven CU, Buitelaar JK. Autism spectrum disorder and attention-deficit/hyperactivity disorder in early childhood: a review of unique and shared characteristics and developmental antecedents. Neurosci Biobehav Rev. Jun 2016;65:229-263. [CrossRef] [Medline]
    79. Sokolova E, Oerlemans AM, Rommelse NN, Groot P, Hartman CA, Glennon JC, et al. A causal and mediation analysis of the comorbidity between attention deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD). J Autism Dev Disord. Jun 2017;47(6):1595-1604. [FREE Full text] [CrossRef] [Medline]
    80. Tate RL, Perdices M, Rosenkoetter U, Wakim D, Godbee K, Togher L, et al. Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: the 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychol Rehabil. 2013;23(5):619-638. [CrossRef] [Medline]
    81. Dallery J, Raiff BR. Optimizing behavioral health interventions with single-case designs: from development to dissemination. Transl Behav Med. Sep 2014;4(3):290-303. [FREE Full text] [CrossRef] [Medline]
    82. Biglan A, Ary D, Wagenaar AC. The value of interrupted time-series experiments for community intervention research. Prev Sci. Mar 2000;1(1):31-49. [FREE Full text] [CrossRef] [Medline]
    83. Deaton A, Cartwright N. Understanding and misunderstanding randomized controlled trials. Soc Sci Med. Aug 2018;210:2-21. [FREE Full text] [CrossRef] [Medline]

    ‎
    ADHD: attention-deficit/hyperactivity disorder
    ASD: autism spectrum disorder
    BC-SMD: between-case standardized mean difference
    CBT: cognitive behavioral therapy
    MAD: median absolute deviation
    NAP: nonoverlap of all pairs
    PEM: percentage of data points exceeding the median
    RoBiNT: Risk of Bias in N-of-1 Trials
    SCED: single-case experimental design
    STAI: State-Trait Anxiety Inventory

    Edited by S Gipson; submitted 30.08.19; peer-reviewed by B Raiff, E Toki, K Mathiasen; comments to author 02.10.19; revised version received 21.11.19; accepted 16.12.19; published 24.03.20.

    Copyright

    ©Rineke Bossenbroek, Aniek Wols, Joanneke Weerdmeester, Anna Lichtwarck-Aschoff, Isabela Granic, Marieke M J W van Rooij. Originally published in JMIR Mental Health (http://mental.jmir.org), 24.03.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.