Potential Reduction of Symptoms With the Use of Persuasive Systems Design Features in Internet-Based Cognitive Behavioral Therapy Programs for Children and Adolescents With Anxiety: A Realist Synthesis

Introduction

Background

Anxiety is one of the most common and early emerging mental health concerns for children and adolescents [1], 20% of whom will experience an anxiety disorder in their lifetime [1]. Often presenting with a chronic and recurring course that extends into adulthood [2], anxiety disorders are associated with considerable developmental, psychosocial, and psychopathological impairments [1,3]. The effectiveness of cognitive behavioral therapy (CBT), an adaptive, skills-based psychotherapeutic approach, has been documented in numerous randomized controlled trials and is recommended as the first line treatment for children and adolescents with mild-to-moderate anxiety symptom severity [4-7]. Internet-based CBT (iCBT) aims to increase access and availability of this beneficial first line treatment [8,9] as the delivery of CBT content no longer hinges on face-to-face appointments with specialized therapists. Recent systematic reviews and meta-analyses have found iCBT to be comparably effective at reducing anxiety symptoms in children and adolescents relative to face-to-face CBT [8,10-12], and more effective in reducing symptoms than waitlist conditions [8,11,13-16]. Overall, these findings indicate that iCBT is an effective treatment option that can increase access to care.

iCBT is a complex intervention [17] and is not merely the upload of therapeutic material onto a Web page. iCBT programs incorporate 3 major components: (1) structured and standardized therapeutic content (ie, CBT), (2) technological features (ie, multimedia and email) used to deliver the content, and (3) interactions between the user and the program to engage users in iCBT content and features. These components are characteristics of a persuasive system—an intervention designed to change user’s attitudes and behavior toward their desired health goal [18-20].

Motivation and Objective

To date, considerable variety exists in terms of how the 3 iCBT components have been incorporated into iCBT program design. Only 2 studies of iCBT effectiveness have attempted to identify or explain what program components can be used to optimize the therapeutic gains of users and for what reasons. Calear et al [21] explored whether the expertise of the adjunct support person had an effect on intervention outcomes (teacher-only support vs teacher plus mental health education officer support [21]), and Spence et al [22] tested for a difference in adolescents’ anxiety reductions because of the specificity of program content (a program with social anxiety disorder–specific content compared with generic anxiety disorder content [22]), but neither study reported a significant difference in outcomes. Therefore, an essential question that remains for the field is, “What components of iCBT work, for whom, and why?” Using realist synthesis methodology, we used a persuasive systems perspective to examine the following:

1. What design and delivery components are described in studies of iCBT programs for children and adolescents with anxiety?
2. What are the components reported in studies that appear to explain the change in anxiety symptoms after completing iCBT?

Methods

Study Design

The realist synthesis provided us with a mixed methods approach to generate proposed explanations (hypotheses) of how and why iCBT programs work despite variations in its design and delivery [23-25]. The synthesis was conducted using the steps recommended by Pawson and Tilley [25,26] and reported in accordance with the Realist and Meta-Narrative Evidence Synthesis: Evolving Standards II [27]. We set out to examine the technological features of iCBT (mechanisms), embedded within delivery and conditions or settings for use (program context), that produced changes in anxiety symptoms for children and adolescents (outcomes). Pawson and Tilley refer to these relationships as context-mechanism-outcome configurations. Thus, the overall purpose of the synthesis was to produce context-mechanism-outcome configurations that hypothesized when and how iCBT programs might be effective in reducing anxiety symptoms among children and adolescents with anxiety.

Candidate Context-Mechanism-Outcome Configuration Development

We started the synthesis with the generation of a list of candidate context-mechanism-outcome configurations. We decided a priori to use an established, valid framework to guide the generation of candidate configurations. This also helped us maintain a consistent and streamlined approach to the synthesis (ie, extract and code data according to framework). We conducted an informal literature scan and held research team discussions to identify preexisting frameworks from the fields of psychology, pediatrics, human-computer interaction, and electronic health (eHealth). The persuasive systems design (PSD) model [28] was selected as the most appropriate framework to direct the candidate configurations and answer our research questions. It is a recent, well-studied model [29] that describes 28 technological design features that can be incorporated into a persuasive system to help the user meet his or her health goals (Multimedia Appendix 1 [30-43]). The model organizes the design features under 4 categories based on their main purpose: primary task support (assists the user in completing their target behavior), dialogue support (provides computer-human communication to guide user toward target behavior), system credibility support (increases user’s perceptions of a system’s credibility), and social support (leverages the interactions and influence of others).

We used the PSD model to identify PSD features (mechanisms) hypothesized to produce anxiety symptom changes (outcomes) in iCBT programs and recorded these as mechanism-outcome dyads. We then considered the program design and delivery features (context) that might support the operation of the mechanism-outcome dyads and combined them in unified but distinct configurations. The result was 8 candidate context-mechanism-outcome configurations (Multimedia Appendix 2), or initial hypotheses, that formed the basis of our analysis. We refined these configurations during the realist synthesis so that they reflected the iCBT literature. Following analysis, we considered our configurations to be fully developed hypotheses ready for future testing.

Literature Search

We required diverse literature to inform this synthesis. We sought to include primary or secondary studies of iCBT interventions, conference proceedings, websites, program evaluations, and government or technical reports. We used 3 search strategies to identify this literature: (1) a systematic, comprehensive search of 8 electronic databases from disciplines relevant to the topic (ie, medicine and psychology)—Medline, Embase, Education Resources Information Center, PsycINFO, Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, ProQuest Dissertations & Theses Global, and PubMed for the period from 1990 to 2017, conducted by an information specialist; (2) a manual search using an internet search engine (Google) and gray literature repositories (eg, Association for Computing Machinery Digital Library, Open Gray, Institute of Electrical and Electronics Engineers Digital Library, and Canadian Agency for Drugs and Technologies in Health); and (3) a hand search of medical informatics journals (Journal of Medical Internet Research, Internet Interventions, Journal of Cybertherapy and Rehabilitation, and Journal of Telemedicine and Telecare) and the reference lists of included documents and reviews (eg, systematic reviews). Medical Subject Headings terms and text words for the search were based on mental health condition (ie, anxiety and phobias), intervention modality (ie, internet-based and mobile apps), intervention type (ie, prevention and treatment), and therapeutic approach (ie, CBT; Multimedia Appendix 3). We applied the search strategies in an initial search (conducted up until February 2015) and then in 2 updated searches (conducted in November 2017 and in June 2019) to ensure the realist synthesis was current and inclusive.

Document and Internet-Based Cognitive Behavioral Therapy Program Selection

We were interested in including documents relevant to iCBT programs that were designed for use by children or adolescents aged ≤19 years diagnosed with an anxiety disorder(s) or with anxiety symptoms associated with a disorder as classified according to the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders [44]. Documents needed to include information about an iCBT program designed for treating child or adolescent anxiety and be available in English. Documents that detailed information for a transdiagnostic program (eg, treating an anxiety disorder plus depression) were eligible for inclusion but documents of programs designed solely for parent use were not. All eligible documents were grouped according to the iCBT program it represented (ie, program name). We included in the realist synthesis documents of programs that detailed its delivery context (ie, the conditions for program use), PSD mechanisms (ie, information on the technological features for how the program was proposed to work), and impact on anxiety outcomes after program delivery (ie, at least one published study of postintervention effects). These details could be described within 1 document, or across multiple documents, but needed to be available so that at least one context-mechanism-outcome configuration could potentially be generated for each iCBT program.

During the document selection progress, 2 reviewers (authors ADR and LW) independently applied the inclusion criteria using a 2-stage approach. In stage 1, titles and abstracts of documents were screened for potential eligibility (yes, no, or unsure). The reviewers randomly selected 100 documents to assess inter-rater agreement and found substantial agreement (Cohen kappa=0.74) [45]. In stage 2, the full text of yes or unsure documents were reviewed by the 2 reviewers for inclusion or exclusion in the synthesis. In both stages, they resolved discrepancies through discussion (with author ASN) until consensus was reached.

Literature Appraisal

Quality appraisal of included documents involved assessing relevance to the synthesis objectives and, in the case of research studies, assessing methodological rigor. A total of 2 reviewers (authors ADR and LW) conducted the quality appraisal. Relevance was assessed by reviewing a document’s level of contribution, the extent to which information was provided on (1) theory and/or the context and sequences for iCBT delivery (context), (2) PSD features, required interactions by the deliverer/user and the program, and/or other proposed program mechanisms (mechanism), and (3) the impact of iCBT on anxiety symptoms outcomes and explanations for the findings (outcome). The level of contribution was rated low if little or no information was provided, medium if some information was provided, and high if information was well-described, relative to other documents for other programs.

To understand the quality of the research studies that provided outcome data for the synthesis, the methodological rigor of studies was assessed using the Mixed Methods Appraisal Tool (MMAT) [46,47]. The MMAT is a reliable, efficient, and validated tool that provides different sections for assessing studies of qualitative, randomized, nonrandomized, descriptive, and mixed methods design [46-48]. Multiple documents using the same, full dataset (ie, thesis plus published paper of the thesis) received the same MMAT score but was only counted once. MMAT scores could range from 0% to 100%, with a greater score signifying more rigorous criteria were met.

Data Extraction and Coding

To identify context-mechanism-outcome configurations, we extracted and coded iCBT program data using a data matrix with 6 major domains: (1) document characteristics (eg, study design), (2) participant characteristics (eg, demographics) and study procedures (ie, eligibility criteria), (3) context of iCBT delivery including a program’s targeted level of prevention according to the Institute of Medicine (IOM) model [49] and adjunct support details (human-derived technological or therapeutic communication complementary to program delivery), (4) program theory and principles behind iCBT program design, (5) program components or proposed mechanisms (ie, CBT content and PSD features and interactions between the user and program), and (6) pre- to postintervention change in anxiety symptoms. For outcome data, not all studies reported within-group analyses; therefore, absolute changes in anxiety symptoms among children or adolescents who received iCBT were recorded. If original authors referred to previous publications of an iCBT program, we included the document and extracted relevant data. We also extracted partial or full context-mechanism-outcome configurations, if provided by the original authors.

We used the PSD model [20] and a customized codebook to guide the coding process (Multimedia Appendix 1). Similar to previous studies [29,50], PSD features of iCBT programs were coded: (1) when a program feature was executed by technology (eg, video demonstration of an adolescent performing deep breathing) rather than by human action (eg, a parent demonstrating deep breathing in person) and (2) when the feature was a part of the iCBT program, not supporting research study materials, such as an informational website. We coded therapeutic content according to the 5 main CBT components found in the American Academy of Child and Adolescent Psychiatry (AACAP) practice parameter for anxiety disorders [6]: psychoeducation, somatic management skills, cognitive restructuring, exposure methods, and relapse prevention. We also extracted other therapeutic content, such as behavioral activation details and interpersonal therapy techniques [51,52]. We contacted corresponding authors associated with each iCBT program to support accurate and complete extraction and coding of the data. Overall, 80% (12/15) of original authors associated with the included iCBT programs responded to the request for more information. The interpretation of data extraction and coding between 2 reviewers (authors ADR and LW) was checked with a random sample of 10 documents and achieved consensus before the remaining documents were coded by a single reviewer (author ADR).

Analysis and Synthesis Process

Analysis was conducted at the program level [25], which meant that multiple documents or studies relating to each unique iCBT program were grouped together for analysis of the iCBT program as a whole. Programs were grouped according to their level of prevention using the IOM model [49]—whether they were a universal prevention, indicated prevention, or treatment program (program type). The IOM model recognizes that the target population (ie, children or adolescents with baseline anxiety symptoms and an associated level of risk for a disorder) differs according to program type and so does the conceptual focus of the intervention, and together, these differences may influence the fundamental design and delivery of a program (ie, context and clinical techniques) and the expected degree of change users may experience (ie, outcomes) [53].

We analyzed the candidate context-mechanism-outcome configurations in 4 stages using meta-ethnographic [54,55] and realist [24-26] techniques (Figure 1). During this process, information from included documents was synthesized to refine each configuration. In stage 1, we recorded the individual components (ie, contexts, mechanisms, and outcomes) and relationships between components (eg, mechanism-outcome dyads) reported in the documents for each iCBT program. We compared the information for each iCBT program with the candidate configurations (initial hypotheses) and documented whether a candidate configuration was supported, unsupported, modified, or newly generated based on the evidence. In stage 2, reciprocal translation analysis was used to determine if common contexts and mechanisms were being described across programs. The candidate configurations were ranked from the most to least supported, based on the number of iCBT programs supporting each configuration. A configuration was required to be supported by at least two iCBT programs to proceed with the next stage of analysis. We considered configurations with the highest rankings to depict the larger patterns or trends (demi-regularities) of iCBT program components. These candidate configurations became our final hypotheses for how iCBT programs were hypothesized to work.

‎

In stage 3, we developed descriptions of how the iCBT program components were linked in our configurations. The descriptions focused on the proposed function (role) of key PSD features in explaining how iCBT programs might reduce anxiety for children and adolescents. To do this, we nested the configurations within our broader understanding of the theoretical underpinnings of the PSD model and CBT, along with original authors’ descriptions of the design or delivery of program features. This process allowed us to explore not only what iCBT program components might be working together but also why they might be doing so. We maintained a level of abstraction that allowed us to express the larger similarities across multiple programs while acknowledging the details that made each configuration unique (using lines-of-argument synthesis). This meant that we did not delve deeper into specific details of contexts, mechanisms, or outcomes. For example, we identified whether adjunct support was provided in programs rather than the specific amount of support provided, and we identified the direction of treatment effect rather than specific effect sizes. This approach was not only necessary because of the data available to us but also ensured that our configurations remained usable and applicable across the programs. During this stage, we also incorporated into the descriptions other factors that could help explain our understanding of the configurations (eg, multiple PSD features working together and differences in user characteristics). In stage 4, we identified the quantity (ie, number of programs) and quality (ie, relevance and methodological rigor) of support associated with each configuration. Research team meetings were used to discuss and improve the final detailed descriptions of our proposed configurations.

Results

Included Documents

Figure 2 is a flow diagram outlining the results of the 2-stage literature search and selection process. A total of 63 documents (30 from the initial search, 15 from the updated search conducted in November 2017, and 18 from the updated search conducted in June 2019) describing 15 iCBT programs were eligible and included in the review. The included documents were published studies (n=29), theses (n=5), registered protocols for trials (n=15), study or program websites (n=9), study flyers (n=2), and conference abstracts or posters (n=3).

‎

Level of Contribution and Methodological Quality

Details of the quality appraisal are provided in Multimedia Appendix 4 [21,22,30-39,41-43,56-91,92-102]. Across documents, the most details were provided for iCBT program contexts and outcomes. Program mechanisms were not so well described in terms of what the mechanisms were (ie, technological and therapeutic features) and how they were proposed to work. A total of 8 documents were rated as having a high level of contribution to understanding contexts, mechanisms, and outcomes [22,31,40,56-60]. These documents were for 6 iCBT programs that provided the greatest contributions to the context-mechanism-outcome configurations we developed: BRAVE-Online, internet-delivered CBT for children with anxiety disorders, internet-delivered CBT for children with specific phobia, internet cognitive behavioral skills–based program, STAY COOL system for test anxiety, and the e-couch Anxiety and Worry program.

MMAT scores were calculated for 35 research studies: 20 randomized controlled trials, 5 nonrandomized studies, 7 quantitative descriptive studies, 1 qualitative study, and 2 mixed methods studies. A total of 22 documents met all 4 MMAT criteria (100%), 7 documents met 3 criteria (75%), 5 documents met 2 criteria (50%), and 1 document met 1 criterion (25%). Lower MMAT ratings were a result of factors such as incomplete outcome data or unacceptable response rates, high withdrawal rates, or no mention of whether groups were comparable.

Overview of the Contexts, Mechanisms, and Outcomes of Internet-Based Cognitive Behavioral Therapy Programs for Children and Adolescents With Anxiety

Contexts: User and Program Delivery Characteristics

Table 1 presents an overview of the user and program delivery characteristics of iCBT programs, organized according to program type. All programs were computer-based and included some form of adjunct support or program facilitation. Most programs (10/15 programs) were designed to solely target users with reported anxiety symptoms. There were similarities in the delivery setting, workflow, and adjunct support of iCBT programs of the same program type. Treatment programs were most often accessed at home, included 7 or more modules, and incorporated weekly Web-based therapist interaction and parent-dedicated modules. Indicated prevention programs demonstrated more variety in their use setting (eg, home, school, or clinic) and the adjunct support provided (eg, not all programs required parent participation), and the workflow tended to involve fewer modules (ie, typically 6 modules or less). Universal prevention programs were delivered with teacher facilitation in a classroom setting and incorporated the least number of modules relative to other program types. iCBT programs were based on relevant theoretical, anxiety, or CBT literature or published treatment recommendations [30,31,34,90,95], established clinic-based programs, manuals or workbooks [37,39,42,57,94], a school syllabus [35], or were adaptations of a developed iCBT program designed to target a different population or mental health condition [32,76,83,88].

Mechanisms: Therapeutic Content and Persuasive Systems Design Features

Figure 3 provides an overview of the proportion of iCBT programs that incorporated specific CBT content and PSD features, organized according to program type. All programs described themselves as CBT-based and contained AACAP recommended CBT components, although considerable variability in the type and quantity of components was found based on program type and age group of target users. Many programs also integrated techniques with an interpersonal focus, such as assertiveness training and problem-solving skills, to reduce environmental stressors and enhance social support [104]. Psychoeducation and somatic skills training were found in all iCBT programs. Cognitive restructuring was reported in more than half of the treatment programs and in nearly all the indicated and universal prevention programs. Relapse prevention was incorporated in a minority of prevention-based programs. Exposure methods were not delivered to users of universal prevention preventions.

Treatment programs incorporated the most PSD features, followed by indicated prevention and then universal prevention programs. Out of the 4 PSD support categories, features from the primary task support and dialogue support categories were most widely used. In terms of primary task supports, iCBT programs of all IOM program types incorporated reduction and tunneling to regulate the logical and incremental presentation of module content to users, mimicking the progressive delivery format of face-to-face CBT. Self-monitoring of users’ iCBT progress was also a primary task support feature common to all programs. Social role, a dialogue support feature, created a virtual presence of others in the program through Web- or email-based messaging between the user and therapist or recurring graphics or videos of real or animated peers. System credibility support features, trustworthiness, and surface credibility, although not explicitly reported, were inherent in all iCBT programs, as they were designed and delivered (tested) within a research study (ie, use of confidentiality and consent processes and declared academic affiliations). Authority features were associated with program content that was presented by a reliable source (ie, therapist) and was, therefore, only incorporated in treatment and universal prevention programs with adjunct therapist involvement. Social learning was the only social support feature included in iCBT programs, but not all indicated prevention programs utilized it. iCBT from different program types varied to the greatest extent on their use of personalization, tailoring, reminders, liking, and similarity features. Multimedia Appendix 1 provides additional examples of how PSD features were reported in the documents included in this synthesis.

‎

Outcomes: Changes in Anxiety Post Intervention

Across program types, there was an overall trend for reduced anxiety symptoms among children and adolescents who received iCBT. An overview of the outcomes is provided in Multimedia Appendix 5. Among treatment programs, anxiety diagnoses, clinical severity, and parent- and user-reported symptoms were reduced post intervention in 98.5% of studies. Among indicated prevention programs, anxiety diagnoses, parent- and user-reported symptoms were reduced post intervention in 100% of studies. Among universal prevention programs, user-reported anxiety symptoms were reduced post intervention in 83.3% of studies.

Key Relationships Between Internet-Based Cognitive Behavioral Therapy Program Contexts, Persuasive Systems Design Mechanisms, and Outcomes

We found that reductions in anxiety outcomes were reported across iCBT programs with many shared mechanisms and delivery contexts. Self-monitoring, simulation, social role, similarity, social learning, and rehearsal were common PSD features across all program types; however, mechanisms for customizing program content (ie, personalization and tailoring) distinguished treatment, indicated prevention, and universal prevention programs from one another. The key aspect of iCBT context that supported the mechanism-outcome interactions was adjunct support. The adjunct support person (eg, therapist, parent, and teacher), their expertise, and the intensity and frequency of their communication (eg, weekly personalized feedback and technical troubleshooting as needed) was associated with the program type, and, therefore, also the characteristics of users, such as age and symptom severity. In this way, treatment programs received the greatest amount of adjunct support relative to indicated and universal prevention programs.

Context-Mechanism-Outcome Configurations

The refined set of context-mechanism-outcome configurations is summarized in Table 2, according to program type. Configurations, organized by PSD mechanism, are based on included information from documents that ranged in level of contribution (low to high).

Treatment Programs, Configurations 1 to 4

Configuration 1: Self-Monitoring

Treatment programs for children with an anxiety disorder, delivered with adjunct therapist, parent, or professional support and include self-monitoring, may produce postintervention reductions in user’s anxiety (diagnoses, clinical severity, self-reported, and parent-reported symptoms). Self-monitoring was part of the workflow for each module of the BRAVE-Online program and included regular tracking of symptoms and interactive activities and end-of-module quizzes to “facilitate attention and comprehension of material” [36]. Chilled Out program participants were presented with a weekly progress chart based on their reports of anxiety interference in their daily lives [83]. During program tasks, self-monitoring was employed using automated pop-ups stating the accuracy of users’ entries (ie, corrective comments) to ensure understanding of important concepts [36]. The adjunct support therapist encouraged users to self-monitor and record details of their in vivo (real world or offline) practice activities [39,41], including changes in anxiety following exposure exercises [37].

Configuration 2: Simulation, Social Role, Similarity, and Social Learning

Treatment programs for children with an anxiety disorder, delivered with adjunct therapist, parent, or professional support and include simulation with a social role, similarity, and social learning features, may produce postintervention reductions in user’s anxiety (diagnoses, clinical severity, self-reported, and parent-reported symptoms). These features were evident in videos or animations of peers, cartoon, and real-life characters to illustrate the experience of different emotions and the application of therapeutic skills, such as goal setting, developing fear hierarchies, and completing exposure activities [36,37,41,83]. Age-appropriate characters “provided ‘models’ for the use of coping strategies to overcome anxiety problems” [36]. Role models were designed to be appealing and relatable to users and their anxiety-related challenges; they represented someone with whom “the child can identify [with] and will be more likely to learn from” [58] (similarity). In another treatment program [37], the development of exposure-based film scenes used for fear extinction [105,106] of dental procedures were based on the principles of observational learning and the development of self-efficacy [107]. Email communication between the user and adjunct therapist (social role) could mimic or complement simulations, as therapists provided additional anxiety management instructions, tutorials or helped users problem-solve and plan exposure activities related to their specific situation or fears [76].

Configuration 3: Rehearsal

Treatment programs for children with an anxiety disorder, delivered with adjunct therapist, parent, or professional support and include rehearsal features, may produce postintervention reductions in user’s anxiety (diagnoses, clinical severity, self-reported, and parent-reported symptoms). Rehearsal was incorporated in brief, interactive tasks to be completed during the module (eg, drag this sentence to the correct term and drop it there) [41], quizzes for comprehension (eg, recap or summary quizzes) [58], or more in-depth, application-based worksheets at the end of the module [37,90]. For example, in BRAVE-Online, “Participants consolidate[d] learning of these [anxiety management] strategies through completion of weekly [Web-based] homework tasks, known as ‘extreme challenges’” [36]. Postmodule rehearsal activities recommended users to apply their target skill in real life anxiety-provoking situations outside of the program (ie, exposure exercises) [58]. An adjunct therapist was available to help structure and monitor some of these rehearsal activities. For example, in preparation for exposure activities, a supportive telephone call or message from the therapist assisted the user in developing a suitable exposure hierarchy [36,88].

Configuration 4: Personalization, Social Role, Trustworthiness, Expertise, and Authority

Treatment programs for children with an anxiety disorder, delivered with adjunct therapist, parent, or professional support and include personalization, a social role, trustworthiness, expertise, and authority, may produce postintervention reductions in user’s anxiety (diagnoses, clinical severity, self-reported, and parent-reported symptoms). Personalization provided a sense of program relatedness or knowing of the user through automated or manual features based on demographic details or program activity of the user. For example, the user’s name and that of his or her adjunct therapist could be populated throughout the modules [42]. Personalized pop-ups with immediate and specific feedback (eg, explanations for correct and incorrect answers [36]) on quizzes and tasks were also provided. In addition, the adjunct therapist (social role) monitored users’ responses to tasks and homework assignments and provided personalized written feedback by email. Personalized feedback was used to “reinforce effort and success and provide corrective information if required” [36], to “answer questions and clarify treatment content, increase motivation and to help solve problems” [41], or to “[ensure] adolescents’ understanding of the program elements” [83]. As the therapist could access user-specific information stored within the program, a response could be crafted with objective and supportive input through the therapist’s professional lens (authority); therefore, trustworthiness and expertise were features considered to be inherent to this personalized feedback process.

Indicated Prevention Programs, Configurations 5 to 8

Configuration 5: Self-Monitoring

Indicated prevention programs for children or adolescents with mild-to-severe anxiety symptoms, delivered with adjunct therapist, parent, or researcher support and include self-monitoring, may produce postintervention reductions in user’s anxiety (diagnoses, self-reported, and parent-reported symptoms). Self-monitoring was incorporated in the Feeling Better program using standardized symptom assessments at the beginning of modules as a way “to monitor symptom change” [43] over the course of the program. For some programs, symptom tracking was an essential part of the ongoing risk management of users [32,43]. The program or the adjunct support therapist would respond (automatically or manually) to safety concerns that arose from these assessments by providing additional mental health coping resources. In addition to mood, the program tracked the user’s progress toward goal achievement via homework completion. Module quizzes in some programs [31,57] were a means for users to review his or her understanding of new program concepts or skills [57].

Configuration 6: Simulation, Social Role, Similarity, and Social Learning

Indicated prevention programs for children or adolescents with mild-to-severe anxiety symptoms, delivered with adjunct therapist, parent, or researcher support and include simulation with a social role, similarity, and social learning, may produce postintervention reductions in user’s anxiety (diagnoses, self-reported, and parent-reported symptoms). Simulation was incorporated in examples or demonstration videos of individuals (social role) “illustrat[ing] certain concepts in the program” [57], providing suggested solutions, or working through their problems (social learning) [43]. The examples and activities provided in the Feeling Better program were specific to target users and their reported stressors (similarity) and were employed to “encourage practice and enhance learning of material” [43].

Configuration 7: Rehearsal

Indicated prevention programs for children or adolescents with mild-to-severe anxiety symptoms, delivered with adjunct therapist, parent, or researcher support and include rehearsal, may produce postintervention reductions in user’s anxiety (diagnoses, self-reported, and parent-reported symptoms). The STAY COOL program described including evidence-based practice activities (rehearsal) for reducing physical and cognitive test anxiety symptoms and pairing these coping activities with desensitizing exposure tasks to improve the program’s effectiveness [31]. In the same program, postmodule quizzes presented users with “a less-threatening, relatively low stakes exposure by testing them on recently obtained information in an untimed scenario” [31]. In the internet-based cognitive behavioral skills program, Talk Time was used to prompt the mother (adjunct parental support) and child to discuss a therapy topic or work together on a task [57]. In addition, exposure hierarchies were used to guide users’ practice (rehearsal) outside of the program as well. Adjunct therapists could provide rehearsal support (eg, encouragement and suggestions), if necessary, through their communications with the user.

Configuration 8: Tailoring

Indicated prevention programs for children or adolescents with mild-to-severe anxiety symptoms, delivered with adjunct therapist, parent, or researcher support and include tailoring, may produce postintervention reductions in user’s anxiety (diagnoses, self-reported, and parent-reported symptoms). iCBT content was tailored according to the user’s symptom profile. In the Feeling Better program, “A standardized assessment of symptoms of distress… [was] built into the start and end of core program modules to monitor symptom change and to help the user choose customized streams of program content specific to their emotional distress [such as anxiety, depression, or stress]” [43]. Another program had gender-specific versions (male and female), so that therapeutic examples matched the sex of the user [43]. For the individually tailored iCBT program for adolescents, the adjunct therapist used results from a baseline diagnostic interview to select module content (ie, psychoeducation and case examples) that corresponded to the user’s primary anxiety concern [32]. According to Silfvernagel (2017), a tailored iCBT program was “designed to identify a participant’s unique symptom profile and to provide information and skills that are likely to be helpful based on said profile” [96], aiming to improve the usefulness of the intervention.

Universal Prevention Programs, Configurations 9 to 11

Configuration 9: Self-Monitoring

Universal prevention programs for adolescents with minimal or no symptoms, delivered with teacher facilitation and include self-monitoring, may produce postintervention reductions in user’s self-reported anxiety symptoms. The MoodGYM program provided anxiety and depression quizzes (self-monitoring) before and after each module. Adolescents’ answers to quizzes and other program tasks were saved in a personal Web-based workbook that could be accessed by them at any time [34], serving as a benchmark for which they could compare changes over the course of iCBT. Electronic questionnaires were administered to adolescent users of Thiswayup Schools who also received notification if their scores were above average [102]. In the case of all 3 universal prevention programs, a teacher was present for iCBT administration and could provide referral advice if an adolescent’s symptoms required professional follow-up [108].

Configuration 10: Simulation, Social Role, Similarity, and Social Learning

Universal prevention programs for adolescents with minimal or no symptoms, delivered with teacher facilitation, and include simulation with a social role, similarity, and social learning, may produce postintervention reductions in user’s self-reported anxiety symptoms. Cartoon vignettes (similarity and social role) provided examples of anxiety management behaviors and responses as a regular part of the modules (simulation and social learning). For example, at the beginning of the MoodGYM program, adolescent users were “introduced to six distinct characters that form the basis of examples and discussion. Each character has a specific way of dealing with stressful situations, which [were] explored in the program.” [34]. Similarly, Thiswayup Schools used a storyline of cartoon teenagers with anxiety or depression to demonstrate ways to solve real life problems [35].

Configuration 11: Rehearsal

Universal prevention programs for adolescents with minimal or no symptoms, delivered with teacher facilitation, that include rehearsal may produce postintervention reductions in user’s self-reported anxiety symptoms. The e-couch Anxiety and Worry program included rehearsal exercises to help users understand themselves and others better [99]. The MoodGYM program also provided opportunities for users to apply therapeutic strategies to their own situation. Both quizzes and homework exercises were incorporated for users to practice their skills. User’s answers were recorded in their Web-based workbook and could be accessed at any time [34]. Rehearsal activities appeared to focus on cognitive restructuring, problem-solving, and interpersonal skills.

Discussion

Principal Findings

Our study systematically documented important similarities and differences in the design and delivery of iCBT components across 15 existing programs, which to our knowledge, is the first study of its kind for children or adolescents with anxiety. Anxiety reductions were reported in more than 98% of studies we reviewed. Our use of realist synthesis methods enabled the development of 11 context-mechanism-outcomes configurations that hypothesized the PSD features (technology-based mechanisms) that might contribute to the observed reductions in anxiety symptoms (outcomes), as they relate to key user and delivery features (context). Our results point to the need for increased emphasis on PSD in the development, evaluation, and reporting of iCBT programs for children and adolescents with anxiety concerns and further research designed to establish their relationship with improved anxiety symptomatology.

Design and Delivery Components of Internet-Based Cognitive Behavioral Therapy for Children and Adolescents With Anxiety

The 11 configurations included PSD features from all 4 support categories. However, some category features were more often linked to iCBT program effects than others. Our findings highlight the central role of primary task supports in iCBT interventions for children and adolescents with anxiety, followed by dialogue support and system credibility support categories. Only 1 social support feature was supported by our analysis. These findings are in line with others [29,109] who also found primary task supports to be the most frequently reported persuasive features in technology-based health interventions. As primary task support features are considered to “aid users in completing their tasks and tracking and achieving their goals” [110], they have a similar aim to the goal-directed nature of iCBT programs. Dialogue support features keep “the user active and motivated in using the system” [111], so the user has more time and opportunities to complete their intended behavior(s) in the program. Both primary task support and dialogue support features have been linked to intervention effectiveness in previous studies in other fields [112-114].

Toward Explanatory Persuasive Systems Design—Informed Models of Internet-Based Cognitive Behavioral Therapy Effects With Children and Adolescents With Anxiety

All iCBT programs in this study contained multiple PSD features. Although detailing the frequency of PSD features in iCBT provides some insight into what a user does within a program (activity), this information does not describe the important patterns or combinations of PSD features or explain why a program may or may not be effective. However, our findings suggest that (1) no PSD feature is applied in isolation and will likely not work as such (ie, some synergy is expected and essential) and (2) different PSD features have different functions, so we cannot assume that more features are better. We identified features from within and across different PSD support categories that were employed simultaneously, suggesting some synergistic or additive effect in their grouping (eg, simulation—a primary task support + social role—a dialogue support + similarity—a dialogue support + social learning—a social support). There have been attempts to examine the quantity and combination of PSD features in relation to the effectiveness of other health-based programs [29,109,111,115-117], but the literature is inconsistent and inconclusive. Wildeboer et al [111] indicated a positive relationship between effect sizes and the number of PSD features used in an intervention [111]. Additive or synergistic effects between multiple features, such as simulation and rehearsal, have been reported [117]. In contrast, other features together may negate or interfere with their persuasive potential [117], depending on the features and what persuasive support category they are from [111]. Future studies are needed to systematically assess the use and combination of multiple features as they relate to program effects to optimize the design of programs.

Overlap with the proposed context-mechanism-outcome configurations we generated and the literature on internet-based interventions indicate larger patterns for how these features operate. For example, others have hypothesized that self-monitoring might be used to increase user’s knowledge, self-awareness, and ability to monitor and manage their health [118,119]. Simulation allows users to cognitively or physically play out hypothetical situations [120], such as health-related decision making [121], to observe their effects before applying strategies to the real world [122]. In face-to-face CBT studies, rehearsal promotes skill acquisition [123], self-efficacy, and confidence with health management techniques [124,125]; rehearsal has been critical to the cognitive improvements found during treatment [126-128]. We propose that rehearsal may have a similar function in iCBT to that in face-to-face CBT, justifying its use in iCBT programs.

The consistent incorporation of specific key PSD features (rehearsal, self-monitoring, and simulation) in configurations across all program types indicates that these may be signature features that appeared to be particularly effective at producing the desired effects of iCBT regardless of the program type—perhaps because of the mental (psychological) activity and the interactions (effortful, suggestive, and engaging) between the user, the program content, and its features they incite. Our proposed key PSD features may produce symptom reductions in iCBT because they initiate higher-order cognitive processes, such as information recall, mental reflexivity, and future planning, much similar to the CBT content in these programs as well (ie, cognitive restructuring) [129], that may lead to longer lasting changes in learning and behavior. This observation supports a foundational conceptualization of persuasive systems as being a medium or tool for behavior change [19,130].

Differences in key PSD features may distinguish iCBT programs of one program type from another (ie, indicated prevention programs used tailoring whereas treatment programs used personalization). It has been recognized that user characteristics (eg, symptom severity and motivation), the focus of a program (eg, technological or therapeutic elements incorporated and the at risk behaviors targeted), and expected benefits (eg, degree of reduction in anxiety) differ depending on program type (eg, universal prevention, indicated prevention, and treatment) [53]. It may be that as the risk level and severity of symptoms of users increases from universal prevention (general population) to indicated prevention (low to high risk) to treatment programs (a diagnosed disorder), so does the program’s ability to adapt to user characteristics to improve its applicability and potential effectiveness (eg, providing relevant content based on user’s age and providing individual feedback on user’s practice activities). We believe maintaining the program type categorization is important in future testing of the PSD-based hypotheses as this categorization may help account for the distinct design and delivery components and the measures of effectiveness used (eg, primary outcomes, instruments, and significance level) across program types. Taking the unique contexts, mechanisms, and outcomes across program types into consideration will also help prepare the evidence base for implementation efforts of iCBT for anxious children and adolescents, for example, identifying important aspects of delivery setting, program support, or intervention features that may influence program effects [76,131,132].

The Context-Mechanism Relationship

Realist synthesis methods focus on uncovering both the mechanisms of a complex intervention and their relationship to context [17,133-135]. We observed the important effect that the delivery context had on the PSD mechanisms that were included in iCBT programs, further confirming the importance of examining iCBT programs of a similar program type together. For example, we found that the context of all universal prevention programs involved widespread delivery in schools, during regular class periods, to all students in attendance, by a teacher with no specialized mental health training. This aspect of context differed considerably from indicated prevention and treatment programs that had a primarily self-led delivery format (ie, users could log into the program from any location at any time) where minimal, but some, Web- or email-based interaction with an adjunct support person was provided. PSD features could be affected by program contexts in a way that determined their presence or absence and the quality or how they were delivered. An example of this is that personalized feedback was provided to users only if an adjunct therapist was available to craft and deliver the message—a feature provided to users of treatment and indicated prevention programs. Another example is, in terms of iCBT practice, at-home or clinic-based delivery of programs required users to complete Web-based homework (rehearsal), whereas with some school-based universal prevention programs, paper-based homework (non-Web-based) was assigned to users during class.

School-based universal prevention programs have aspects of context (ie, setting of program use) that are different from indicated prevention and treatment programs, making their design and delivery unique [136]. In this study, we did not include the setting (home, clinic, or school delivery) in the proposed configurations as an understanding of how this context relates to specific PSD features to affect users’ anxiety did not emerge in our analyses. However, indications of a relationship between use setting, adjunct support, and program type (eg, the self-led delivery format of treatment and indicated prevention programs) was evident, indicating that the consideration of the impact of setting in future studies is warranted.

Strengths and Limitations

We took a high-level perspective to develop hypotheses that may explain the effects of iCBT as a complex intervention. To our knowledge, this is the first study to systematically describe what and how PSD features may relate to symptom reductions in iCBT across programs for children and adolescents with anxiety. Although our findings may be similar to those of adult-based studies of internet-based interventions (eg, rehearsal [117] and self-monitoring [118]) or intuitive to researchers and developers in the iCBT field, no formal exploration of the effects of the PSD features on iCBT program outcomes for children and adolescents has previously been conducted. Our study acknowledged that there might be PSD features unique to programs designed for users within our age range (eg, social learning and peer demonstrations, ie, simulation, seemed especially important for children and adolescents using iCBT). Previous research suggests that the age and developmental stage of program users (eg, cognitive development: autonomous thinking and socioemotional: theory of mind) can affect the acceptability of an intervention [16,137,138] and intervention features (eg, tailoring, interactivity, and reinforcement) [139-141], indicating that there are unique iCBT design and delivery considerations to account for with children and adolescents that cannot be presupposed based on the adult literature [142,143].

This review has several strengths. We followed established and rigorous methods for conducting and reporting realist syntheses [24,26,27]. We included diverse, high-quality evidence from published and gray literature and used mixed methods for our analysis. Our approach to program evaluation was inward-looking in that it used data from iCBT programs and its users only. This allowed us to focus our analytic efforts to uncovering the within-iCBT relationships between design and delivery features (program contexts and mechanisms) that may produce the outcomes observed. With increasing emphasis being placed on the need for theory to guide internet-based intervention development [144-146], especially theories that consider intervention content, technology, and context together [147], our findings may inform theoretical developments in the field by providing new ideas around intervention processes and elements to test in future clinical trials of program effectiveness. We took some important first steps in the theory-building process (laying a foundation of evidence) by bringing together the fragmented and diverse data of iCBT programs, attempting to clearly define and report iCBT design and delivery features, highlighting important relationships between variables [24,148], and creating generalizable hypotheses. Another strength of this study is our use of the PSD and IOM models to organize the collection, analysis, interpretation, and presentation of data [149]. Although not applied by the original authors of the included documents, the models helped us identify and link contexts, mechanisms, and outcomes in a systematic way within and across programs.

Several challenges placed limitations on our findings. The 11 context-mechanism-outcome configurations we developed were dependent on the level of detail provided in the documents included in this synthesis. iCBT program descriptions were brief and details around therapeutic or technological features used (and associated theory or justification) were limited. Thus, the specifications of each technological (PSD) feature are not accounted for with our approach (which required a high level of abstraction) and readers should consider that the differences within features of the same type may be just as large as the differences across feature types (eg, rehearsal activities may differ among iCBT programs but these differences are not included in our configurations). We hope the hypotheses that we have generated can be applied to more detailed studies in the future that explore this important issue. In addition, few ineffective interventions (those that did not generate anxiety reductions) were identified for our review; therefore, we were unable to explore aspects of the delivery context or PSD features that might contribute to undesirable treatment effects with iCBT. As the dissemination and use of reporting standards (eg, Consolidated Standards of Reporting Trials–eHealth [150]) and requirements to document the design and delivery components for internet-based interventions become more common, we may not need to rely on additional models to operationalize data for comparisons across studies. We also acknowledge that information on other factors (ie, mediators or moderators) that may affect how iCBT programs work, such as user’s psychological characteristics (eg, cognitive processing style, beliefs or attitudes, skills, and literacy [151]), user engagement (eg, adherence, satisfaction, and motivation [152]), or environmental and cultural influences (eg, health care policies, user’s location, and societal perceptions of health), were not included in the configurations because of the lack of or inconsistent reporting. Thus, our proposed hypotheses for how iCBT programs for children and adolescents with anxiety work provide a constructive start to understanding their function but may not be complete. For example, once we better understand user characteristics, we may be able to identify subgroups of users who respond to iCBT, or some features of it, more than others. Although organizing our findings by program types led to a redundancy in the PSD mechanisms in configurations across program types, this redundancy also demonstrated the similarities that were universally found in iCBT programs. In the future, we recommend a more formal consideration of program type (ie, explicitly identifying the program’s targeted level of prevention) before designing and evaluating a program, as there are important differences in the target users, program design and delivery, and outcome measures used that may have a significant impact on program effects that should not be overlooked.

Future Directions

As more complex and sophisticated technological mediums or delivery methods (ie, mobile phones and wearables) and features (ie, gamification, virtual reality, and virtual agents) are being developed and incorporated into new technology-based treatments, we need to understand the first principles for how the individual and most basic applications of PSD features in iCBT programs work, so we can scale up our understanding of their effects in parallel with advancing technology and the complexity of program design. This review highlights 2 recommendations for future directions in the iCBT field.

First, studies designed to assess the impact and functions of identified program components and identify other components that are relevant for the design of new iCBT programs for children and adolescents with anxiety are needed. Evaluating individual program features to understand their theoretical level of action (what the feature intends to do; eg, based on CBT or the PSD model), instantiation (how the feature was executed; eg, timing and volume), quality (a distinguishing aspect of the feature; eg, degree of personalization, size, or color), and their effect (the result or consequence of a feature; eg, initiating and reinforcing behavior) [153-156] may provide insights into what the most persuasive features are and how they can be deliberately combined to support users’ desired behavior change.

Evaluation of individual features requires the use of certain methodological frameworks [157,158] and study designs that allow for more timely feedback, iteration, and fewer resources for testing (ie, participants, multiple health care centers, and funds). For example, modeling and predictor analyses [159,160], multifactorial designs [161-163], trials with multiple treatment arms [164], adaptive evaluation strategies (ie, the multiphase optimization strategy [165,166]), or the use of mixed methods and the triangulation of data [167,168] may be attractive alternatives to traditional clinical trials [169].

Second, to advance our understanding of the causal mechanisms that underpin effective iCBT programs, we will need to address what and how therapeutic content (ie, CBT skills) is delivered using PSD (technology-based) features to produce the intended and actual attitude or behavior changes. This will involve developing a framework that integrates the PSD model with the CBT framework and a theory of behavior change (for a review of theories see [170]) to identify specific combinations of therapeutic content and technological features designed to help users meet their health goals. A holistic framework by Wang et al [171] combines behavioral theories and the PSD model to provide a starting point for more theoretical and comprehensive designing, reporting, and evaluation of persuasive systems [171].

Conclusions

Although iCBT effectiveness for children and adolescents with anxiety has been demonstrated, not all programs benefit users in the same way. This leaves room for programs to be further optimized. PSD (technological) features can be intentionally selected and incorporated into the design and delivery of iCBT programs, making it an aspect of treatment that is under the control of developers. The hypotheses that we generated suggest that multiple key PSD features may work together to help users actively engage with therapeutic content and practice newly acquired skills. The type and degree of adjunct support will vary based on the level of prevention and user characteristics (ie, symptom severity) the program was designed to target and can influence what and how certain features operate within the program. The key PSD features and aspects of context identified require formal testing to understand whether, and to what extent, they are effective and how they function. These next steps may involve new conceptualizations of effectiveness and evaluation methods. As we improve our understanding of how the components of iCBT work (their proposed purpose) and what users prefer and need, we can create programs with better objective and subjective effectiveness. This systematic and deliberate approach to iCBT design and evaluation will increase the pool of evidence-based interventions available to prevent and treat anxiety in children and adolescents.