Published on 24.08.16 in Vol 3, No 3 (2016): Jul-Sept
Gamification and Adherence to Web-Based Mental Health Interventions: A Systematic Review
Background: Adherence to effective Web-based interventions for common mental disorders (CMDs) and well-being remains a critical issue, with clear potential to increase effectiveness. Continued identification and examination of “active” technological components within Web-based interventions has been called for. Gamification is the use of game design elements and features in nongame contexts. Health and lifestyle interventions have implemented a variety of game features in their design in an effort to encourage engagement and increase program adherence. The potential influence of gamification on program adherence has not been examined in the context of Web-based interventions designed to manage CMDs and well-being.
Objective: This study seeks to review the literature to examine whether gaming features predict or influence reported rates of program adherence in Web-based interventions designed to manage CMDs and well-being.
Methods: A systematic review was conducted of peer-reviewed randomized controlled trials (RCTs) designed to manage CMDs or well-being and incorporated gamification features. Seven electronic databases were searched.
Results: A total of 61 RCTs met the inclusion criteria and 47 different intervention programs were identified. The majority were designed to manage depression using cognitive behavioral therapy. Eight of 10 popular gamification features reviewed were in use. The majority of studies utilized only one gamification feature (n=58) with a maximum of three features. The most commonly used feature was story/theme. Levels and game leaders were not used in this context. No studies explicitly examined the role of gamification features on program adherence. Usage data were not commonly reported. Interventions intended to be 10 weeks in duration had higher mean adherence than those intended to be 6 or 8 weeks in duration.
Conclusions: Gamification features have been incorporated into the design of interventions designed to treat CMD and well-being. Further research is needed to improve understanding of gamification features on adherence and engagement in order to inform the design of future Web-based health interventions in which adherence to treatment is of concern. Conclusions were limited by varied reporting of adherence and usage data.
JMIR Ment Health 2016;3(3):e39
- Web-based mental health interventions;
- patient compliance;
- patient nonadherence
Common mental health disorders and poor well-being have significant economic, social, and individual costs [- ]. The issue of promoting well-being while improving and managing mental health conditions remains a worldwide priority [ ].
Web-based apps have been widely accepted and recognized as a cost-effective means by which to deliver proven and effective evidence-based therapies that were traditionally face-to-face, such as cognitive behavioral therapy (CBT), to improve mental health and well-being outcomes [- ]. Web-based interventions provide an advantage over traditional face-to-face delivery due to their potential to reach wider populations through the removal of many access barriers, such as limited numbers of trained and available therapists, long waiting lists, high delivery costs, transportation, geographical issues, and social stigma attached to treatments [ - ].
An increasing number of Web-based platforms have been developed that provide treatment and resources for a wide range of conditions, common mental disorders (CMDs), serious mental health disorders, well-being, and lifestyle improvement. However, dropout and nonadherence are often high and vary widely. Reported rates of attrition range between 35% and 99% [- ]. Context effects and health conditions influence adherence [ , ]. This is of critical importance because greater adherence to Web-based interventions is associated with improved mental health outcomes [ , ], whereas low adherence is reported to limit effectiveness of treatments [ ].
A growing body of research has identified a range of technology-driven features that contribute to program adherence, quality, design, and usability of Web-based interventions [, ]: persuasive technology [ ], including “push factors” and short message service (SMS) text message notifications, alerts or personalized reminders [ , ], weekly tracking [ ], incentives [ , ], interactive features [ ], and social networks [ ]. However, variation in reporting and measuring adherence has complicated understanding of the role of technological features [ ].
Findings from the gaming literature have suggested that the inclusion and use of gamified features in Web-based health interventions may increase interest and enjoyment, improving user experience. This, in turn, may positively influence engagement and program adherence and encourage desired health behavior changes [- ]. Gamification has been defined as “the use of game design elements in nongame contexts” [ ]. It differs from serious games, which refers to the use of games in their entirety within nongaming contexts (as opposed to selected elements or individual features of a game). Thus, gamification is the use of individual features of game design applied in a context not usually associated with video gaming or game play. However, agreement of conceptual understanding remains debated [ ] and academic opinion is varied. Gamification has enjoyed a recent explosion of success and increasing interest in a wide array of contexts beyond entertainment, health, education, news, and sustainability [ - ]. However, interest in game design has been researched in the fields of human-computer interaction and motivational psychology for much longer.
Recent research has called for the continued identification of features and “active” components that are most effective in improving program adherence while ensuring treatment remains effective [, , ]. A number of important adherence review studies have been published. For example, Kelders et al [ ] identified predictors of high adherence such as randomized controlled trial (RCT) study design, frequency of counselor interaction (frequency of peer interaction was not found to predict adherence), more frequent updates and reminders, more extensive use of dialog support, and more frequent intended usage. In addition, van Ballegooijen et al [ ] reported adherence to guided Internet CBT (iCBT) interventions for depression were equal to that of face-to-face delivery. Before that, Brouwer et al [ ] reported that elements of interventions associated with human support (guided) were associated with higher adherence in physical health interventions. Schubart et al [ ] identified that tailored advice, feedback, and guided programs increased user engagement in chronic health interventions. Earlier reviews focused on reporting the extent of the problem in the context of mental health interventions [ , ]. However, no prior reviews were identified that explicitly examined the role of gamification on adherence in the context of Web-based health interventions designed to treat CMD and improve well-being.
This review seeks to (1) explore, through systematic review of published peer-reviewed studies, the role of gaming features in Web-based interventions for the treatment of common mental health disorders or well-being and (2) to identify the “active ingredients” that influence treatment adherence.
The specific objectives of this study were to:
1. Identify studies that have incorporated gaming features into the design of their intervention to improve outcomes for CMDs and well-being;
2. Identify gamification features that influence adherence;
3. Report current rates of adherence;
4. Determine whether effects of the gamification feature on adherence varies across subgroup populations; and
5. Identify all terms commonly used to report adherence and maintenance with Web-based CMD and well-being and report the extent to which these are commonly reported in studies.
This review was registered with PROSPERO on April 16, 2015 (CRD42015017689).
A comprehensive search of seven electronic databases was conducted: Medline (Ovid interface), PsychINFO (Ovid interface), Cochrane Library, the Cumulative Index to Nursing and Allied Health Literature (CINHAL; EBESCO interface), Business Source Complete (EBSCO interface), Inspec (Ovid interface), and the ACM Digital Library. Search dates were between database inception and April 2015. Search strategies were customized for each database.
A combination of search terms were used to identify all relevant articles under the following categories: “Web-based,” “intervention,” “CMD/well-being,” and “adherence” ().
The inclusion criteria included:
1. The study must have included one or more gamification feature in the intervention;
2. The study was designed to manage any CMD or improve well-being (including physical conditions that report CMD/well-being outcome);
3. The intervention was delivered via the Web (Internet);
4. The intervention was designed to be accessed on more than one occasion;
5. RCT study design; and
6. The study must have reported at least one measure of attrition, adherence, engagement, dropout, or other term referring to such.
The exclusion criteria were (1) the intervention was delivered via paper, face-to-face, CD-ROM, or other non-Web-based method and (2) participants were younger than age 18 years.
The definition of gamification used in this review was “the use of game design elements in nongame contexts” . Ten gamification features were reviewed. The features reviewed were those identified by Cugelman [ ]. These were informed by Hamari et al [ ] and are described in . Two authors (MB, AJ) discussed the selection of this list.
Two reviewers (MB, NoN) independently reviewed the title for relevance, then the abstract against inclusion/exclusion criteria. A third reviewer (HvW) resolved any disagreements. Measures of agreement were calculated (kappa statistic). Full-text articles of those included were retrieved at this stage. Two reviewers (MB, NoN) independently reviewed each article. Each was assessed against the inclusion/exclusion criteria outlined previously. The first instance where it did not meet eligibility was recorded as the reason for exclusion and the study was not assessed against additional inclusion criteria . Reviewers discussed all articles that were not unanimous (see the PRISMA flowchart in ).
A data extraction form was developed and piloted with five studies meeting the inclusion criteria. The following data were extracted for review (MB):
1. Participant characteristics: including recruitment setting, use of diagnostic interview, total number of participants randomized to intervention, sample size, gender, and age.
2. Intervention characteristics: including intervention name, number of trial arms, primary condition, therapeutic approach, intended duration (weeks), modules to be completed, automated or guided delivery, format of delivery, and outcome measures used.
3. Interactive elements of intervention: including automated email reminders, interactive quizzes, social networking (community forum), homework, or diary tasks.
4. Gamification features: a record of the feature(s) used in the intervention design.
5. Adherence: including adherence to study protocol, completion rate, and term used to refer to adherence.
Assessment of Risk of Bias in Included Studies
The quality of each included study from a risk of bias perspective was assessed (NoN) using the Cochrane Collaboration Risk of Bias tool as described in the Cochrane Handbook for Systematic Reviews of Interventions . Each included study was assessed against the six bias domains and source of bias subdomains outlined in order to produce a summary risk of bias assessment score (low, high, or unclear). The majority risk level in each subdomain was utilized and summarized across all domains. If there were four or more subdomains with a low risk of bias, then it would be judged that the study showed an overall low risk of bias.
Descriptive and exploratory analyses were conducted in SPSS 22 (IBM Corp, Armonk, NY, USA) and Review Manager 5.3 (RevMan 5.3; The Cochrane Collaboration, Copenhagen, Denmark).
An adherence rate to study protocol was calculated as the principal summary measure. A percentage score for adherence to each intervention was calculated to allow comparison across interventions. This was the percentage of those completing postassessment by the number of participants initially randomized (to an intervention trial arm) because limited data were available on total completion rate of interventions.
A series of procedures were carried out. First, the adherence rates of interventions using only one gamification feature were visually presented in a series of forest plots, shown in comparison to adherence rates for inactive controls (where available). The mean adherence rate of interventions using only one gamification feature was calculated by adding the adherence rate for each study that used this feature and dividing it by the number of these studies. A one-way ANOVA was conducted to identify statistical differences between adherence rates for studies using different, single gamification features. Second, adherence rates for interventions using one, two, or three (total number of) gamification features were similarly calculated and presented visually in a bar chart. Forest plots showing adherence compared to inactive control (where available) are also presented. A one-way ANOVA explored statistical differences in adherence. Third, the mean adherence rate was calculated per condition and displayed in a bar chart. A one-way ANOVA explored statistical differences in adherence per condition. Finally, following these comparisons, an independent t test was conducted to examine statistical differences in adherence as a result of additional interactive features (in dichotomous features; ie, sequential or free navigation and automated or guided delivery). A one-way ANOVA was conducted to explore differences in features which included three or more categories (intended duration and modules, total number of interactive intervention characteristics). Values within each were recategorized to form three distinct categories.
A standard multiple regression analysis was performed to explore the role of interactive intervention characteristics in explaining adherence. Independent variables were entered into the model as a block using the enter method (total number of gamification features, guided or automated, sequential or free navigation, intended duration, modules, and total number of interactive intervention features). Adherence was entered as the dependent variable. It is recommended that 15 cases be included per predictor variable in social sciences .
After duplicates were removed, 2170 titles and 774 abstracts were reviewed. Following full-text review, 61 RCTs remained (). The kappa statistic showed good agreement between reviewers at the title and abstract stage (κ=.933 and κ=.694, respectively).
In all, 47 RCTs were two-armed trials, 12 were three-armed trials, one was a four-armed trial, and one was a six-armed trial. Of the two-armed trials, 21 compared to a wait-list control group, three to treatment as usual, one to placebo, one reported no treatment, and 20 used an active comparator. These included 11 interventions and nine attention controls. Of the 12 three-armed trials, two compared to two inactive controls, nine compared to an active intervention plus wait-list control or treatment as usual, and one included two interventions using different therapeutic approaches. The four-armed trial compared to a Web-based intervention plus tracking and two inactive conditions. The six-armed trial consisted of six active interventions.provides a full reference list of all 61 included articles and a summary of intervention characteristics of all 82 included arms (where no arm is recorded this is to indicate it was the additional trial arm in an RCT).
Cochrane Risk of Bias Score
Of the 61 RCTs included in this systematic review, 37 (61%) were judged to be of high risk of bias, eight (13%) were judged to be of low risk of bias, and an unclear risk of bias was assigned to 16 (26%) of the included studies (). The quality of the evidence provided within the included studies was variable. Sources of bias included inconsistent implementation of interventions, follow-up methods, completion rates, and studies being underpowered to statistically detect intervention effects, and self-selected study populations ( ).
The main results table presented inreports a summary of key characteristics for all included intervention arms (n=82).
Across the 61 RCTs, 14,726 participants were randomized to either an intervention or control condition. The RCTs varied widely in size from a total of 24 to 23,213 randomized participants. Overall, 41 RCTs had sample sizes less than 200, 15 had sample sizes between 200 and 999, and five had sample sizes more than 1000. Four RCTs included females only. Four RCTs restricted inclusion to those older than 45 years and one included a sample of those between the ages of 18 and 24 years; the remainder (n=56) recruited from age 18 years and older. Participants were recruited from the general population (n=39), clinical populations (n=11), students (n=4), military (n=2), and organizational workplaces (n=5). The majority of RCTs were conducted in Australia (n=20) and the United States (n=18). The majority of participants self-referred into a trial (87%).
From the 61 RCTs, a total of 82 active intervention arms were identified. As such, the following section presents the adherence and gamification results from 82 interventions.
Interventions were designed to treat a range of symptomology: depression (n=30), depression with comorbid anxiety (n=5), anxiety including social anxiety disorder and generalized anxiety disorder (n=9), well-being (n=7), social phobia (n=7), posttraumatic stress disorder (n=4), obsessive compulsive disorder (n=1), panic disorder (n=1), stress (n=3), binge eating disorder (n=1), and physical conditions (n=14). A total of 37 interventions reported use of clinical diagnostic interview.
The 14 interventions designed to manage physical conditions were physical activity (n=3), smoking cessation (n=1), sexual dysfunction in female cancer patients (2), headache (n=2), insomnia (n=4), and weight loss (n=2). Pre- and postoutcome measures for a CMD or well-being were reported in each of these trials.
All interventions were Web-based and available via personal computers, laptops, and Internet-enabled devices. In total, 47 different therapeutic interventions were identified and a number of these were utilized in successive RCTs: MoodGYM (n=6), Beating the Blues (n=3), MoodGYM and BluePages combined (n=2), deprexis (n=2), SHUTi (n=2), and The Shyness Program (n=5). In this review, “intervention” refers to the Web-delivered therapeutic treatment program.
Of the 82 interventions, 50 were automated. Automated delivery of an intervention refers to the use of an intervention treatment program without any human support. The remainder (n=32) were guided. Guided delivery refers to support of a human guide during the course of the treatment. Guided interventions included a range of guided interactions: therapeutic telephone contact (n=13), face-to-face therapy (n=5), and therapeutic emails (n=21).
In total, 59 interventions were based primarily on CBT, one of which used CBT in combination with psychoeducation and interpersonal psychotherapy, two used cognitive restructuring without behavioral activation, two used mindfulness, two used positive psychology, one was based on a stress and coping model, two used Internet psychotherapy, five employed health behavior change techniques, and nine did not specify a therapeutic approach. Some studies noted additional elements used in the intervention. These included cognitive bias modification online (n=1), Internet-delivered supportive counseling (n=1), psychoeducation (n=2), interpersonal therapy (n=2), problem solving (n=2), motivational interviewing or motivational principles (n=2), and physical activity (n=1).
Format of Delivery
In all, 63 intervention arms were released sequentially in a predetermined order over time, 16 could be freely navigated, two [, ] presented modules in sequence but allowed participants free navigation, and one [ ] included free navigation once a specific module had been completed.
The duration of the interventions ranged between 3 and 20 weeks (mean 7.8, SD 2.4). One did not specify the intended duration , although it clearly stated that the intervention was to be used more than once. Many were eight (n=25), six (n=22), or 10 (n=8) weeks in duration.
The number of modules within each intervention ranged between zero and 13 (mean 6.4, SD 2.6). Three did not use a modular format. Most interventions included six (n=30), eight (n=11), or five (n=9) modules.
Interactive Intervention Elements
Information available regarding interactive elements employed in each intervention varied. Text was presented in all, accompanied by a range of additional elements, automated email reminders (n=36), SMS text message reminders (n=13), telephone reminders (n=12), interactive quizzes (n=37), social media (n=11), and homework (n=47).
Eight of 10 gamification features reviewed were identified in use: story/theme, progress, feedback, goal setting, rewards, challenge, badges/trophies, and points. No study incorporated levels or game leaders. The majority of interventions used only one gamification feature (n=58); the maximum number used in any one intervention was three. Of the interventions employing only one gamification feature, story/theme was most commonly used (n=33), followed by progress (n=10), goal setting (n=6), rewards (n=6), and feedback (n=3). Of those using more than one feature (n=24), 19 used two features and five incorporated three features.
A wide variety of terms were used to report a measure of adherence: adherence, attrition, dropout, noncompleters, lost to follow-up, participant withdrawal, nonresponse, completion rate, did not complete, retention rate, loss, and compliance.
Overall adherence to study protocol ranged between 3.37% and 100% (n=82, mean 71.7%, SD 20.3%). Adherence to control groups ranged from 5.98% to 100% (n=58, mean 78.2%, SD 19.1%). The mean adherence rate of studies excluded for not including a gamification feature was 75.2% (SD 19.6%) with a range of 5.3% to 100%. There were differences between the ways in which studies classified adherence and reported their data, making meaningful comparison complicated. The limitations of such are addressed in the Discussion.
Reasons for nonadherence were provided in 33 RCTs. The following reasons were provided: lack of time, disinterest, no need for treatment, hardware or technical issues, program perceived as noneffective, life events, felt better after a few modules, disappointed by group assignment, holiday, work commitments, poor health, and no longer wish to participate. One RCT  reported removal of 19 participants due to fraudulent participation. One RCT only reported data for those participants who completed the entire intervention (due to a programming error).
Limited usage data were reported, mean number of modules completed (n=39 reported this data), program completion (n=45), with a mean completion rate of 54.0% (SD 24.6%), and log data. The way in which log data was reported varied further; mean time spent per visit in minutes (n=4), mean log-on rate (n=5), total time duration (n=2), total page views (n=1), and activities opened (n=1).
Statistical Analysis of Intervention Characteristics and Adherence
Adherence was examined per gamification feature for those interventions that employed only one gamification feature (n=58). Forest plots present the adherence per intervention arm in comparison to its control condition (where a control condition was used as opposed to an active intervention). The following forest plots show two columns: the intervention arm and the control group. The term “events” refers to the number of randomized participants remaining at postassessment, whereas “total” refers to the total number randomized to that intervention at the start of the trial. If a score of zero is recorded, either the data was unavailable or there was no control group to compare against. For example, in some RCTs the comparator group was another (treatment) intervention or a modified version of the same intervention. The weight is automatically calculated by RevMan based on the total number of participants in the trial. A mean adherence is also reported; this does not include the control arm data (unlike the forest plots).
Goal setting was defined as users informed of a goal or are required to establish their own goals to achieve over the duration of the program (intervention). Six interventions incorporated goal-setting activities. Adherence compared to control is shown in. Mean adherence for the six interventions was 72.3% (SD 22.8%).
Progress was defined as progression through the program or game. Participants could monitor progress with self or others. Ten interventions incorporated progress. Adherence compared to control is shown in. Mean adherence was 53.5% (SD 31.2%).
Feedback was defined as automated feedback provided on progress. Three interventions incorporated automated feedback. Adherence compared to control is shown in. Mean adherence was 75.9% (SD 24.0%).
Rewards for achievement included in-game goods or artifacts (functional or nonfunctional to the program). Six interventions utilized rewards. Adherence compared to control is shown in. Mean adherence was 72.1% (SD 13.3%).
A story/theme included fun and playfulness, playing out an alternate reality, an avatar, or an illustrated story. In all, 33 interventions used a story/theme feature . Adherence compared to control is shown in. Mean adherence was 76.3% (SD 17.0%).
A one-way ANOVA did not reveal any statistical differences between interventions using the preceding gamification features (n=58, P=.19).
Comparison of Adherence Rates per Use of Total Number of Gamification Features
The mean adherence rates for interventions incorporating one, two, and three gamification feature were 71.5% (SD 21.6%), 70.5% (SD 17.9%), and 78.2% (SD 12.3%), respectively. A one-way ANOVA did not reveal any statistically significant differences (P=.74). Adherence compared to control are displayed in three forest plots to visualize differences in studies employing one, two, and three gamification features ().
Gamification Use by Condition
shows the frequency each individual gamification feature was employed in an intervention per condition. The total number is more than 82 because some interventions used two or more features. The mean adherence rate per condition is presented in . One-way ANOVA did not reveal any statistical differences (P=.18).
Examination of Additional Intervention Characteristics
Delivery format, such as sequential (n=65, mean 72.1%, SD 21.3%) and free navigation (n=17, mean 70.2%, SD 16.2%), did not influence adherence to intervention (P=.20). Automated interventions had a mean adherence of 67.9% (n=50, SD 21.8%) compared to guided interventions (n=32, mean 77.5%, SD 16.2%). An independent t test did not reveal this to be statistically significant (P=.05).
One-way ANOVA did not reveal any statistical difference for intended duration (6 weeks: mean 65.1%, SD 24.3%; 8 weeks: mean 74.0%, SD 17.3%; 10 weeks: mean 76.4%, SD 17.1; P=.15), number of modules (<6 modules: mean 70.5%, SD 24.3%, 7-9 modules: mean 73.5%, SD 17.3%; ≥10 modules: mean 73.6%, SD 17.1%; P=.80), or total number of interactive features (0-2 features: mean 67.5%, SD 22.3%, 3-4 features: mean 77.5%, SD 15.3%; 5-6 features: mean 77.8%, SD 19.0%; P=.08).
Standard multiple regression indicated that the independent variables only explained 10.3% (P=.22) of the variance in adherence rate.
This review sought to identify RCTs that incorporated gaming features into the design of Web-based health interventions to treat CMDs or well-being. Physical health interventions that included an outcome measure for CMD or well-being were included when identified. This is the first review that has examined the use and role of gamification features on adherence in this context. Ten key gamification features were examined .
A total of 61 RCTs comprising 82 intervention arms were analyzed and 47 separate interventions were identified. Interventions designed to treat depression, which were intended to be 8 or 6 weeks in duration, incorporating six modules, and utilizing CBT were most common. This is shorter than the typical 10-week duration identified previously . The most common format of delivery was a weekly sequential release of modules. Interventions allowing free navigation were less common. Interventions were more likely to be automated rather than guided. The majority of RCTs were found to have a high risk of bias.
One aim was to explore whether gamification features have been incorporated into the design of interventions developed to manage CMD or improve well-being. This review identified eight gaming features in use. The majority of studies used only one feature (goal setting, progress, feedback, reward, or story/theme). No studies specifically compared the impact of different gamification features on program adherence in the same RCT; however, one trial compared six versions of the same intervention (MoodGYM). Two of these trial arms were found to incorporate two gamification features, whereas the remaining four arms only included one . However, the purpose of the trial was not to compare use of these features. Overall, the most common feature utilized was story/theme . Interventions using this did not commonly incorporate additional features; only six were found which did [ - ]. Progress and feedback were used together in six interventions [ , , - ]. Points and challenge were not frequently implemented and levels and game leaders were not incorporated at all.
The main aim of this review was to explore whether incorporating gamification features into the design of these interventions influenced adherence to treatment. In order to examine this, adherence was examined first. Adherence to intervention was lower overall than adherence to control when control was inactive (means 71.7% and 78.2%, respectively). Previous reviews reported higher adherence to guided interventions compared to automated interventions . This review supported this (77.5% and 67.9%, respectively) lending further support for the role of guides in self-help treatments. However, this difference was not statistically significant.
Looking at the role of gamification features, adherence rates were compared across those using different features when only one feature was incorporated. No statistical difference was observed, which supported use of one single feature over another, despite the mean adherence rates ranging from 53.5% to 75.9% for progress and feedback, respectively. Nor was there any significant difference found between studies using different total numbers of gamification features (one, two, or three features). However, the forest plots suggest that as additional features are added, adherence moved closer to favoring the intervention over control.
An additional aim of this review was to determine whether adherence to interventions using gamification differed across health conditions. Interventions designed to treat social phobia had higher adherence than those designed to treat well-being (P=.048). However, no other statistical difference was observed. Findings reported here are in line with established published findings. Kelders et al  reviewed the impact of persuasive features and system design. They characterized typical studies and identified that RCT design, more frequent usage, updates, and dialog support predicted higher adherence. Interventions covered lifestyle, physical health, and mental health programs. Health care context did not predict adherence.
As a result, additional intervention features were also examined in an effort to shed light on active ingredients influencing adherence. Again no statistically significant differences were observed and none of the variables were found to explain any significant proportion of the variance in adherence rate (total variance explained was only 9.4%). However, mean adherence increased as intended duration increased from 6 or 8 weeks to 10 weeks’ duration.
Criticisms of gamification have been levied and discussed in the literature . For example, a Gartner report [ ] stated “gamification is currently driven by novelty and hype,” whereas Bogost [ ] considered it a quick fix adopted by businesses to increase and promote engagement. Underpinning these criticisms is the concern that implementation of individual features such as points and leader boards actually miss the real essence and power of games as motivational techniques, which have the potential to positively encourage behavior change [ ] or positively encourage adherence to treatment programs that reduce individual suffering through reductions in clinical symptoms. Although many studies were found to have incorporated one game feature into their treatment program, it is possible that such negative opinions may have reduced wider application in this health context due to concerns of appropriateness. However, Cugleman [ ] highlighted that gamification, like other persuasive architectures, has merit if implemented in the right way.
It is important to consider the way in which gamification features identified in use were incorporated into intervention designs. There were only three examples in which the use of game mechanics was clearly acknowledged and the intention of use identified as a means to address and increase user engagement and enhance enjoyment. Cobb and Porier  used in-game rewards, badges, and challenges to engage participants in a daily challenge to improve well-being. In this example, adherence was high and usage data well reported. More than half the participants continued to engage with the program at 60 days and 92.4% were reported to have completed one challenge. Authors reported a positive dose-response relationship for well-being in which higher program engagement predicted better well-being at postassessment and follow-up. Similarly, a guided physical activity intervention that assessed well-being outcomes applied motivational principles and game elements, including visualization of progress and automated goal setting activities, specifically to enhance engagement and participation [ ]. Imamura et al [ ] incorporated comic strip stories in an effort to “foster learner’s interest in the program” (p 3). However, the remaining interventions did not commonly acknowledge or describe their use of gamification features. For example, Titov et al [ ] implemented story/theme, goal setting, and challenge in The Shyness Program, without acknowledgement that game mechanics were incorporated in the intervention. Indeed incorporation of such features may not have been considered (by those who developed the intervention) to represent implementation of game mechanics. Further examples include Sheeber et al [ ] who incorporated three features, without recognition of such, in a guided intervention to manage maternal depression. In this example, intervention design and development was focused on principles that promoted self-regulated learning. Adherence and completion rate was high (97% and 63%, respectively). Intervention descriptions focused on the theoretical basis rather than the technological aspects of development. The intentional use of game design elements has recently been suggested as a defining feature of the operationalization of gamification [ ]; as such, this highlights the potential importance of intended use in operationalization of features. Doherty et al [ ] outlined the importance of encouraging engagement with, or adherence to, treatment rather than technology and that it is important to bear this in mind during discussion on use of gamification features in this context in which the ultimate intention is to alleviate suffering and improve well-being.
Strengths and Limitations
This review was based on an extensive search of a large number of health and computer science databases. Hand searching was not conducted, but the expertise of the multidisciplinary team means that although publication bias cannot be excluded, this comprehensive review did identify a large number of relevant studies.
This review aimed to explore the potential role of gamification to increase program adherence and engagement, adherence being an issue that has plagued Web-based health interventions for some time [, ]. In order to examine the role of gamification on adherence, adherence to study protocol was used. This was considered an objective, comparable measure calculated as a percentage of those (randomized) who completed postassessment outcome measures. Although this is useful, it offers less insight than module completion rates would. However, limited reporting of data, such as log-on rates, module completion, and mean access time, meant this was not possible. Only 34 studies reported a percentage for program completion and only 10 provided data for log-on rates, with one exception [ ]. These studies were all reported after 2009. A more comprehensive and standardized usage report across trials would assist and inform further analyses of adherence and program engagement. This finding is in line with previous discussion on adherence reporting [ , , ]. Morrison and Doherty [ ] provided a useful analysis of log data that could be replicated in future studies.
Interventions evaluated via RCT methodology was a specific inclusion criteria of this review; as such, it is possible that a body of literature pertaining to management of CMD or well-being that incorporate gamification features may have been excluded. However, RCTs follow robust methodological procedures and are considered to provide the highest quality evidence, so the approach adopted is of value .
Varied reporting complicated initial identification of studies for inclusion. Not all studies provided a detailed description of the intervention programs. However, seven provided clear, detailed descriptions of intervention features, including screenshots and illustrations [, , , - ].
Interventions using gamification features in conditions other than depression were small in number, which limited opportunity to explore the influence of gamification features on adherence across health conditions.
Furthermore, the way in which specific gamified features were incorporated warrants discussion. In this study, rewards were commonly seen to be financial in nature, whereas progress was often controlled progression through the system. Goal setting and feedback were aligned with established strategies used in therapeutic treatment of CMD and their role is well defined in terms of supporting and encouraging behavior change. In reviewing intervention designs, it was not always possible to identify the intention behind each feature and they are also commonly used features in Web-based programs. However, they were not employed in all interventions and so remain of interest in this context.
It is important to acknowledge that adherence also may be influenced by additional factors that could not be assessed in this review. This is highlighted in the small variance rate (10.3%). Furthermore, attrition to mental health treatments is also experienced in face-to-face delivery formats.
Implications for Practice
Future research should look to examine whether application of specific gamification features influences adherence to protocol and completion rate. No RCT was identified that specifically considered the role of gamified features on promotion of adherence to mental health programs. This could be achieved through comparisons of the same intervention (in the same clinical population) adjusted to include either different gamification features, different combinations of gamification features, increasing numbers of gamification features, or use of one specific gamification feature compared to none. Studies looking to explicitly make these comparisons may shed further light on the role of individual features extracted from game design on adherence to Web-based health interventions. These effects should also be explored across different health and well-being contexts to identify whether inclusion of gamification features are more or less effective at increasing engagement and adherence across different patient populations and subgroups, such as different levels of clinical symptomology.
It would also be beneficial to explore the use of gamification in interventions based on alternative therapies to that of CBT (which comprised the majority of those reviewed here); for example, whether they have a role to play in encouraging engagement to interventions based on acceptance and commitment therapy. In addition to this, future research might benefit from exploration of gamification in interventions, allowing free navigation as opposed to a linear, weekly format as identified here. This may shed further light on the potential role of game mechanics on program engagement and adherence to treatment.
Assessment of participant’s motivation to complete the full intervention on entering the program might also offer an alternative way to explore the role of gamification. Use of extrinsic motivation features may influence some people more than others. Exploration of people’s reasons for participating at the onset of a RCT might shed light on the role of gamification features. Gamification promotes motivation through external means, which means those who are internally motivated may not be influenced to the same extent.
Research findings have indicated that higher adherence is associated with increased treatment effectiveness (dose-response relationship). Some have discussed a beneficial level of engagement that facilitates a positive health outcome  and this is certainly an area for future interest. This was not examined in this review, but could be further explored in relation to the inclusion of gamification features.
Gaming features have explicitly been implemented into the design of interventions to treat CMDs and well-being. However, this was not common. This review did not find any evidence that use of specific gamification features was associated with higher adherence to the intervention program as measured by adherence to protocol. Furthermore, no evidence was found to suggest that interventions incorporating additional gamification features had any statistically significant influence on adherence. However, no studies explicitly examined the role of gamification on program adherence or engagement.
What the review did show was that guided interventions and interventions intended to last 10 weeks, as opposed to 6 or 8 weeks duration, and those incorporating three gamification features had a higher mean adherence rate. This may provide initial insight into the design of future interventions wishing to utilize gamification features in an attempt to address adherence and contribute to the ongoing discussions surrounding the use of game design elements in nongame contexts.
Conflicts of Interest
Multimedia Appendix 1
Keyword search table.PNG File, 998KB
Multimedia Appendix 2
Gamification features.PNG File, 299KB
Multimedia Appendix 3
PRISMA Flow diagram of included and excluded studies.PNG File, 2MB
Multimedia Appendix 4
Main results table and references of included studies.PDF File (Adobe PDF File), 78KB
Multimedia Appendix 5
Forest plots showing mean adherence in studies using one, two and three gamification features.PNG File, 6MB
Multimedia Appendix 6
Gamification features used per condition.PNG File, 727KB
Multimedia Appendix 7
Mean adherence rate per condition.PNG File, 1MB
- Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, et al. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One 2013 Aug;8(8):e70086 [FREE Full text] [CrossRef] [Medline]
- Wells KB, Miranda J, Bauer MS, Bruce ML, Durham M, Escobar J, et al. Overcoming barriers to reducing the burden of affective disorders. Biol Psychiatry 2002 Sep 15;52(6):655-675. [Medline]
- Newport F, editor. The Gallup Poll: Public Opinion 2010. Lanham, MD: Rowman & Littlefield Publishers; 2010.
- European Pact for Mental Health and Well-being. EU High-Level Conference Together for Mental Health and Wellbeing Brussels, 12-13 June 2008. 2008. URL: http://ec.europa.eu/health/ph_determinants/life_style/mental/docs/pact_en.pdf [accessed 2016-03-01] [WebCite Cache]
- Andersson GP, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009;38(4):196-205. [CrossRef] [Medline]
- Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet-based interventions for traumatic stress-related mental health problems: a review and suggestion for future research. Clin Psychol Rev 2009 Jul;29(5):410-420 [FREE Full text] [CrossRef] [Medline]
- Barak A, Hen L, Boniel-Nissim M, Shapira N. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J Technol Hum Serv 2008 Jul 03;26(2-4):109-160. [CrossRef]
- Morrell CJ, Sutcliffe P, Booth A, Stevens J, Scope A, Stevenson M, et al. A systematic review, evidence synthesis and meta-analysis of quantitative and qualitative studies evaluating the clinical effectiveness, the cost-effectiveness, safety and acceptability of interventions to prevent postnatal depression. Health Technol Assess 2016 May;20(37):1-414 [FREE Full text] [CrossRef] [Medline]
- Reynolds J, Griffiths KM, Cunningham JA, Bennett K, Bennett A. Clinical practice models for the use of e-mental health resources in primary health care by health professionals and peer workers: a conceptual framework. JMIR Ment Health 2015;2(1):e6 [FREE Full text] [CrossRef] [Medline]
- Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? A systematic review of the published literature. J Med Internet Res 2006;8(2):e10 [FREE Full text] [CrossRef] [Medline]
- Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health 2009;30:273-292. [CrossRef] [Medline]
- Finkelstein J, Lapshin O. Reducing depression stigma using a web-based program. Int J Med Inform 2007 Oct;76(10):726-734. [CrossRef] [Medline]
- Finkelstein J, Lapshin O. Reducing depression stigma using a web-based program. Int J Med Inform 2007 Oct;76(10):726-734. [CrossRef] [Medline]
- Klein B, Meyer D, Austin DW, Kyrios M. Anxiety online: a virtual clinic: preliminary outcomes following completion of five fully automated treatment programs for anxiety disorders and symptoms. J Med Internet Res 2011;13(4):e89 [FREE Full text] [CrossRef] [Medline]
- van den Berg S, Shapiro DA, Bickerstaffe D, Cavanagh K. Computerized cognitive-behaviour therapy for anxiety and depression: a practical solution to the shortage of trained therapists. J Psychiatr Ment Health Nurs 2004 Oct;11(5):508-513. [CrossRef] [Medline]
- Rothert K, Strecher VJ, Doyle LA, Caplan WM, Joyce JS, Jimison HB, et al. Web-based weight management programs in an integrated health care setting: a randomized, controlled trial. Obesity (Silver Spring) 2006 Feb;14(2):266-272 [FREE Full text] [CrossRef] [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- Melville KM, Casey LM, Kavanagh DJ. Dropout from Internet-based treatment for psychological disorders. Br J Clin Psychol 2010 Nov;49(Pt 4):455-471. [CrossRef] [Medline]
- Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012;14(6):e152 [FREE Full text] [CrossRef] [Medline]
- Cuijpers P, van Straten A, Warmerdam L, Andersson G. Psychological treatment of depression: a meta-analytic database of randomized studies. BMC Psychiatry 2008;8:36 [FREE Full text] [CrossRef] [Medline]
- Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]
- Manwaring JL, Bryson SW, Goldschmidt AB, Winzelberg AJ, Luce KH, Cunning D, et al. Do adherence variables predict outcome in an online program for the prevention of eating disorders? J Consult Clin Psychol 2008 Apr;76(2):341-346. [CrossRef] [Medline]
- Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther 2012 Aug;50(7-8):463-468. [CrossRef] [Medline]
- Morrison D, Wyke S, Thomson NC, McConnachie A, Agur K, Saunderson K, et al. A Randomized trial of an Asthma Internet Self-management Intervention (RAISIN): study protocol for a randomized controlled trial. Trials 2014;15:185 [FREE Full text] [CrossRef] [Medline]
- Sharry J, Davidson R, McLoughlin O, Doherty G. A service-based evaluation of a therapist-supported online cognitive behavioral therapy program for depression. J Med Internet Res 2013;15(6):e121 [FREE Full text] [CrossRef] [Medline]
- Kelders SM, Bohlmeijer ET, Van Gemert-Pijnen JE. Participants, usage, and use patterns of a web-based intervention for the prevention of depression within a randomized controlled trial. J Med Internet Res 2013;15(8):e172 [FREE Full text] [CrossRef] [Medline]
- Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, DeBar LL, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005;7(2):e16 [FREE Full text] [CrossRef] [Medline]
- Vervloet M, Linn AJ, van Weert JC, de Bakker DH, Bouvy ML, van Dijk L. The effectiveness of interventions using electronic reminders to improve adherence to chronic medication: a systematic review of the literature. J Am Med Inform Assoc 2012;19(5):696-704 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Korten AE, Brittliffe K, Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J Med Internet Res 2004 Dec 22;6(4):e46 [FREE Full text] [CrossRef] [Medline]
- Khadjesari Z, Murray E, Kalaitzaki E, White IR, McCambridge J, Thompson SG, et al. Impact and costs of incentives to reduce attrition in online trials: two randomized controlled trials. J Med Internet Res 2011;13(1):e26 [FREE Full text] [CrossRef] [Medline]
- Alexander GL, Divine GW, Couper MP, McClure JB, Stopponi MA, Fortman KK, et al. Effect of incentives and mailing features on online health program enrollment. Am J Prev Med 2008 May;34(5):382-388 [FREE Full text] [CrossRef] [Medline]
- Gliddon E, Lauder S, Berk L, Cosgrove V, Grimm D, Dodd S, et al. Evaluating discussion board engagement in the MoodSwings online self-help program for bipolar disorder: protocol for an observational prospective cohort study. BMC Psychiatry 2015;15:243 [FREE Full text] [CrossRef] [Medline]
- Maher CA, Lewis LK, Ferrar K, Marshall S, De Bourdeaudhuji BI, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res 2014;16(2):e40 [FREE Full text] [CrossRef] [Medline]
- Primack BA, Carroll MV, McNamara M, Klem ML, King B, Rich M, et al. Role of video games in improving health-related outcomes: a systematic review. Am J Prev Med 2012 Jun;42(6):630-638 [FREE Full text] [CrossRef] [Medline]
- McGonigal J. Reality Is Broken: Why Games Make Us Better and How They Can Change the World. New York: The Penguin Press; 2011.
- Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games 2013;1(1):e3 [FREE Full text] [CrossRef] [Medline]
- Hamari J, Koivisto J, Pakkanen T. Do persuasive technologies persuade?-A review of empirical studies. In: Persuasive Technologies. Cham, Switzerland: Springer International Publishing; 2014:118-136.
- Baranowski T, Buday R, Thompson DI, Baranowski J. Playing for real: video games and stories for health-related behavior change. Am J Prev Med 2008 Jan;34(1):74-82 [FREE Full text] [CrossRef] [Medline]
- Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining gamification. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments 2011 ACM. 2011 Presented at: 15th International Academic MindTrek Conferencenvisioning Future Media Environments; Sep 28-30, 2011; Tampere, Finland p. 9-15. [CrossRef]
- Seaborn K, Fels DI. Gamification in theory and action: a survey. Int J Hum-Comput St 2015 Feb;74:14-31. [CrossRef]
- Werbach K, Hunter D. For the Win: How Game Thinking Can Revolutionize Your Business. Philadelphia, PA: Wharton Digital Press; 2012.
- Kapp KM. The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. San Francisco, CA: John Wiley & Sons; 2012.
- Silva F, Analide C, Rosa L, Felgueiras G, Pimento C. Social networks gamification for sustainability recommendation systems. In: Distributed Computing and Artificial Intelligence. Berlin: Springer; 2013:307-315.
- van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, et al. Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One 2014;9(7):e100674 [FREE Full text] [CrossRef] [Medline]
- Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011;13(1):e2 [FREE Full text] [CrossRef] [Medline]
- Schubart JR, Stuckey HL, Ganeshamoorthy A, Sciamanna CN. Chronic health conditions and internet behavioral interventions: a review of factors to enhance user engagement. Comput Inform Nurs 2011 Feb;29(2 Suppl):TC9-TC20. [CrossRef] [Medline]
- Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
- Hamari K, Koivisto J, Sarsa H. Does gamification work? A literature review of empirical studies on gamification. In: Proceedings of the 2014 47th Hawaii International Conference on System Sciences. 2014 Presented at: 47th Hawaii International Conference on System Sciences; Jan 6-9, 2014; Waikoloa, HI p. 6-9. [CrossRef]
- Pallant J. SPSS Survival Manual, 5th edition. Berkshire, UK: McGraw-Hill Education; 2013.
- The Cochrane Collaboration. In: Higgins JP, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Chichester, UK: Wiley-Blackwell; 2008.
- Imamura K, Kawakami N, Furukawa TA, Matsuyama Y, Shimazu A, Umanodan R, et al. Effects of an Internet-based cognitive behavioral therapy (iCBT) program in Manga format on improving subthreshold depressive symptoms among healthy workers: a randomized controlled trial. PLoS One 2014;9(5):e97167 [FREE Full text] [CrossRef] [Medline]
- Ritterband LM, Bailey ET, Thorndike FP, Lord HR, Farrell-Carnahan L, Baum LD. Initial evaluation of an Internet intervention to improve the sleep of cancer survivors with insomnia. Psychooncology 2012 Jul;21(7):695-705 [FREE Full text] [CrossRef] [Medline]
- Schover LR, Yuan Y, Fellman BM, Odensky E, Lewis PE, Martinetti P. Efficacy trial of an Internet-based intervention for cancer-related female sexual dysfunction. J Natl Compr Canc Netw 2013 Nov;11(11):1389-1397 [FREE Full text] [Medline]
- Sanders MR, Baker S, Turner KM. A randomized controlled trial evaluating the efficacy of Triple P Online with parents of children with early-onset conduct problems. Behav Res Ther 2012 Nov;50(11):675-684. [CrossRef] [Medline]
- Irvine AB, Gelatt VA, Seeley JR, Macfarlane P, Gau JM. Web-based intervention to promote physical activity by sedentary older adults: randomized controlled trial. J Med Internet Res 2013;15(2):e19 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Mackinnon AJ, Brittliffe K. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol Med 2006 Dec;36(12):1737-1746. [CrossRef] [Medline]
- Kajiyama B, Thompson LW, Eto-Iwase T, Yamashita M, Di Marioa J, Marian TY, et al. Exploring the effectiveness of an internet-based program for reducing caregiver distress using the iCare Stress Management e-Training Program. Aging Ment Health 2013;17(5):544-554 [FREE Full text] [CrossRef] [Medline]
- Santucci LC, McHugh RK, Elkins RM, Schechter B, Ross MS, Landa CE, et al. Pilot implementation of computerized cognitive behavioral therapy in a university health setting. Adm Policy Ment Health 2014 Jul;41(4):514-521. [CrossRef] [Medline]
- Moritz S, Schilling L, Hauschildt M, Schröder J, Treszl A. A randomized controlled trial of internet-based therapy in depression. Behav Res Ther 2012 Aug;50(7-8):513-521. [CrossRef] [Medline]
- Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, et al. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med 2014 Mar;44(4):741-752 [FREE Full text] [CrossRef] [Medline]
- Titov N, Andrews G, Schwencke G, Robinson E, Peters L, Spence J. Randomized controlled trial of Internet cognitive behavioural treatment for social phobia with and without motivational enhancement strategies. Aust N Z J Psychiatry 2010 Oct;44(10):938-945. [CrossRef] [Medline]
- Whitton AE, Proudfoot J, Clarke J, Birch M, Parker G, Manicavasagar V, et al. Breaking open the black box: isolating the most potent features of a web and mobile phone-based intervention for depression, anxiety, and stress. JMIR Ment Health 2015;2(1):e3 [FREE Full text] [CrossRef] [Medline]
- Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Sheehan J, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety: a randomized controlled trial. Behav Ther 2015 Mar;46(2):206-217. [CrossRef] [Medline]
- Knaevelsrud C, Maercker A. Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: a randomized controlled clinical trial. BMC Psychiatry 2007;7:13 [FREE Full text] [CrossRef] [Medline]
- Twomey C, O'Reilly G, Byrne M, Bury M, White A, Kissane S, et al. A randomized controlled trial of the computerized CBT programme, MoodGYM, for public mental health service users waiting for interventions. Br J Clin Psychol 2014 Nov;53(4):433-450. [CrossRef] [Medline]
- Seaborn K, Fels DI. Gamification in theory and action: a survey. Int J Hum-Comput St 2015 Feb;74:14-31. [CrossRef]
- Tech Trends 2012: Elevate IT for Digital Business. Sydney, Australia: Deloitte Australia; 2012. URL: http://www2.deloitte.com/content/dam/Deloitte/au/Documents/technology/deloitte-au-technology-tech-trends-2012.pdf [accessed 2016-08-18] [WebCite Cache]
- Bogost I. Gamasutra. 2011. Persuasive games: exploitationware URL: http://bogost.com/writing/exploitationware/ [accessed 2016-08-17] [WebCite Cache]
- Antin J, Churchill E. Badges in social media: a social psychological perspective. 2011 Presented at: ACM CHI Conference on Human Factors in Computing Systems; May 7-12, 2011; Vancouver, BC URL: http://gamification-research.org/wp-content/uploads/2011/04/03-Antin-Churchill.pdf
- Cobb NK, Poirier J. Effectiveness of a multimodal online well-being intervention: a randomized controlled trial. Am J Prev Med 2014 Jan;46(1):41-48 [FREE Full text] [CrossRef] [Medline]
- Sheeber LB, Seeley JR, Feil EG, Davis B, Sorensen E, Kosty DB, et al. Development and pilot evaluation of an Internet-facilitated cognitive-behavioral intervention for maternal depression. J Consult Clin Psychol 2012 Oct;80(5):739-749 [FREE Full text] [CrossRef] [Medline]
- Wangberg SC, Bergmo TS, Johnsen JK. Adherence in Internet-based interventions. Patient Prefer Adherence 2008;2:57-65 [FREE Full text] [Medline]
- Litz BT, Engel CC, Bryant RA, Papa A. A randomized, controlled proof-of-concept trial of an Internet-based, therapist-assisted self-management treatment for posttraumatic stress disorder. Am J Psychiatry 2007 Nov;164(11):1676-1683. [CrossRef] [Medline]
- Morrison C, Doherty G. Analyzing engagement in a web-based intervention platform through visualizing log-data. J Med Internet Res 2014;16(11):e252 [FREE Full text] [CrossRef] [Medline]
- Akobeng AK. Understanding randomised controlled trials. Arch Dis Child 2005 Aug;90(8):840-844 [FREE Full text] [CrossRef] [Medline]
- Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, et al. Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in the general population: randomized controlled trial. J Med Internet Res 2013;15(1):e2 [FREE Full text] [CrossRef] [Medline]
- Høifødt RS, Lillevoll KR, Griffiths KM, Wilsgaard T, Eisemann M, Waterloo K, et al. The clinical effectiveness of web-based cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled trial. J Med Internet Res 2013;15(8):e153 [FREE Full text] [CrossRef] [Medline]
- Watts S, Mackenzie A, Thomas C, Griskaitis A, Mewton L, Williams A, et al. CBT for depression: a pilot RCT comparing mobile phone vs computer. BMC Psychiatry 2013;13:49 [FREE Full text] [CrossRef] [Medline]
|CBT: cognitive behavioral therapy|
|CMDs: common mental disorders|
|iCBT: Internet cognitive behavioral therapy|
|RCT: randomized controlled trial|
|SMS: short message service|
Edited by J Torous; submitted 01.03.16; peer-reviewed by M Larsen, R Crutzen; comments to author 22.03.16; revised version received 06.06.16; accepted 11.07.16; published 24.08.16
©Menna Brown, Noelle O'Neill, Hugo van Woerden, Parisa Eslambolchilar, Matt Jones, Ann John. Originally published in JMIR Mental Health (http://mental.jmir.org), 24.08.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.