This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.
Childhood anxiety disorders are a prevalent mental health problem that can be treated effectively with cognitive behavioral therapy, in which exposure is a key component; however, access to treatment is poor. Mobile-based apps on smartphones or tablets may facilitate the delivery of evidence-based therapy for child anxiety, thereby overcoming the access and engagement barriers of traditional treatment. Apps that deliver therapeutic content via immersive gaming technology could offer an effective, highly engaging, and flexible treatment proposition.
In this paper, we aim to describe a preliminary multi-method evaluation of Lumi Nova, a mobile app intervention targeting mild to moderate anxiety problems in children aged 7-12 years using exposure therapy delivered via an immersive game. The primary objective is to evaluate the effectiveness, user engagement and experience, and safety of the beta version of Lumi Nova.
Lumi Nova was co-designed with children, parents, teachers, clinicians, game industry experts, and academic partnerships. In total, 120 community-based children with mild to moderate anxiety and their guardians were enrolled to participate in an 8-week pilot study. The outcome measures captured the app’s effectiveness (anxiety symptoms, child-identified goal-based outcomes, and functional impairment), user engagement (game play data and ease-of-use ratings), and safety (mood ratings and adverse events). The outcome measures before and after the intervention were available for 30 children (age: mean 9.8, SD 1.7 years; girls: 18/30, 60%; White: 24/30, 80%). Additional game play data were automatically generated for 67 children (age: mean 9.6, SD 1.53 years; girls: 35/67, 52%; White: 42/67, 63%). Postintervention open-response data from 53% (16/30) of guardians relating to the primary objectives were also examined.
Playing Lumi Nova was effective in reducing anxiety symptom severity over the 8-week period of game play (
This preliminary study provides initial evidence that an immersive mobile game app may safely benefit children experiencing mild to moderate anxiety. It also demonstrates the value of the rigorous evaluation of digital interventions during the development process to rapidly improve readiness for full market launch.
Anxiety disorders are among the most common and impairing mental health difficulties experienced in childhood and are characterized by excessive fear, worry, and negative beliefs that can result in distress and functional impairments in social, academic, and family life [
Substantial clinical evidence suggests that anxiety in children can be effectively treated using psychological approaches [
Importantly, although expert consensus and dismantling studies indicate that exposure-based elements of CBT are active components that are effective in treating anxiety disorders, exposure-based CBT is infrequently included in interventions for children [
Digital mental health interventions (including web-based or computer-based programs) that draw on CBT-based techniques are effective in reducing anxiety symptom severity in children and young people [
Outcomes may be optimized for children when the capabilities of mobile technologies (smartphones and tablets) are fully leveraged. Many children are comfortable and familiar with processing information and engaging with content via mobile devices. The levels of digital independence for children are increasing, with around 50% of those aged 8-11 years using a smartphone and 72% using a tablet [
The application of game design elements is heralded as a strategy to increase engagement and adherence with mHealth interventions, offering an intrinsically motivating option for therapeutic delivery for children, particularly where content supports user preferences for being interactive, personable, and relatable [
This preliminary study evaluates the effectiveness, user engagement and experience, and safety of a novel app for smartphones and tablets (Lumi Nova), which uses immersive gaming technology to deliver exposure therapy for children aged 7-12 years with mild to moderate anxiety difficulties. Specifically, the primary objective is to evaluate the following: (1) whether exposure therapy delivered via Lumi Nova is associated with a reduction in guardian-reported anxiety symptoms and functional impairment in children and progression toward the child-identified goals related to anxiety; (2) user engagement, ease of use, and experience of Lumi Nova; and (3) whether Lumi Nova is safe to use (ie, is not associated with harm or unintended negative consequences). Our expectation is that playing Lumi Nova would be associated with lower anxiety symptom severity and interference after the intervention and positive progression toward treatment goals. No further hypotheses have been offered for this exploratory study.
Multiple quantitative and qualitative methods were used. A pre-post design was used to compare the guardian-rated outcome measures captured via survey before (T1) and after the intervention (T2). In addition, game play data were collected over the course of the intervention, and player ratings and guardian open survey responses were collected after the intervention (T2). Data were collected during a 10-week intervention period between January and March 2020 with game play data generated over approximately 8 weeks of play. The study was approved by the Faculty of Medicine and Health Sciences Research Ethics Committee, University of Nottingham (Reference: 452-1911; December 19, 2019).
A total of 120 English-speaking children aged 7-12 years and their guardians completed T1 anxiety measures. Children were identified by school-based staff in 12 participating schools as experiencing difficulties with anxiety and not concurrently receiving psychological treatment. The participating schools were 9 primary schools and 3 secondary schools in the South East England identified through a partnership with the local council Personal, Social, Health and Economic education curriculum and Healthy School Lead and supported by a Children and Young People’s Mental Health and Wellbeing Steering Group. The mean eligibility for free school meals across these schools (a proxy for socioeconomic status) was 18.1% (SD 7.3%), indicating that the school sample from which children were drawn was broadly representative of nationally reported proportions (15.8%) across all primary school types (Department for Education, January 2019). Most children (88/120, 73.3%) had not sought or received previous treatment for anxiety before starting the pilot intervention through the Children and Adolescent Mental Health Service (CAMHS), a general practitioner or nurse (92/120, 76.7%), or a psychologist or counselor (94/120, 78.3%).
Of the 120 participants with complete anxiety-related outcome measures at T1, follow-up measures at T2 were available for 30 (25%) children aged 6-13 years (mean 9.8, SD 1.7 years); 2 (1.7%) children were marginally outside the target age range of 7-12 years (aged 6.97 and 13.0 years) at the point of entering the study and were retained in the analysis. Of the 120 guardians from the T1 sample, 95 (79.2%) completed an additional anxiety measure survey following an automated SMS text message prompt to be provided with a game key, and 74 (61.7%) guardians activated the game key and downloaded Lumi Nova. Subsequent game play data were recorded for 67 (71%) out of 95 participants. Among the 30 participants with complete T1-T2 anxiety-related outcome measures, game play data were recorded for 25 (83%). Details of the study recruitment and attrition are shown in
Overview of the recruitment and study process.
Analyses were conducted on the two subsamples for whom there was complete data: the T1-T2 complete outcome measure subsample (n=30) and the game play analytics subsample (n=67). The demographic characteristics and outcome variables of these samples are presented in
Demographic data and clinical characteristics for study subsamples.
Demographic details | T1-T2 subsample (n=30) | Game play subsample (n=67) | ||
Agea (years), mean (SD) | 9.81 (1.70) | 9.6 (1.53) | ||
|
||||
|
Male | 12 (40) | 31 (46) | |
|
Female | 18 (60) | 35 (52) | |
|
||||
|
Yes | 10 (33) | 17 (25) | |
|
||||
|
No | 29 (97) | 64 (96) | |
|
||||
|
Asian or Asian British | 1 (3) | 2 (3) | |
|
Black or African or Caribbean or Black British | 3 (10) | 8 (12) | |
|
Mixed or multiple ethnicities | 1 (3) | 4 (6) | |
|
Other ethnic groups | 1 (3) | 1 (1) | |
|
White | 24 (80) | 42 (63) | |
|
||||
|
English | 30 (100) | 57 (85) | |
|
||||
|
|
|||
|
|
No | 25 (83) | 53 (79) |
|
|
Yes | 2 (7) | 3 (4) |
|
|
Do not know | 1 (3) | 1 (1) |
|
|
|||
|
|
No | 23 (77) | 46 (69) |
|
|
Yes | 6 (20) | 10 (15) |
|
|
Do not know | 1 (3) | 1 (1) |
|
|
|||
|
|
No | 25 (83) | 48 (72) |
|
|
Yes | 5 (16) | 8 (12) |
|
|
Do not know | 0 (0) | 1 (1) |
|
||||
|
SCAS-P-8e | 8.33 (4.56) | 7.83 (3.71) | |
|
RCADS-Pf,g (total anxiety) | 30.30 (16.92) | 28.97 (14.45) | |
|
CAIS-Ph (total) | 20.57 (15.40) | 18.39 (13.25) | |
|
||||
|
At clinical cutoff | 8 (40) | 13 (23) | |
|
At borderline cutoff | 1 (5) | 4 (7) | |
|
Within normal range | 11 (55) | 39 (70) |
aGame play subsample age was based on 59 responses.
bTreatment history was based on the previous 3 months.
cCAMHS: Children and Adolescent Mental Health Service.
dGP: general practitioner.
eSCAS-P-8: Spence Child Anxiety Scale–Parent version.
fRCADS-P: Revised Child Anxiety and Depression Scale–Parent version.
gClinical characteristics were based on 58 responses for the Revised Child Anxiety and Depression Scale–Parent version.
hCAIS-P: Child Anxiety Impact Scale–Parent version.
iClinical thresholds describe the top 2% of scores of unreferred children of the same age and the top 7% for borderline clinical threshold.
jClinical cutoffs were based on 56 participants who met the age range for standardized Revised Child Anxiety and Depression Scale–Parent version
kFor the T1-T2 subsample, the clinical cutoffs were based on 20 participants who met the age range for standardized Revised Child Anxiety and Depression Scale–Parent version
Lumi Nova combines evidence-based therapeutic content (exposure therapy) and psychoeducational content within an immersive game designed to provide timely support to children aged 7-12 years, who are facing difficulties with anxiety. The app uses a diverse range of techniques, including storytelling, photographs, videos, 360° videos, and game mechanics with a progressive narrative, rewards, customization of avatars, and unlocking new levels to deliver an immersive experience to users. The development and design of Lumi Nova resulted from a robust coproduced and collaborative user-centered design process that involved children, parents, teachers, clinical practitioners, academics, and game industry experts to build the game concept, design, and clinical model parameters. In the initial phase of development, the aim was to develop a prototype game that delivered exposure therapy in a way that would be engaging, effective, and viable for children. The development phase involved multiple and multi-school site cocreation and user-testing sessions, and early prototype testing sessions with key stakeholders over a period of 5 months.
The game narrative is an intergalactic role-playing adventure in which players assume the role of a treasure hunter on a quest to save the galaxy and explore the universe, helping characters on various planets while training to overcome real-world fears (
Example screenshots from Lumi Nova game play.
The mechanics of the intervention facilitates players to set anxiety-related goals and build a graded ladder of exposure steps (challenges) and to undertake these steps recording their before, after, and future exposure reflection in response to clinical psychologist determined prompts (eg, “What do you think might happen during this challenge?” “How worried did you feel during the challenge?” “How worried would you feel if you have to do it again?”). This approach is underpinned by strategies for optimizing learning during exposure, based on inhibitory learning perspectives. Negative expectancies associated with a perceived aversive outcome are countered by emphasizing the mismatch between what is expected to occur and what actually occurs [
Example screenshots of VitaMind Hub. Progress data are accessible to authorized professionals to facilitate active remote monitoring and care decisions.
Guardian-completed survey items captured demographic information (age, gender, ethnicity, primary language spoken at home, and eligibility for free school meals) and clinical history (such as previous treatment for anxiety, contact with CAMHS, or a general practitioner or nurse because of anxiety in the previous 3 months) for their child.
The parent-rated brief SCAS-P-8 [
The RCADS-P [
The CAIS-P [
The GBO tool [
Anonymized game play data, automatically generated during game play and uploaded to the hub when connected to Wi-Fi, captured game play information, for example, the frequency (total number) of play sessions per player, and duration of play (number of days playing). One question (“How easy is Lumi Nova to play?”) was adapted from the Program Content and Usability questionnaire [
The safety of Lumi Nova was assessed using three indices: (1) change in the major depressive disorder subscale of the RCADS-P (see
Optional open-response questions for guardians (within the guardian-rated survey at T2) solicited thoughts about the following: (1) guardian perceived changes (positive or negative) associated with playing Lumi Nova, (2) general comment regarding accessing or playing the game, and (3) additional comments. Responses pertinent to the study objectives, that is, those describing (1) effectiveness, (2) user engagement and experience, and (3) safety were summarized.
All guardians provided informed consent, and the children provided verbal assent before participating in the study. The guardians were asked to complete the demographic and anxiety outcome questionnaires (SCAS-P-8, RCADS-P, and CAIS-P) at T1 using a web-based survey platform. Subsequently, authorized school staff with access to the VitaMind Hub set up child profiles, which automatically triggered an SMS text message to their guardians with access to Lumi Nova via a game key. Participating families were asked to encourage their children to play Lumi Nova multiple times a week over the course of 8 weeks. At the end of the intervention (T2), guardians were asked to complete the anxiety outcome questionnaires (SCAS-P-8, RCADS-P, and CAIS-P).
Analyses were performed using SPSS (version 26; IBM Corp). Descriptive statistics were used to summarize sample data; 2-tailed paired sample
Mean scores relating to symptom severity and interference before (T1) and after intervention (T2) are reported in
Mean change in primary outcome measures for the T1-T2 sample.
Measure | T1, mean (SD) | T2, mean (SD) | ||
|
||||
|
SCAS-P-8a,b (total) | 8.33 (4.56) | 7.43 (3.28) | .009 |
|
RCADS-Pc (total anxiety) | 30.73 (13.94) | 30.30 (16.92) | .20 |
|
||||
|
CAIS-Pd,e (total) | 20.57 (15.40) | 20.97 (15.49) | .80 |
|
||||
|
RCADS-P (MDDf) | 7.07 (4.91) | 6.60 (3.94) | .46 |
aSCAS-P-8: Spence Child Anxiety Scale–Parent version.
bOnly 1 variable (Spence Child Anxiety Scale–Parent version total) was associated with a statistically significant finding (
cRCADS-P: Revised Child Anxiety and Depression Scale–Parent version.
dCAIS-P: Child Anxiety Impact Scale–Parent version.
eSignificance testing was based on Wilcoxon signed-rank tests for the Child Anxiety Impact Scale–Parent version home and social subscales; otherwise, significance was based on paired sample
fMDD: major depressive disorder.
Comparison of the first and last ever child-rated GBO in relation to an active goal established if playing Lumi Nova was associated with therapy-aligned improvement as determined by users. In total, 54 (81%) of the 67 players with game play data selected a goal and subsequently recorded a GBO score for exposure challenges. Out-of-game exposure challenges associated with that goal were recorded for 43 (64%) of the 67 players with game play data, and 45 (67%) players rated their progress by completing at least two GBO scores. A Wilcoxon signed-rank test showed that there was a significant difference between the first and last outcome score over the course of the intervention (
Of the 30 guardians who completed the follow-up survey at T2, 16 (53%) provided optional open-response comments. The responses were collated and systematized in relation to the primary study objectives: effectiveness, user engagement and experience, and safety (
Guardian open-response content summarized by research domain (n=16).
Research domain and summarized content | Comments, n (%) | ||
|
|||
|
Increased confidence and bravery to tackle challenges | 6 (38) | |
|
Increased appreciation that taking small steps is helpful | 3 (19) | |
|
Perceived progression in relation to goal choice | 2 (13) | |
|
Facilitated discussion about anxiety | 1 (6) | |
|
Beneficial in conjunction with other support | 1 (6) | |
|
|||
|
Neutral endorsement of use | 5 (31) | |
|
Laudatory comments | 4 (25) | |
|
Barriers to adoption (design and process) | 6 (38) | |
|
Barriers to adoption (technical barriers) | 2 (13) | |
|
Increased frustration | 1 (6) | |
|
|||
|
Adverse outcomes | 0 (0) |
Regarding effectiveness, comments in this domain were all related to positive improvements in anxiety-related outcomes; 6 (20%) of the 30 guardians described witnessing an increase in confidence or bravery in their child and suggested that children were able to recognize fears and successfully challenge their thoughts:
When she did the challenge, getting an answer wrong, that gave her a bit of confidence that [a] little mistake doesn’t put one in trouble by teachers.
For one child, playing Lumi Nova prompted greater discussion around fears and worries. The child’s guardian said, “He seems more willing to talk about feeling anxious, he asks questions about anxiety” [guardian of a boy, aged 9 years]. Guardians felt that Lumi Nova had generated new learning in line with core processes of exposure therapy about what happens when an anxiety-provoking situation occurs and was effective in helping children work through a step-by step approach:
She liked knowing that she could take small steps towards a recognised fear and liked remembering that she coped with all those steps comfortably.
He took to the game very well and I think it helped him rationalise one of his fears – staying away from home...I definitely think the game put in some excellent groundwork for him to draw on going forward.
In one case, a guardian reported that the game had proved effective in conjunction with existing support: “This, along with weekly play therapy, has helped her anxiety” [guardian of a girl, aged 8 years].
Average frequency and duration of game play.
|
Game play sample (n=67) | T1-T2 sample (n=25) | |||
|
|||||
|
Value, mean (SD) | 11.22 (9.41) | 12.16 (10.45) | ||
|
Value, median (range) | 8 (1-46) | 8 (1-46) | ||
|
|||||
|
Value, mean (SD) | 18.37 (14.75) | 18.28 (14.60) | ||
|
Value, median (range) | 15 (1-53) | 16 (1-53) |
aDuration of play from the first recorded date to the last date of game play per participant.
In total, 10 (15%) of the 67 players with game play data rated how easy they found playing Lumi Nova on a scale from 1 (very easy) to 5 (very hard); 8 (12%) players provided a positive or neutral evaluation, with most (6/8, 75%) finding the game easy or very easy, and the rest (2/8, 25%) finding the game neither easy nor hard. Furthermore, 3% (2/67) of players reported finding the game very hard to play.
In total, 18 open-response comments related to player engagement and experience of using Lumi Nova in the T2 guardian survey (
My daughter lost interest in the game and thought it was more aimed at younger children. She has specific worries that weren’t covered.
The feelings bit at the beginning was good, but the tasks following this could be repetitive.
Two guardians commented specifically on technical difficulties (subsequently redressed), which affected the player experience (eg, difficulties downloading the game or saving progress). One guardian simply reported that playing Lumi Nova made her daughter (aged 10 years)
Playing Lumi Nova was not associated with increased symptoms of low mood over the course of the intervention, that is, the mean RCADS-P major depressive disorder scores did not increase from T1 to T2 (
This small-scale preliminary evaluation study examined the effectiveness, user engagement and experience, and safety of Lumi Nova, a mobile app delivering targeted exposure-based CBT strategies for children with mild to moderate difficulties with anxiety. Over an 8-week period of game play, we found that playing Lumi Nova was associated with a reduction in anxiety symptom severity and progress toward treatment goals, and this effectiveness was positively endorsed by guardians. The children engaged with the content and did so safely.
Regarding the app’s effectiveness, there was a reduction in the guardian-rated mean anxiety symptom severity (SCAS-P-8) between T1 and T2 with a small to moderate effect. Such findings are consistent with the literature showing moderate effectiveness in computer-based CBT for childhood anxiety [
For player-rated effectiveness, children recorded positive movement toward achieving a self-identified therapy-aligned goal (ie, GBO) over the course of the intervention, on average moving up 2 points toward achieving their goal. Clinically, involving children in the setting and tracking of therapeutic goals provides an essential element of agency and personal activation, which may improve treatment outcomes [
In terms of user engagement and experience, evidence was provided from game play data capturing the quantity of play (frequency and duration of sessions) to indicate game adoption and repeated use over the intervention period. On average, children played Lumi Nova 11 times (SD 9.41) over 18 days (SD 14.75). However, these engagement metrics varied considerably among the players. In addition, data were not reported on the duration of each session of game play, which would help establish that the sessions involved meaningful interaction. In addition to objective (game play) markers, there was also modest support from the limited data that children found the game easy to use. Open-response comments reinforced that children played the game on multiple occasions, sometimes with parents, over many weeks and appeared to enjoy doing so.
It is interesting to note that there is little shared understanding or agreement of what constitutes sufficient engagement for mHealth apps [
Gamification is seen as a strategy to increase engagement and adherence with digital mental health interventions by delivering therapeutic content in a format with intrinsic appeal for children [
Relatively few apps for anxiety in childhood implemented in
Lumi Nova was developed using a robust co-design framework that involved children, parents, teachers, clinicians, academics, and technical experts in prototype design, development, and evaluation via rapid user-testing. This is a strength of the app and in line with guidance, which has called for increased co-design processes that actively engage the intended users and other stakeholders throughout the development cycle of digital game-based innovations for mental health [
Although children adopted and engaged with Lumi Nova, and the game play sample was sufficient to demonstrate its use, the evidence of at least one or more sessions of game play was available for only around half of those consenting to play at T1. Analytic information about game play sessions was captured for analysis only when the player’s device had internet connectivity, enabling data to be sent to the data hub, which was not always achieved consistently every week, as directed. Therefore, it is possible that our data underrepresent true player interest and the adoption of the game (ie, game play occurred offline). The drop from those with preintervention consent (n=120) to those with guardians activating an access key (n=74) may have resulted from technical difficulties that guardians faced in downloading the beta version of the game as well as the additional requirement on guardians to complete the SCAS-P-8 to generate the game key. Therefore, poorer uptake may index the study burden on guardians rather than the game’s appeal among players. It would be interesting to analyze adoption and use in a natural (nonstudy) setting. Close partnerships working with teachers and guardians, including practical support with processes of enrollment in the study and game setup, were provided to maximize engagement in the study; nonetheless, guardian retention was a challenge, and this was consistent with other evaluation studies in digital mental health [
Further work to establish the maintenance of treatment gains over the short and long term would be an important next step in establishing the effectiveness of Lumi Nova. A study powered to explore potential moderators of effectiveness, engagement, and experience (eg, age, gender, anxiety presentation, additional comorbidities, and disability) would also help clarify who is likely to benefit from playing Lumi Nova and in what circumstances. Contextual factors associated with home-based engagement, such as the level of parental involvement, could be explored [
App-based treatment platforms that deliver therapeutic content via gaming technology may provide an opportunity to offer effective early intervention for childhood anxiety disorders and address documented barriers to successful treatment by delivering an appealing and acceptable option for children experiencing difficulties with anxiety that can be accessed within a home environment. This small-scale evaluation study provides early evidence in support of the effectiveness, safety, and acceptability (user engagement and experience) of Lumi Nova, a coproduced and collaboratively developed self-help app delivering exposure-based CBT strategies via immersive technology. Further evaluation is recommended to support and extend these preliminary findings.
Child Anxiety Impact Scale–Parent version
Children and Adolescent Mental Health Service
cognitive behavioral therapy
goal-based outcome
mobile health
Revised Child Anxiety and Depression Scale–Parent version
Spence Child Anxiety Scale–Parent version
The authors thank all the study participants who contributed to this research. JL, LW, and JM acknowledge the financial support of the National Institute of Health Research Nottingham Biomedical Research Centre and National Institute of Health Research MindTech Med Tech Co-operative.
This paper details a preliminary independent evaluation study completed by MindTech Med Tech Co-operative as part of a collaborative project with BfB Labs Ltd (the creator of Lumi Nova) and the University of Reading. The development of Lumi Nova was funded through a small business research development contract awarded to BfB Labs Ltd by National Health Service England.