Background: Men have different mental health needs as compared with women, and women make up the primary audience of most digital mental health interventions. An Australian football-themed (specifically Australian Football League, AFL) app named MindMax incorporating psychoeducation, gamification, mini-games, and social connection was developed in an effort to address this issue.
Objective: The aim of this study was to identify the best way to structure and present MindMax, an app that aims to deliver psychoeducational modules, and create a Web-based community centering on well-being, AFL, and video games for men aged 16 to 35 years who are interested in AFL or video games.
Methods: We conducted 6 participatory design (PD) workshops with people aged 16 to 35 years in 3 cities in Australia, to identify the best way to present MindMax, and contracted a digital development agency to develop MindMax. We then iteratively tested MindMax prototypes with 15 user experience testing interviews across 3 separate time points: 2 before app launch and 1 after app launch.
Results: A total of 40 individuals (25 male and 15 female) participated in the PD workshops, and a total of 15 individuals (10 male and 5 female) participated in user experience interviews. Broadly, participants expressed a preference for activities requiring active engagement that practiced useful skills. They were also sensitive to how content was presented and wanted the ability to customize their own app experience. Although participants agreed that social motivations were important for engagement with an app, they recommended not to mimic existing social networks.
Conclusions: In basing itself strongly within the AFL subculture and by incorporating gamification as well as mini-games, MindMax aimed to tackle mental health help-seeking barriers for people who enjoy AFL or video games, with a particular emphasis on men, and to provide psychoeducation on strategies to increase mental health and well-being. If MindMax is successful, this would indicate that generalizing this approach to other traditional sporting codes and even competitive video gaming leagues (esports) would be fruitful.
Men’s Well-Being and Internet Interventions
As participants in mental health research are heavily biased toward being female [, ], research outcomes may not be fully generalizable to men. A growing body of evidence suggests that men’s experiences of mental health problems and treatment differ to those of their female counterparts [ , ]. For example, young men have higher rates of suicide prevalence and lower rates of mental health literacy and health care service access than young women [ , ]. Furthermore, women are more receptive to structured internet health interventions than men [ ]. This problem is especially urgent for young people as most mental health problems are developed during young adulthood [ ]. Furthermore, mental health outcomes at a young age persist and potentially worsen in the long term, even among those receiving clinical care [ ]. A targeted approach to improving mental health that aims to give younger people the tools to manage their own well-being would be helpful in addressing this issue.
Western norms of masculinity (in particular, the emphasis on self-reliance and on silently coping with psychological distress) act as barriers to help seeking  and contribute to the worse outcomes displayed by young men. This is exacerbated by the small but significantly higher tendency for young men to avoid addressing their friends’ mental health problems directly and to avoid recommending they seek help from external sources, relative to their female peers [ ]. Furthermore, the high levels of mental health stigma that persist in young men and their negative perceptions of mental health professionals work in tandem with the previous to contribute to high reluctance to formally seek help from mental health professionals [ ]. Instead, young men tend to seek informal help from the internet, with 1 study on 16- to 24-year-old men finding nearly 55% of their sample reported having done so [ ]. The same study further reports that, within their sample, younger men were more likely to seek informal help from the internet than older men.
An approach to internet interventions that enables and informs such informal help seeking would be a natural fit to this pattern of behavior. An evidence-based approach toward men’s mental health and well-being that addresses the problems outlined above should therefore be (1) accessible on the internet, (2) action-based and informal (not clinical), (3) anonymous with the potential for social connection, (4) self-directed, and (5) based in subcultures men are already present in . Importantly, women should not be excluded as in addition to benefiting from the intervention themselves, they can also act as supportive others, connecting these interventions and other mental health initiatives to the men in their lives.
Sports and video games are mainstream topics with significant male fan bases. The sporting code that is the focus of this study, Australian Football League (AFL), is a type of Australian football and enjoys the support of 6 million people across Australia . Similarly, a recent nationwide survey reports that over three-quarters of Australians aged 15 to 34 years play video games and that 70% of all men surveyed were video game players [ ]. Both cultures are further combined in esports (competitive video gaming) [ ]. esports is a growing industry popular among younger people, with nearly as many millennials preferring watching their favorite esport to watching their favorite traditional sport (40% vs 42%) [ ]. Furthermore, many sporting leagues including AFL have partnered with esports teams [ ] and video game companies to host esports events [ ]. Although they remain distinct subcultures, sports and video games are highly compatible, mainstream in the general population, and well suited to utilization in a mobile health (mHealth) app intended to promote mental health and well-being.
Gamification and Applied Games
The general usage of games and game features for nonentertainment purposes is known as “applied games” . This includes not just applying commercial video games outside of an entertainment context, for example, psychological therapy [ ], but also serious games (video games developed for a primary purpose other than player enjoyment [ ]) and gamification [ , ]. In the same way that traditional video game design works to engage both the extrinsic and intrinsic motivation of its players, applied games have inherent “effectiveness potential” [ ], where users of interventions that incorporate applied games can be motivated to explore the intervention deeper for additional motivations besides self-improvement. Furthermore, the inherent design characteristics of video games have been shown to be complementary to subjective well-being concepts, for example, the Seligman positive emotions, engagement, relationships, meaning, and achievement (PERMA) model [ , ]. There is also evidence for a positive impact of moderate video game play on well-being [ - ].
Gamification, in particular, has been named a promising strategy with which to promote engagement in digital health interventions [, , ]. Although the most well-known gamification elements are points, badges, and leaderboards [ ], prioritizing these elements can undermine the complex series of cognitive, emotional, and social affordances that make games intrinsically motivating and enjoyable to play [ ]. The most successful mHealth initiatives that incorporate gamification have been carefully designed to include both extrinsic and intrinsic motivators [ , ].
The term gamification has been defined as the “use of game design elements in non-game contexts”  as well as “a process of enhancing a service with affordances for gameful experiences in order to support user’s overall value creation” [ ]. The latter definition by Huotari and Hamari is particularly useful in an mHealth (both mental and physical health) context, with “value creation” potentially being the improvement of the user’s health; the adoption of health behaviors; the provision of a fun, engaging educational experience; or all of the above. This definition also emphasizes the goal of gamification rather than its methods and recognizes that what some individuals may term “game design elements” may not be considered as such by others, complementing the overlap between gamification and other health behavioral change frameworks such as persuasive systems design [ ]. Finally, this definition is drawn from a service marketing approach. By viewing the digital health intervention as a core service, it becomes easier to visualize how the components of this service can be enhanced with motivational affordances. Approaching applied games (including gamification) from this perspective could thus lead to a more compatible and natural integration of applied games (including gamification) into mental health care.
Although empirical study of the effects of gamification is still in its infancy, there is evidence, albeit outside of health, that it leads to higher and more involved user engagement with an app or service [, ]. However, the impacts of gamification within electronic health and mHealth remain poorly understood [ - ]. Although application varies by health domain, many mHealth apps do not utilize gamification [ ], and those that do tend to contain limited applications of it [ ]. As evidence of individual differences in gamification element preferences is emerging [ ], it is clear that to provide the enjoyable and engaging experiences initially hoped for, when gamification (and applied games in general) is applied to mHealth, it must be with due consideration.
Participatory Design and Knowledge Translation
Although researcher-led mHealth initiatives have a key strength in applying evidence-based best practice, it is often at the expense of user experience. It is difficult to compete with large corporations who invest millions of dollars into creating seamless, intuitive, and engaging user experiences to entertain their consumers. This level of investment is near impossible in academia, which may result in a jarring experience for users accustomed to a contemporary internet experience . Another key tension within mHealth initiatives is that their aims and objectives often act as barriers to uptake, especially among populations that engage the most in behaviors the intervention hopes to reduce (eg, drinking alcohol [ ]). It is important to identify how best to present the health and therapeutic content of mHealth initiatives to the target audience. One method of achieving this is through participatory design (PD) [ ].
The key principle of PD is to involve all stakeholders of a project in an iterative cycle of design and development [, ]. This allows them to influence its design to better suit their past, present, and future needs, ideally leading to higher effectiveness and engagement among the target population [ ]. Furthermore, when executed well, PD methodologies increase the acceptability of interventions to stakeholders [ ] and can be harnessed to make knowledge translation of research outcomes more efficient [ , ]. This is especially important in mHealth, as given the rapid pace of technological development, the field must reduce the lag between health research and translation as much as possible.
Study Context and Objectives
As part of its daily operations, the Australian Football League Players’ Association (AFLPA) offers mental health and well-being training to more than 800 players across the National League . This training focuses primarily on resilience and well-being. Well-being is a separate construct of positive mental health that is distinct to mental illness [ ]. A focus on well-being is more broadly applicable to the general population as both people with and without mental illness can directly benefit from learning how to maintain and improve their well-being. Notably, increasing subjective well-being leads to improvements in individuals’ lives, such as healthier relationships, more positive emotions, increased feelings of autonomy, and increased self-acceptance [ ].
In collaboration with Queensland University of Technology and The University of Sydney’s Brain and Mind Centre, the AFLPA obtained funding to execute a multipronged initiative to improve mental health and well-being, focusing on men aged 16 to 35 years but not excluding other groups of people . The app resulting from this collaboration was named MindMax and aimed to deliver a modified version of the AFLPA’s existing mental health programs in a portable, digital format. The target audience is hence men aged 16 to 35 years who are interested in AFL or video games.
MindMax was designed according to the 5 recommendations made by previous research  outlined in the first section of the Introduction. For example, educational content was split into multiple small modules lasting around 10 min each, to enable self-directed learning and to give users a choice in what aspects of their well-being they wish to focus on [ ]. A secondary aim of MindMax was to create a Web-based community centering on well-being, sports (in this case AFL), and video games. To achieve this aim, the AFLPA engaged a select number of AFL players as ambassadors for the app. Their role would include being spokesperson within the app modules and community area as well as promoting the app to the AFL industry and general public.
Although the basic components of MindMax were drawn from the literature and decided on by researchers and the AFLPA, it was unclear how best to present them in a way that would be acceptable to the target audience. The aim of this study was, therefore, to use PD and user testing methodologies on multiple iterations of MindMax to obtain key insights from end users on how best to present its content, design, and features.
Our recruitment strategies consisted of putting up posters, distributing postcards, and advertising in student mailing lists. We also asked affiliated organizations to assist in recruitment for locations not in Sydney. We reimbursed PD workshop participants with a gift voucher worth Aus $50 and user experience interview participants with a gift voucher worth Aus $30 to thank them for volunteering their time and expertise. The University of Sydney’s Human Research Ethics Committee (Protocol No. 2016/652) approved this project before the start of research activity.
Phase 1: Participatory Design Workshops
The PD methodologies used in this study are based on recommendations by the Young and Well Cooperative Research Centre . Specifically, we adapted the iterative PD and knowledge translation methodology used by Ospina-Pinillos et al [ ] to fit the needs of our project.
In phase 1, we held 2 PD workshops in each of 3 Australian capital cities in early September 2016, making 6 workshops in total. The aim of these workshops was to identify how best to frame the well-being concepts discussed in MindMax and, more broadly, how to structure a mental health and well-being app to the intended audience. Moderators took notes during the workshops. Workshops lasted 3 hours and consisted of 3 stages: discovery, evaluation, and prototype.
Workshop moderators facilitated participant discussion of their knowledge and usage of and preferences for, health and well-being apps/websites. Specifically, participants discussed their preferences for app design and content, their social usage of health and well-being apps, applying gaming concepts to mental health and well-being, and data tracking and privacy. Although moderators focused on mental health in particular, both mental and physical health were discussed.
Moderators then presented screenshots of existing health and well-being apps/websites to participants for their critical evaluation. These apps were a combination of popular commercial health and well-being apps (including physical health) as well as output of previous academic and government mental health and well-being mHealth projects. Screenshots portrayed a variety of features of interest, including social connection (dashboard and community pages), gameful elements (mini-games, challenges, and progress bars), and psychoeducation. Marker pens were provided for participants to annotate the screenshots.
Finally, in the context of the previous 2 stages of discussion, moderators asked participants to design concepts, specifications, or potential user journeys for a mental health and well-being app. Sketchbooks and marker pens were provided for this activity.
Phase 2: Knowledge Translation
Following the PD workshops, all moderator notes and participant artifacts (produced during the evaluation and prototype stages) were collated and analyzed by an independent knowledge translation team consisting of a group of young people (aged under 25 years) who were short-term interns at The University of Sydney’s Brain and Mind Centre. The team adopted an approach similar to descriptive content analysis , manually coding the notes and artifacts by 3 overarching semantic themes: content (the information and activities within the app), design (the visual design of the app), and features/concept (the conceptual design and features of the app). The team then used these themes and codes as guidelines to produce a knowledge-translated design of MindMax (see ).
Phase 3: User Experience Testing Interviews
Timeline and App Details
We presented the outcomes of the PD workshops and the resulting knowledge translation to the AFLPA, who concurrently contracted a digital agency (Long Division Digital, Melbourne) to produce a prototype of MindMax, drawing principles from the outcomes as appropriate and feasible. This prototype and further iterations were tested in one-on-one user experience interviews at The University of Sydney’s Brain and Mind Centre across 3 time points: December 2016, March 2017, and November 2017. MindMax was launched to the public in June 2017. Hence, 2 time points were before launch and 1 time point was after launch. Drawing on previous recommendations in user experience research , we aimed for a total of 15 participants (5 participants per round) to allow for as many insights to be captured across multiple iterations of MindMax, as efficiently as possible.
During the first time point, we tested a hybrid Web-based prototype with limited functionality, and the moderator assisted participants in accessing the app build through a mobile phone internet browser. At this point, only the Fit Minds psychoeducational module was available. During the second time point, we tested an in-progress native build with greater functionality (see). At this point, 3 psychoeducational modules were available: Fit Minds, Values, and Thoughts. During the third time point, which was 5 months after MindMax was launched on the App Store and Google Play Stores, we tested an updated version of MindMax (see ) and asked participants to download it onto their mobile phones. At this point, 5 psychoeducational modules were available: Fit Minds, Values, Thoughts, Mindfulness, and Emotions, as well as “Flick Footy,” a casual game involving kicking a football to score goals.
Along with the social component (community feed), the psychoeducational modules and Flick Footy formed the reward system within MindMax, where completing psychoeducational modules and posting in the community feed earned users points, called “footies,” which could then be spent to play Flick Footy. In addition to this, modules also contained mini-games (see) that aimed to allow users to interact with the lessons in a more active way.
Psychoeducation modules were accessible through a tab named “Train/Training” (depending on the version of MindMax) and were designed to last around 10 min. They consisted of information pages, interactive activities, and videos that contained information about the module topic and featured AFL players as spokespersons. The videos with AFL players were a montage of informal interviews relating to the module topic and were presented as a way to get to know another side of the player. During certain points of each module, shareable content (a “shareable”) was generated and posted to the community feed. These posts could be toggled to display to all MindMax users (public) or to the user only (private).
Fit Minds was an introductory module with the objective of creating a “MaxFive” plan to improve well-being, Values aimed to help users identify their values and ways to act upon them, Thoughts aimed to help users identify and deal with unhelpful thoughts (including a mini-game illustrating the concept of letting unhelpful thoughts pass by; see), Mindfulness aimed to introduce users to mindfulness meditation, and Emotions aimed to help users gain a better understanding of their emotions and how to deal with negative emotions. The Mindfulness and Emotions modules contained 2 guided audio meditation tracks recorded with 2 different AFL player ambassadors, and users could choose which player to meditate with. Information was presented in a casual, masculine tone to reflect the target audience of the app (men aged 16 to 35 years interested in AFL or video games).
Finally, although this study details the iterative design and development process of MindMax, a more in-depth explanation of the theory behind MindMax, including its applied games components, has been previously published . As per the recommendations of Tondello et al [ ], MindMax was also designed to incorporate multiple types of applied games to appeal to a wide variety of users.
User experience testing interviews followed a semistructured format. We first collected demographic information, specifically gender, age, mobile phone model, and operating system, and how many hours per week participants spent playing video games and watching AFL matches. We then explained to participants that interviews would be conducted using a think-aloud protocol, where participants verbalize their thoughts while completing a series of predetermined tasks. Although there are weaknesses with think-aloud protocols, such as their reliance on participant subjectivity and their inability to capture subconscious cognitive processes, concurrent think-aloud protocols nonetheless have the ability to capture crucial insights at a low cost . The predetermined tasks included procedures such as registering an account, creating an avatar, playing a casual game, and completing psychoeducation modules. The predetermined task list spanned all contemporarily available app features and was hence updated for each subsequent time point.
Participants were allowed to complete the tasks in their preferred order. They were given flexibility as to which modules they chose but were on occasion directed to complete specific modules to ensure an even spread of feedback. All participants completed at least two modules. Although participants were given the opportunity to ask questions during the interview, they were encouraged to complete the tasks unprompted and to the best of their ability. A researcher typed notes of the process.
Due to the convenience sampling strategy employed while recruiting for PD workshops (phase 1), participants naturally tended toward video game design students in Brisbane, AFLPA-affiliated individuals in Melbourne, and mental health and technology academics and students in Sydney. In Brisbane and Melbourne, the workshops were divided into 2 groups: aged 16 to 25 years and aged 26 to 35 years. Workshop and participant characteristics are summarized inand listed in the chronological order they were conducted.
shows the characteristics of the user experience testing interviews and participants (phase 3). One participant (aged 39 years) at the second time point (March 2017) was discovered not to fall within the age range of the target audience, but we made the decision to proceed to gain any relevant insights the participant had to offer toward making the app more broadly accessible. Finally, to eliminate the possibility of bias arising from previously brainstorming this topic at length, user experience interviewees were different across all time points and none had previously attended any MindMax PD workshops.
Participatory Design Workshop: Descriptive Content Analysis
App likes and dislikes were collected directly during the evaluation phase of PD workshops, where we asked participants to annotate screenshots of multiple existing health and well-being apps/websites. These preferences were coded by an independent knowledge translation team consisting of young people (younger than 25 years) according to the semantic themes content, design, and features/concept. Codes that were observed 3 or more times are presented below.
Compared with the other 2 semantic themes, content had the least observations. The most frequently observed preference was that participants disliked activities perceived to be “useless”, “anticlimactic”, “simplistic”, “condescending”, “childish”, or “boring”. Although they were receptive to activities promoting self-reflection, participants disliked vague suggestions (eg, “do something kind for yourself”) and liked having examples of how to do so. Ultimately, participants preferred information that was clear, nonrepetitive, and instructive (not descriptive—“tell me how, not why”). Although participants appreciated explanations for how an activity would benefit them, they preferred this information to be contained within a collapsible content box they could open if desired.
Participants overwhelmingly expressed dislikes of excessive blank space and also excessive amounts of text. “Gimmicky” user interfaces (UIs) and graphics received more criticism for looking “childish” and “cheap” than praise for looking “interesting” and having “cool colours”. Furthermore, dark colors such as dark green and brown were criticized for being “depressing” and “ugly”. Instead, participants preferred more conservative UIs with multiple pleasant colors (eg, pastel colors or light to medium blue) paired with simple graphics and names that clearly described the purpose and features of the app. Finally, participants liked how one app listed the number of people that had joined a particular psychoeducational course, as it showed the course appeared to be popular and, therefore, worth trying.
|Workshop (n=40), n||Location||Age group (years)||Gender split||Recruitment pool|
|1 (6)||Melbourne||16-25||3 males, 3 females||AFLPAa-affiliated individuals|
|2 (5)||Melbourne||26-35||3 males, 2 females||AFLPAa-affiliated individuals|
|3 (10)||Sydney||Mixed||4 males, 6 females||Mental health and technology academics and research students|
|4 (4)||Sydney||Mixed||2 males, 2 females||Mental health and technology academics and research students|
|5 (8)||Brisbane||16-25||8 males||Video game design and research students and staff|
|6 (7)||Brisbane||26-35||5 males, 2 females||Video game design and research students and staff|
aAFLPA: Australian Football League Players’ Association.
|Time point and participant||Age (years)||Gender||Mobile phone (operating system, OS)||Video game play (hour/week)||AFLa match watching (hour/week)|
|December 2016: Web-based alpha prototype (n=4)|
|P1||29||Female||iPhone 6 (iOS 10.1)||0||1|
|P2||33||Female||iPhone 5 (iOS 9)||0.5||1|
|P3||26||Male||iPhone 6 (iOS 10.1)||1||1|
|P4||29||Female||Samsung S7 Edge (Android; OS unsure)||0||0|
|March 2017: native app beta (n=5)|
|P5||19||Male||iPhone 6 (iOS 10.2.1)||1||2|
|P6||24||Male||iPhone 6 (iOS 10.2.1)||20||2|
|P7||39||Female||iPhone 6 (iOS 10.2.1)||20||0|
|P8||21||Female||iPhone 6 (iOS 10.2.1)||3||0|
|P9||20||Male||iPhone 6 (iOS 10.2.1)||30||0|
|November 2017: native app 5 months after launch (n=6)|
|P10||19||Male||Xiaomi RedMe Note 4 (Android 6.0)||5||0|
|P11||22||Male||Oneplus 3T (Android 6.0)||5||0|
|P12||22||Male||Samsung S5 (Android 6.0.1)||5||0|
|P13||22||Male||Samsung J7 Prime (Android 6.0.1)||20||0|
|P14||22||Male||Samsung Galaxy S5 (Android 6.0.1)||1 on average; 3 in holidays||0|
|P15||20||Male||iPhone 5 (iOS 11.0.3)||24||0|
aAFL: Australian Football League.
A large number of participants liked the idea of graphs and similar indicators such as goal progress bars, finding it motivational to track their progress. Similarly, a large number of participants also liked the concept of challenges/missions, encouraging them to go beyond their comfort zone. Quizzes attracted both positive and negative feedback, though the former was greater than the latter. Although participants liked having long-term goals and unlockable achievements, their reaction to rewards was ambivalent. Those who liked the concept felt they were “helpful and keep people coming back,” whereas those who did not found there to be “no reward for the player outside of a small number going up.” Participants also liked the skeuomorphic activities in several of the presented apps/websites, where participants could interact with the object on the screen similar to real life (such as scrunching up a piece of paper).
Finally, although competition was seen as “motivating” and “healthy” in the context of a physical health app, participants were ambivalent toward how it, and other types of social sharing options, could be implemented in mental health contexts. Participants felt that any social option that mimicked a major social network would be redundant and that they would not use it. Participants also felt that compared with physical health, mental health was a more private issue that complicated social sharing, both for the sharer and the people they would be sharing their mental health status with. In particular, participants raised the inappropriateness of such features for someone with poor mental health or who was in distress. Instead, participants suggested that social features be used to promote social connection and communication. That is, they should be “supportive rather than competitive.”
Furthermore, participants emphasized the importance of being able to customize their app experience, for example, through being able to customize their display image or avatar (if appropriate) or by having their responses to in-app questions influence their app notifications or recommendations. Participants also suggested the incorporation of design elements common to video games, including regular content updates, events, team competitions, and cosmetic digital rewards (eg, avatar hairstyles or clothing). Finally, the issue of mental health stigma was raised, and participants suggested the app have a function for facilitating conversations between men, for example, scheduling real-life activities between friends, where difficult topics could be broached in shoulder-to-shoulder conversations. Participants also specified that the app should adopt an approach of self-improvement, rather than fixing a deficiency.
User Experience Testing Interviews
Summary Across Time Points
Below, we present insights participants expressed during the user experience testing interviews and relevant quotes.
At alpha and beta build, participant feedback comprised identifying software bugs and glitches, criticizing unintuitive UIs and unclear wording, and raising privacy concerns. However, participants also expressed appreciation for the opportunity to see a more personal side of AFL athletes and for the underlying concept of the app:
It’s great! I like the idea and concept of it. Mindfulness, wellbeing. What it’s trying to achieve.
[It was] entertaining, kinda helpful—showed me a lot of values I didn’t know about—so, informative, engaging.
At 5 months after launch, negative participant feedback included identifying software bugs and glitches as well as questioning whether the social component would be used and critiquing its similarity to existing social networks such as Instagram.
However, most participants at this time point found MindMax to meet their user experience standards and to be an overall positive experience that provided some value:
I think overall this app is just for people to try out for curiosity. [...] Sometimes you want to say things to vent, but you can’t really say things to an app. This is like the fries, if the psychologist is a Big Mac.
It’s nice to have prominent masculine role models showing it’s okay to express emotion. Actively saying it’s okay to do so seems like a good thing to do for males in general.
Privacy concerns were raised across all time points. Although participants felt the information they provided MindMax was not particularly sensitive (and some provided false information to MindMax as a further precaution), they were worried that this information would be mistreated (eg, sold to marketers).
Content and Delivery
A total of 2 participants were red-green color blind and expressed that the colors used within MindMax were easy to differentiate.
There was a wide range of reactions to the casual tone of the app. Some participants appreciated it:
Good sense of humour. It like makes you feel relaxed.
If it’s too formal, I feel a bit of pressure.
However, on the other hand, some disliked it:
It’s cringey, like those Facebook memes. Makes me take it less seriously.
Participants also asked for more detail and specific, contextualizing examples:
Having an example [...] guidance as to what kind of behaviours are definitive of these values. [...] [Something] more personalised to my chosen value.
Videos were commonly skipped or watched for only the first few seconds. Participants requested the ability to rewind and fast-forward through videos and an indication of what to expect (through subtitles or a transcript). Although all videos were a maximum of 90 seconds long, they were still considered too long:
Once I see a video, I think “this is going to take a while.” Maybe if there was the length of the video on the bottom left or bottom right. If it was like 10 seconds I might watch it.
On 1 occasion, a participant skipped a video that contained key context explaining a later activity in the module, leading to brief confusion.
MindMax’s psychoeducational modules contained a variety of short interactive activities illustrative of the information in the modules. These activities ranged from uploading selfies and creating “shareables” (eg, a “MaxFive” plan for improving well-being) to share on the community feed, to more in-depth, reflective activities such as guided meditation.
Participant reactions to the social activities were mixed. Although some participants thought they were different and new, others felt they were inappropriate:
There’s places for selfies and this is not one of them.
(In response to negative feedback to the tone of the activity and privacy concerns, the selfie activity was removed in the launch version of the module.)
Participants who completed the Mindfulness and Emotions modules tended to skip the guided meditation activity halfway through but were also overall more positive about the activity:
That was pretty cool. [...] I don’t meditate usually so this was a nice experience.
Overall, participants were more positive about activities that required more focus and active participation, particularly in the context of MindMax’s psychoeducational modules being presented as Training:
If I didn’t have to physically write the postcard it wouldn’t have resonated so much.
The Mindfulness module fits the concept of “Train” the most. Thinking of a motivational quote and putting it on a picture isn’t training.
All female participants perceived avatar customization choices to lack feminine options and expressed feelings of alienation:
None of the options look like me, so I’ll make something that’s representative of something else.
This was exacerbated by the default avatar (presented as a base for users to customize) having a mustache. Female participants preferred starting with a blank avatar to this. Male participants did not report feeling alienated by the avatar customization choices.
Although participants preferred the freedom of being able to upload their own profile picture, the majority recognized that uploaded pictures would be difficult to moderate and that avatars provided increased anonymity.
Some participants were wary about posting content to the community feed, expressing not only privacy concerns but also more general image management concerns:
[While filling in a shareable]
I guess this would be posted to the main board or something?
Ah. So I don’t want to write something too silly.
Overall, participants were negative toward the social component of the app, thinking it was unnecessary and that neither they nor anyone else would use it:
Why care about likes, I’m here to improve mental wellbeing.
Everything’s on Facebook already. I don’t use any other apps other than Facebook to communicate with others.
When asked, most said they would not consciously post anything to the feed (beyond shareables, which were automatically posted through completing modules), though they were more willing to interact with posts on the feed.
Ultimately, participants wanted to use the app with people they already knew or had something in common with:
I just realised you can’t friend people in this, which, I don’t know. With AFL. If I used it a bit longer I would have eventually thought “Why can’t I join a group of just my AFL team rather than everyone?”
[I would like to see] a concept of Circles or Groups [...] I want a way to connect with friends. Like if a few of your friends had similar goals—or putting people with similar goals in a community together.
Applied Game Elements
Most participants stated they had never seen games combined with mental health and well-being before and expressed appreciation for the concept. When prompted to spend a “footy” (earned by completing psychoeducational modules and posting in the community feed) to play the casual game Flick Footy, participants at the final time point found the controls intuitive and the experience enjoyable. However, although it was broadly enjoyed, some participants also found it unoriginal and potentially not compelling enough to keep them using MindMax.
One participant found the integration of games and gameful elements within MindMax clumsy and half-hearted:
The app makes it seem the video games section is important, but it’s more than secondary—it’s so far from the general approach of the app that it seems put in in the last minute. Which is fine, but don’t make it seem so important.
The same participant also questioned whether it was appropriate to tie such a simple game so strongly to MindMax’s reward system:
I like video games, and these are very rudimentary, not very interactive, not very engaging [...] if these are the rewards for the activity you’re doing it’s a low reward for something so personal that you have to engage in. Games kind of cheapen the experience.
This series of studies aimed to use PD and user testing methods to determine the best way to present MindMax, an AFL-themed app aiming to deliver psychoeducation on mental health and well-being and to create a Web-based community centering on well-being, AFL, and video games. Our results suggest that the concept of combining mental health and well-being with sports and video games was well received by users. Participants gave further insights throughout MindMax’s development period that future mental health and well-being mHealth initiatives can learn from, whether or not they intend to incorporate applied games.
A consistent finding across PD workshops and user experience interviews was that participants did not have strong feelings about what content was presented, but rather how it was presented. Although participants found being presented with too much information at once off-putting, they did not want to be deprived of additional, contextualizing content as a result. Instead, they wanted to be able to control the flow of information, for example, through collapsible content boxes, and to have alternate modes of information, for example, through video subtitles and transcripts. Videos, in particular, were skipped on multiple occasions. This, along with previous research , suggests that key information should be presented in a variety of mediums in a way that minimizes repetitiveness.
Participants were sensitive to the formality (and lack thereof) of the various tones adopted by the health and well-being apps presented in PD workshops and in MindMax. Although a formal tone was perceived as intimidating, participants considered an informal tone less pressuring, but some also took the content less seriously as a result. Participants also did not want the tone to be patronizing or paternalistic and yet appreciated specific, direct instructions. Careful writing is needed to achieve this balance. Appropriate levels of formality and prescriptiveness likely vary based on the target audience and must be tested thoroughly with target users. Participants also expected a personalized experience , anticipating that MindMax would remember information they entered about themselves (eg, gender or the value they wanted to affirm) and that it would use that information in ways to help the user improve their well-being.
We also observed that participants preferred activities that required active engagement and practiced useful skills, paralleling earlier research on the topic . Activities such as guided meditation and writing a postcard to a loved one, which required comparatively more effort to complete, were better received than activities such as posting a selfie or choosing a quote from a preselected set of quotes. This suggests that our initial intention of reducing cognitive effort and making modules as easy to complete as possible was ineffective and may have undermined the purpose of MindMax. Instead, these findings suggest that user effort should be funneled toward completing activities.
Participants appreciated fun touches to app design (eg, skeuomorphic interactions), but ultimately the majority of participants seemed to prioritize the functional aspects of health apps, such as the ability to track their own health and well-being and information on how to improve their health and well-being. This does not mean there is no room for fun and enjoyment in mental health and well-being apps (multiple successful implementations of gamification in mental health and well-being [, ] support this). Rather, any implementation of playfulness or supplementary conceptual flavor should be considered and tested carefully.
Social and Applied Game Elements
Participants overall agreed that social motivations were important to attract and keep them engaged with a mental health and well-being app and that they would prefer the social component to be in relation to their existing social networks. Although they found social comparison and competition (both common gamification features ) motivating for improving physical health, they felt that incorporating comparison and competition in a mental health and well-being app could be inappropriate, particularly in cases where the user or someone in their network was in distress. During user experience testing, we observed image management behaviors where participants moderated how they expressed themselves as they were conscious of their potential audience. This was not a desired user behavior and could potentially undermine the improvement of mental health and well-being.
In PD workshops, participants suggested that MindMax and similar apps should complement and enable, rather than emulate, online and offline social connection. Crucially, this social component should be a different experience compared with using a mainstream social networking service. This was difficult to implement in practice, and MindMax ended up failing to follow this recommendation, instead implementing a community feed that user experience interviewees found extremely similar to Instagram. However, future conceptualizations of a social component could draw inspiration from cooperative games and incorporate more gamification elements that appeal to multiple types of users (players), such as social discovery, gifting, and unlockable content .
Our user experience interviewees, mostly experienced video game players, also expressed concerns that MindMax’s implementations of applied games lacked depth, which undermined their user experience. Their familiarity with video games may have contributed to their reduced interest in Flick Footy, which was a simple casual game with only 1 aim (score as many goals as possible). In addition to offering a unique, engaging experience, a more sophisticated implementation of cooperative game mechanics could potentially address these concerns and increase MindMax’s appeal to more hardcore video game fans.
The largest limitation of our findings is that during the user experience interview stages, we recruited locally (in Sydney) and therefore found it difficult to recruit AFL fans. However, much of our feedback was not AFL-related and would be useful to anyone designing an app incorporating applied games for mental health and well-being. MindMax’s usage analytics will give an indication of how people interested in AFL perceive and use MindMax and will be the focus of future investigation.
Another limitation is that although the iterative nature of our user testing allowed us to evaluate and improve on subsequent versions of MindMax, in practice, we were limited by financial, technological, and organizational constraints. The time frame of the grant required that development work occur in tandem with the user testing reported in this study. As a result, many features recommended by participants were determined to be unfeasible and descoped. For example, the social component that heavily resembled Instagram was implemented as ultimately there were not enough resources or time to create and implement an alternate concept for the social component. Similarly, although our female participants’ dissatisfaction with the masculine tone of the app was noted, it was not directly actioned given the focus of the project on reaching men in the target age demographic. Finally, given the need to prioritize basic functionality and content inclusion initially, the amount of time available for applied game design and development was reduced, which is reflected in the negative feedback on the games from interviewees with significant video games experience. Although it is also possible that the casual games in MindMax may never have appealed to more experienced video game players, more PD involving more stakeholders than just the research team and potential end users may have led to a smoother development process.
However, participant feedback, especially late-stage feedback, was able to be incorporated later in the project. MindMax underwent continual improvements beyond the time frame covered in this study, introducing new features and events until February 2018. This included new psychoeducational modules, redesigns of UI and existing modules, trophies, team functionality, and a “Flick Footy Max” campaign in December 2017 to promote engagement with the app. In “Flick Footy Max,” MindMax users competed to score the highest in “Flick Footy” to win a PlayStation 4 Pro and a MindMax-themed AFL football. Participant feedback hence continued to influence the development of MindMax beyond what is described in this study.
MindMax is based in both the AFL and video games subcultures, with the aim of appealing to men who enjoy these subcultures . Although we encountered some difficulties during development, many of which are inevitable on projects with defined time frames and funding windows, we ultimately produced an app containing psychoeducation, applied game elements, and a social component that was considered by user experience interviewees to be satisfactory. We plan to analyze the impact of MindMax via usage analytics and multiple time point survey data assessing users’ levels of well-being and judgments of MindMax’s usability.
Lessons learned from MindMax can be broadly applied to any app intending to help users improve their mental health and well-being, especially those planning to incorporate applied games. They can also be extended to other traditional sporting codes such as cricket, rugby, and soccer, and furthermore, to esports. The increasing collaboration between traditional sports and esports  and the growing popularity of esports among younger people [ ] may make a broad sports approach including esports suitable to apply to youth mental health. Finally, in the same way that traditional sporting codes such as AFL, rugby union, rugby league, and cricket are now promoting awareness of mental health problems, esports leagues can consider doing the same.
This study details the PD workshops and user experience testing that was conducted to obtain insights on how best to present MindMax. As an AFLPA initiative funded by Movember, MindMax presents a novel approach to evidence-based mental health and well-being education focusing on Australian men aged 16 to 35 years who enjoy AFL or video games. MindMax incorporates applied games and is couched in the Australian rules football (specifically AFL) subculture. If the implementation of MindMax is successful, there is the potential to generalize its model to other sporting codes such as rugby and cricket and even to partner with esports initiatives.
The authors thank all study participants for contributing to this research. The project was a collaboration funded by Movember Association; led by Australian Football League Players’ Association (AFLPA); and supported by Queensland University of Technology, The University of Sydney (via the former Young and Well Cooperative Research Centre headed by Professor Jane Burns), and The Mind Room. The authors particularly thank the knowledge translation team led by Ms Ellena Daniele and thank Ms Anna Hanna, Mr Fouad Yasin, and Ms Diana Chang for presenting the knowledge-translated design to the AFLPA.
Conflicts of Interest
IBH was an inaugural Commissioner on Australia’s National Mental Health Commission (2012-18). He is the Co-Director, Health and Policy at the Brain and Mind Centre (BMC), University of Sydney. The BMC operates an early-intervention youth service at Camperdown under contract to headspace. IBH has previously led community-based and pharmaceutical industry-supported (Wyeth, Eli Lily, Servier, Pfizer, and AstraZeneca) projects focused on the identification and better management of anxiety and depression. He was a member of the medical advisory panel for Medibank Private until October 2017, a board member of Psychosis Australia Trust, and a member of Veterans Mental Health Clinical Reference group. He is the Chief Scientific Advisor to, and an equity shareholder in, Innowell. Innowell has been formed by the University of Sydney and PwC to deliver the Aus $30 million Australian Government-funded “Project Synergy.” Project Synergy is a 3-year program for the transformation of mental health services through the use of innovative technologies. None of the other authors declare any conflicts of interest.
- Ellis LA, McCabe KL, Rahilly KA, Nicholas MA, Davenport TA, Burns JM, et al. Clinical Investigation. 2014 Oct. Encouraging young men's participation in mental health research and treatment: perspectives in our technological age URL: https://insights.ovid.com/clinical-practice/clni/2014/10/000/encouraging-young-men-participation-mental-health/3/01597081 [WebCite Cache]
- Diviak KR, Wahl SK, O'Keefe JJ, Mermelstein RJ, Flay BR. Recruitment and retention of adolescents in a smoking trajectory study: who participates and lessons learned. Subst Use Misuse 2006 Jul;41(2):175-182. [CrossRef] [Medline]
- Rice SM, Purcell R, McGorry PD. Adolescent and young adult male mental health: transforming system failures into proactive models of engagement. J Adolesc Health 2018 Mar;62(3S):S9-17 [FREE Full text] [CrossRef] [Medline]
- Batterham PJ, Calear AL. Preferences for internet-based mental health interventions in an adult online sample: findings from an online community survey. JMIR Ment Health 2017 Jun 30;4(2):e26 [FREE Full text] [CrossRef] [Medline]
- Patel V, Flisher A, Hetrick S, McGorry P. Mental health of young people: a global public-health challenge. The Lancet 2007 Apr;369(9569):1302-1313 [FREE Full text] [CrossRef]
- Iorfino F, Hermens DF, Cross SP, Zmicerevska N, Nichles A, Badcock CA, et al. Delineating the trajectories of social and occupational functioning of young people attending early intervention mental health services in Australia: a longitudinal study. BMJ Open 2018 Dec 27;8(3):e020678 [FREE Full text] [CrossRef] [Medline]
- Ellis LA, Collin P, Hurley PJ, Davenport TA, Burns JM, Hickie IB. Young men's attitudes and behaviour in relation to mental health and technology: implications for the development of online mental health services. BMC Psychiatry 2013;13:119 [FREE Full text] [CrossRef] [Medline]
- Ellis LA, Collin P, Davenport TA, Hurley PJ, Burns JM, Hickie IB. Young men, mental health, and technology: implications for service design and delivery in the digital age. J Med Internet Res 2012 Nov 22;14(6):e160 [FREE Full text] [CrossRef] [Medline]
- Ipsos Australia. 2015. Play On! - emma football trends and insights URL: https://www.emma.com.au/wp-content/uploads/2015/09/emma-football-report__FINAL.pdf [accessed 2018-08-20] [WebCite Cache]
- Brand J, Todhunter S, Jervis J. Digital Australia 2018. Eveleigh, NSW: IGEA; 2017. URL: https://www.igea.net/wp-content/uploads/2017/07/Digital-Australia-2018-DA18-Final-1.pdf [accessed 2018-08-20] [WebCite Cache]
- Kelly SJ. The Conversation. 2018. Esports are taking off and the Commonwealth Games needs to catch up URL: https://theconversation.com/esports-are-taking-off-and-the-commonwealth-games-needs-to-catch-up-94177 [accessed 2018-04-12] [WebCite Cache]
- Meeker M. Kleiner Perkins. 2017. Internet Trends 2017-Code Conference URL: https://www.kleinerperkins.com/perspectives/internet-trends-report-2017 [accessed 2018-08-20] [WebCite Cache]
- Stensholt J. Financial Review. 2018. AFL dives into e-sports with League of Legends deal URL: http://www.afr.com/business/sport/afl-dives-into-esports-with-league-of-legends-deal-20180503-h0zlyo [accessed 2018-10-24] [WebCite Cache]
- Schmidt R, Emmerich K, Schmidt B. Applied Games – In Search of a New Definition. In: Lecture Notes in Computer Science, vol 9353.: Springer International Publishing; 2015 Dec 01 Presented at: 14th International Conference, ICEC 2015; September 29 - October 2, 2015; Trondheim, Norway p. 100-111 URL: https://link.springer.com/chapter/10.1007%2F978-3-319-24589-8_8 [CrossRef]
- Fish MT, Russoniello CV, O'Brien K. The efficacy of prescribed casual videogame play in reducing symptoms of anxiety: a randomized controlled study. Games Health J 2014 Oct;3(5):291-295. [CrossRef] [Medline]
- Michael D, Chen SL. Serious Games: Games That Educate, Train, and Inform. Boston, MA: Cengage Learning PTR; 2005.
- Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining “gamification”. New York, NY, USA: ACM; 2011 Presented at: 15th International Academic MindTrek Conference: Envisioning Future Media Environments; September 28-30, 2011; Tampere, Finland p. 9-15. [CrossRef]
- Huotari K, Hamari J. Defining gamification: a service marketing perspective. New York, NY, USA: ACM; 2012 Presented at: 16th International Academic MindTrek Conference; October 3-5, 2012; Tampere, Finland p. 17-22. [CrossRef]
- Fleming TM, Bavin L, Stasiak K, Hermansson-Webb E, Merry SN, Cheek C, et al. Serious games and gamification for mental health: current status and promising directions. Front Psychiatry 2017;7:215 [FREE Full text] [CrossRef] [Medline]
- Jones CM, Scholes L, Johnson D, Katsikitis M, Carras MC. Gaming well: links between videogames and flourishing mental health. Front Psychol 2014 Mar;5:260 [FREE Full text] [CrossRef] [Medline]
- Seligman M. Flourish: A Visionary New Understanding of Happiness and Well-being. New York: Free Press; 2012.
- Johnson D, Jones C, Scholes L, Carras MC. Videogames and Wellbeing: A Comprehensive Review: Gaming Research Group, Young and Well Cooperative Research Centre. 2013. URL: https://eprints.qut.edu.au/105915/1/2013%20CRC%20Report%20Videogames_and_Wellbeing.pdf [WebCite Cache]
- Durkin K, Barber B. Not so doomed: computer game play and positive adolescent development. J Appl Dev Psychol 2002;23(4):373-392. [CrossRef]
- Przybylski AK, Weinstein N. A large-scale test of the Goldilocks hypothesis: quantifying the relations between digital-screen use and the mental well-being of adolescents. Psychol Sci 2017 Dec;28(2):204-215. [CrossRef] [Medline]
- Allahverdipour H, Bazargan M, Farhadinasab A, Moeini B. Correlates of video games playing among adolescents in an Islamic country. BMC Public Health 2010 May 27;10:286 [FREE Full text] [CrossRef] [Medline]
- Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games 2013 Dec;1(1):e3 [FREE Full text] [CrossRef] [Medline]
- King D, Greaves F, Exeter C, Darzi A. 'Gamification': influencing health behaviours with games. J R Soc Med 2013 Mar;106(3):76-78. [CrossRef] [Medline]
- Werbach K, Hunter D. For the Win: How Game Thinking Can Revolutionize Your Business. Pennsylvania: Wharton Digital Press; 2012.
- Robertson M. Kotaku. 2010. Can't Play, Won't Play URL: https://www.kotaku.com.au/2010/11/cant-play-wont-play/ [accessed 2018-08-20] [WebCite Cache]
- Fox LG, Barnes D. BoosterBuddy: using gamification as a compensatory strategy for motivational deficits. Psychiatr Serv 2016 Jan;67(1):141-142. [CrossRef] [Medline]
- Roepke AM, Jaffee SR, Riffle OM, McGonigal J, Broome R, Maxwell B. Randomized controlled trial of SuperBetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J 2015 Jun;4(3):235-246. [CrossRef] [Medline]
- Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst 2009;24(1):28 [FREE Full text]
- Looyestyn J, Kernot J, Boshoff K, Ryan J, Edney S, Maher C. Does gamification increase engagement with online programs? A systematic review. PLoS One 2017 Mar;12(3):e0173403 [FREE Full text] [CrossRef] [Medline]
- Hamari J. Do badges increase user activity? A field experiment on the effects of gamification. Comput Human Behav 2017 Jun;71:469-478. [CrossRef]
- Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform 2017 Dec;71:31-48 [FREE Full text] [CrossRef] [Medline]
- Hoffmann A, Christmann CA, Bleser G. Gamification in stress management apps: a critical app review. JMIR Serious Games 2017 Jun 7;5(2):e13 [FREE Full text] [CrossRef] [Medline]
- Johnson D, Deterding S, Kuhn KA, Staneva A, Stoyanov S, Hides L. Gamification for health and wellbeing: a systematic review of the literature. Internet Interv 2016 Nov;6:89-106. [CrossRef]
- Lister C, West JH, Cannon B, Sax T, Brodegard D. Just a fad? Gamification in health and fitness apps. JMIR Serious Games 2014 Aug;2(2):e9 [FREE Full text] [CrossRef] [Medline]
- Tondello GF, Mora A, Nacke LE. Elements of Gameful Design Emerging from User Preferences. New York, NY, USA: ACM; 2017 Presented at: Annual Symposium on Computer-Human Interaction in Play; October 15-18, 2017; Amsterdam, The Netherlands p. 129-142. [CrossRef]
- Fleming TM, de Beurs D, Khazaal Y, Gaggioli A, Riva G, Botella C, et al. Maximizing the impact of e-Therapy and serious gaming: time for a paradigm shift. Front Psychiatry 2016;7:65 [FREE Full text] [Medline]
- Earle AM, LaBrie JW, Boyle SC, Smith D. In pursuit of a self-sustaining college alcohol intervention: deploying gamified PNF in the real world. Addict Behav 2018 May;80:71-81. [CrossRef] [Medline]
- Muller MJ, Kuhn S. Participatory design. Commun ACM 1993;36(6):24-28. [CrossRef]
- Orlowski SK, Lawn S, Venning A, Winsall M, Jones GM, Wyld K, et al. Participatory research as one piece of the puzzle: a systematic review of consumer involvement in design of technology-based youth mental health and well-being interventions. JMIR Human Factors 2015 Jul 9;2(2):e12. [CrossRef]
- Hagen P, Collin P, Metcalf A, Nicholas M, Rahilly K, Swainston N. Westernsydney.edu.au. Melbourne: Young and Well Cooperative Research Centre; 2012. Participatory design of evidence-based online youth mental health promotion, prevention, early intervention and treatment URL: https://www.westernsydney.edu.au/__data/assets/pdf_file/0005/476330/Young_and_Well_CRC_IM_PD_Guide.pdf [accessed 2018-10-29] [WebCite Cache]
- Ospina-Pinillos L, Davenport TA, Ricci CS, Milton AC, Scott EM, Hickie IB. Developing a mental health eClinic to improve access to and quality of mental health care for young people: using participatory design as research methodologies. J Med Internet Res 2018 May 28;20(5):e188 [FREE Full text] [CrossRef] [Medline]
- Mitchell J, Hassed C. The Mindful AFL Player: Engagement, Mobile Apps, and Well-Being. In: Baltzell AL, editor. Mindfulness and Performance. Cambridge: Cambridge University Press; 2006:268-299.
- Keyes CL. Mental illness and/or mental health? Investigating axioms of the complete state model of health. J Consult Clin Psychol 2005 Jun;73(3):539-548. [CrossRef] [Medline]
- Mitchell J, Vella K, Johnson D, Peever N, Wan Sze Cheng V, Davenport T, et al. MindMax: Using Videogames and Sport to Engage Young Men and Improve Wellbeing. 2017 Presented at: 2nd Symposium Computing and Mental Health; May 6, 2017; Denver, CO URL: http://mentalhealth.media.mit.edu/wp-content/uploads/sites/46/2017/05/CMH_2017_paper_16.pdf
- Bauer MW, Gaskell GD. Qualitative Researching with Text, Image and Sound: A Practical Handbook for Social Research. London: SAGE Publications; 2000.
- Nielsen J. Nngroup. 2000. Why you only need to test with 5 users URL: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ [accessed 2018-07-06] [WebCite Cache]
- Tondello GF, Wehbe RR, Diamond L, Busch M, Marczewski A, Nacke LE. The Gamification User Types Hexad Scale. New York, NY, USA: ACM; 2016 Presented at: Annual Symposium on Computer-Human Interaction in Play; October 16-19, 2016; Austin, Texas, USA p. 229-243. [CrossRef]
- Kuusela H, Paul P. A comparison of concurrent and retrospective verbal protocol analysis. Am J Psychol 2000;113(3):387-404. [CrossRef]
|AFL: Australian Football League|
|AFLPA: Australian Football League Players’ Association|
|mHealth: mobile health|
|PD: participatory design|
|UI: user interface|
Edited by G Eysenbach; submitted 17.05.18; peer-reviewed by S Rice, T Fleming, M Nomali, K Stasiak; comments to author 28.06.18; revised version received 22.08.18; accepted 12.09.18; published 19.11.18Copyright
©Vanessa Wan Sze Cheng, Tracey A Davenport, Daniel Johnson, Kellie Vella, Jo Mitchell, Ian B Hickie. Originally published in JMIR Mental Health (http://mental.jmir.org), 19.11.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.