This paper is in the following e-collection/theme issue:

Anxiety and Stress Disorders (1002) Quality Evaluation and Descriptive Analysis/Reviews of Multiple Existing Mobile Apps (362) Depression and Mood Disorders; Suicide Prevention (1272)

Published on in Vol 7, No 6 (2020): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/16525, first published .
Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores

Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores

Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores

Authors of this article:

Jamie M Marshall1 Author Orcid Image ;   Debra A Dunstan1 Author Orcid Image ;   Warren Bartik1 Author Orcid Image
Article Authors Cited by (41) Tweetations (42) Metrics

    Original Paper

    School of Psychology, Faculty of Medicine and Health, University of New England, Armidale, Australia

    Corresponding Author:

    Jamie M Marshall, MClinPsych

    School of Psychology

    Faculty of Medicine and Health

    University of New England

    Elm Ave

    Armidale, 2351

    Australia

    Phone: 61 2 6773 3012

    Email: jmarsh21@myune.edu.au


    Background: Mobile mental health apps have become ubiquitous tools to assist people in managing symptoms of anxiety and depression. However, due to the lack of research and expert input that has accompanied the development of most apps, concerns have been raised by clinicians, researchers, and government authorities about their efficacy.

    Objective: This review aimed to estimate the proportion of mental health apps offering comprehensive therapeutic treatments for anxiety and/or depression available in the app stores that have been developed using evidence-based frameworks. It also aimed to estimate the proportions of specific frameworks being used in an effort to understand which frameworks are having the most influence on app developers in this area.

    Methods: A systematic review of the Apple App Store and Google Play store was performed to identify apps offering comprehensive therapeutic interventions that targeted anxiety and/or depression. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist was adapted to guide this approach.

    Results: Of the 293 apps shortlisted as offering a therapeutic treatment for anxiety and/or depression, 162 (55.3%) mentioned an evidence-based framework in their app store descriptions. Of the 293 apps, 88 (30.0%) claimed to use cognitive behavioral therapy techniques, 46 (15.7%) claimed to use mindfulness, 27 (9.2%) claimed to use positive psychology, 10 (3.4%) claimed to use dialectical behavior therapy, 5 (1.7%) claimed to use acceptance and commitment therapy, and 20 (6.8%) claimed to use other techniques. Of the 162 apps that claimed to use a theoretical framework, only 10 (6.2%) had published evidence for their efficacy.

    Conclusions: The current proportion of apps developed using evidence-based frameworks is unacceptably low, and those without tested frameworks may be ineffective, or worse, pose a risk of harm to users. Future research should establish what other factors work in conjunction with evidence-based frameworks to produce efficacious mental health apps.

    JMIR Ment Health 2020;7(6):e16525

    doi:10.2196/16525

    Keywords

    mHealth; apps; app store; depression; anxiety; e-mental health; smartphone; mobile mental health; digital mental health; mobile phone 


    Background

    The practice of psychotherapy is underpinned by evidence-based therapies and interventions. These techniques are used when they are proven to be efficacious via thorough experimental methods. In most developed countries, mental health clinicians are discouraged by professional associations and government regulations from using therapies without such evidence.

    New technology is changing the way therapy can be delivered. Therapies previously only delivered face-to-face can now be accessed electronically, and the way that many people do this is via their smartphone. Smartphone apps are software programs that can be downloaded from specialist websites, known as app stores. The two biggest app stores are the Apple App Store and Google Play. Global expenditure on apps in 2018 was approximately US $92.1 billion [1], which attests to the widespread use of apps and value of smartphones to consumers.

    Mental Health: Is There an App for That?

    Health, including mental health, apps are emerging as one of the most important categories of apps. A total of 75% of consumers believe that technology is important in managing their health, and 88% are willing to share mobile health (mHealth) data with their health care provider [2]. Approximately 48% of consumers are currently using a health app [2], and many more have downloaded at least one [3]. Health apps are in the top-10 list of worldwide categories for consumer spending on apps [4]. Although definitions about what constitutes a health app varies, and this influences the estimates of how many health apps are available, a 2017 report calculated that there were approximately 318,000 health apps available for download [5], with 10,000 of these relating to mental health [6].

    People with mental illness are attracted to the possibility of utilizing apps to manage their mental health [7]; however, there is doubt in this population about how apps manage confidential information [8]. There is also uncertainty about the available functions of mental health apps and which apps are the most useful [9]. For guidance about what app to download, mental health consumers rely mainly on ratings and reviews in app stores [10] or on advice given through social media or word of mouth [11]. However, if a mental health expert is not involved in the development of an app, and if it has not been developed using an evidence-based theoretical framework, the app may not be effective or, worse, may do harm to its user.

    There are many reasons why research on the efficacy of apps for anxiety and/or depression should be prioritized. Firstly, many people are already using mental health apps [6], and with five billion people around the world currently using a smartphone [12], there is potential for many more to do so in the future as this figure rises. Apps provide greater accessibility to mental health resources and offer instant mental health assistance. These features are especially relevant for traditionally difficult-to-reach cohorts, such as teenagers [13] and people in rural communities [14]. Mental health apps may also offer more cost-effective options for lower socioeconomic groups [15] and greater anonymity and flexibility for others. For example, using an app may be more convenient for consumers with limited time to access other therapy, and more convenient for clinicians who may prefer their clients and patients to do homework activities on their smartphone so that results can be digitally sent back to the clinician. Certain clients in therapy may also just prefer the novel value of doing interactive homework activities on their phone, rather than with a pen and paper. For many people, being able to use their phone for such activities is simply more convenient (eg, it may be easier for them to use a smartphone to do homework while they are travelling on a bus rather than handling awkward pieces of paper in such situations). In this way, apps may be used as adjuncts to other therapies. Apps can also be used in more practical ways, such as setting reminders to assist with treatment and medication compliance. All of these factors create the potential for apps to reduce the burden on existing mental health services [16].

    Apps and Evidence-Based Frameworks

    For a mental health app to be efficacious, it is fundamental that it be underpinned by an evidence-based framework; that is, an established therapy technique for reducing psychological distress, such as cognitive behavioral therapy (CBT). An evidence-based framework provides a road map or blueprint for an app’s functions and performance. However, even when an app claims to offer CBT, the contents and functioning of the app may not align with CBT principles [17], especially if the app has been developed without expert input.

    To date, there has been little research that addresses the potential harm a mental health app may do to an individual. It seems conceivable, however, that an app developed without a proven theoretical framework and method of intervention could have the potential to do harm to a user [18]. For instance, a recommendation to use herbal supplements could adversely interact with prescribed medication.

    Over the last two decades, e-mental health programs with a CBT framework and designed for use on computers have been found to be effective for both adults and children in reducing anxiety and depression [19,20] and improving happiness and well-being [21-23]. Some studies have compared the effectiveness of e-mental health programs with face-to-face therapy and have found comparable [24,25], and, in some cases, even more favorable, results [26]. There is now an interest in how stand-alone apps for smartphones (ie, programs that can be downloaded and used without an internet connection) based on CBT or other evidence-based theoretical frameworks may further enhance a clinician’s digital toolbox [27].

    CBT proposes that emotions can be more effectively managed by adjusting thinking and behavior and recognizing physiological responses [28]. CBT has an established evidence base and history of successfully treating anxiety and depression, both in individual therapy [29] and in group settings [30]. The CBT framework offers a broad range of efficacious interventions [31], including the following: the use of thought diaries to challenge negative thinking; coming up with personally meaningful affirmations; different types of physical exercise, such as walking and dancing; increasing social connectedness and having personally meaningful social interaction; increasing the amount of time doing pleasurable, or once pleasurable, activities; structured problem solving; and others. Many of these CBT interventions have been incorporated into mental health apps.

    Other therapies that have been developed from traditional CBT have also been used successfully to treat anxiety and depression. Positive psychology, for instance, has become a more recent framework that is, under a strict definition, a type of CBT but has a different focus. Rather than simply fixing psychopathology, positive psychology aims to help individuals reach their full well-being potential by increasing optimism and happiness [32]. Studies on various positive psychology interventions have revealed efficacious empirical support for reducing symptoms of anxiety and depression as well as increasing well-being and happiness. For example, Freedman and Enright [33] showed that forgiveness significantly decreased depression, Froh et al [34] implemented a gratitude intervention that resulted in significantly greater optimism and life satisfaction as well as decreased negative affect, Lyubomirsky et al [35] reported an increase in happiness and well-being for participants who completed acts of kindness, and Seligman et al [36] successfully used three strategies to increase happiness and decrease depressive symptoms: (1) using one’s signature strengths in new ways, (2) writing down three positive things each day, and (3) writing a letter of gratitude. All of these interventions could conceivably be incorporated into mental health apps and most already have.

    Similarly, other frameworks, often referred to as third-wave therapies [37], have offered up interventions that are suitable for use in mental health apps. Various mindfulness interventions have proved efficacious in the past [38], and these meditation and breathing activities are easily incorporated into an app. Dialectical behavior therapy (DBT) is an intersection of various interventions, including mindfulness, distress tolerance, emotional regulation, and improving social relatedness activities. These may involve dialectical interaction with a therapist, including recording distress ratings, among other interventions that have proven effective in reducing symptoms of anxiety and/or depression, with particular focus on individuals with a diagnosis of borderline personality disorder [39]. Interpersonal therapy (IPT) involves interventions designed to improve current interpersonal relationships and social dysfunction, rather than focusing on personality, as an efficacious way of improving symptoms of depression [40]. Acceptance and commitment therapy (ACT) focuses on an individual accepting distressing thoughts and performing committed actions guided by one’s values; ACT has proven efficacious for both anxiety [41] and depression [42]. All the frameworks listed here potentially comprise interventions that may conceivably be incorporated into an app.

    Objectives of This Study

    This study involved a systematic search of app stores and concentrated on apps that offered a comprehensive therapeutic treatment for anxiety and/or depression, as opposed to apps that may offer singular or novel interventions (see inclusion criteria in the Methods section for more details about this definition). The research questions were as follows: (1) What proportion of publicly available apps offering a therapeutic treatment for anxiety and/or depression have used an evidence-based framework in their development? and (2) In an effort to understand which specific frameworks are having the most influence on mental health app developers, what are the proportions of specific frameworks? No previous study or review could be located that examined this issue by focusing on publicly available apps listed in the app stores.


    This systematic review used the AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2 [43] and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [44] protocols for guidance in conducting the app store search. Although these protocols were developed specifically for literature searches, they offer appropriate direction that can be applied to searching app stores in the absence of such a specific guide. However, there are limitations in their use for guidance in app store searches. We adapted both of these protocols into a unique protocol and checklist specifically for app store searches: the Protocol for App Store Systematic Reviews (PASSR) checklist is a combination and reworking of the items from AMSTAR 2 and PRISMA that can be applied to the systematic search of app stores for any category of apps. The same wording as used in both AMSTAR 2 and PRISMA has been retained wherever possible to enable ease of comparison (see Multimedia Appendix 1).

    Four researchers, including the lead author (JMM), systematically searched the Apple App Store and Google Play store in December 2018, and then again in July 2019, to ensure newly available apps were included for review. These two marketplaces attract more than three times as much revenue from downloads as their next competitor, the Windows Store [45], and over 90% of available apps for depression can be found in either of these stores [46].

    Searches in each marketplace were made with the following 19 keywords across all categories: mental health, depression, anxiety, wellbeing, happiness, psychological distress, positive psychology, suicide, mental illness, CBT, cognitive behaviour therapy, cognitive behavior therapy, ACT, acceptance and commitment therapy, DBT, dialectical behaviour therapy, dialectical behavior therapy, IPT, and interpersonal therapy.

    Apps were shortlisted based on their app store descriptions, and apps that were available in both stores were only counted once.

    Inclusion criteria were as follows:

    1. The app language and store description are in English.
    2. The app offers a therapeutic treatment for anxiety and/or depression, not just singular elements of a therapy, such as monitoring symptoms, recording thoughts, or diagnosing the disorder, although apps could have any of these as part of a therapeutic treatment. In this way, apps were excluded if they used only singular elements. This approach is in contrast to other similar reviews [27] that have identified apps with single tools rather than comprehensive treatments. Therapeutic treatment was defined for this study’s purpose as “offering focused treatment suggestions specifically for reducing symptoms of anxiety and/or depression in a manner comprehensive enough to be considered a type of therapy.”

    A search of the Apple App Store and Google Play store uncovered a shortlist of 293 apps whose app store descriptions inferred that they offered a therapeutic treatment for anxiety and/or depression; the full list of these apps is available from the corresponding author (JMM). Of these 293 apps, a total of 162 (55.3%) claimed to have an evidence-based theoretical framework informing the app’s development. Differences between the Apple App Store (112/197, 56.9%) and the Google Play store (50/96, 52%) were negligible.

    The evidence-based frameworks found in the apps represented the following proportions: CBT, 30.0% (88/293); mindfulness, 15.7% (46/293); positive psychology, 9.2% (27/293); DBT, 3.4% (10/293); ACT, 1.7% (5/293); and others, 6.8% (20/293). Note, some apps claimed to use multiple frameworks—each time a framework was mentioned, it was counted.

    When including only the 162 apps with a theoretical framework in the analysis, the breakdown was as follows: CBT, 54.3% (88/162); mindfulness, 28.4% (46/162); positive psychology, 16.7% (27/162); DBT, 6.2% (10/162); ACT, 3.1% (5/162); and others, 12.3% (20/162).

    Of those 162 apps with evidence-based frameworks, 10 (6.2%) were found to have published evidence for their effectiveness (see Table 1). The nine research articles [47-55] that examined the 10 apps were made up of seven randomized controlled trials (RCTs)—the Flett et al article contained one RCT with 2 of the listed apps: Headspace and Smiling Mind [52]—and two feasibility or pilot studies: Kinderman et al [48] and Carey et al [55]. Total participant numbers were comprised of 1017 in intervention conditions and 447 in control groups. The mean age across those studies that provided enough information was 32.7 years (SD 9.3). Intervention phases fluctuated between 2 and 12 weeks. Three studies reported long-term follow-up, the longest period being 3 months (see Table 2 for a summary of research characteristics).

    Table 1. Coding used for app store search, and the search results.
    CodeDescription (ie, framework)Apps with this framework (N=293), n (%)Apps with published research, n (%)Name of the app (or apps) with published research
    CBTCognitive behavioral therapy88 (30.0)5/88 (6)Agoraphobia Free
    Catch It
    PTSD Coach
    MoodMission
    Thought Challenger
    MINDMindfulness46 (15.7)3/46 (7)Headspace
    Smiling Mind
    DeStressify
    POSPositive psychology27 (9.2)1/27 (4)SuperBetter
    DBTDialectical behavior therapy10 (3.4)0 (0)N/Aa
    ACTAcceptance and commitment therapy5 (1.7)0 (0)N/A
    OTHOther recognized framework20 (6.8)1/20 (5)MindSurf
    NONENo theoretical framework131 (44.7)0 (0)N/A

    aN/A: not applicable.

    Table 2. Summary of published research for shortlisted apps.
    App nameReferenceSample characteristicsIntervention period;
    long-term follow-up period    		Statistically significant improvementsOutcome measure used
    Agoraphobia FreeChristoforou et al [47]Intervention group: n=73
    Control group: n=69
    Mean age (SD): 39.7 years (11.3)    		12 weeks;
    no follow-up    		AnxietyPASa
    Catch ItKinderman et al [48]Intervention group: n=285
    Control group: none
    Mean age (SD): 48.2 years (SD not given)    		6 weeks (but varied among participants);
    no follow-up    		Positive and negative moodNoneb
    PTSD CoachKuhn et al [49]Intervention group: n=62
    Control group: n=58
    Mean age (SD): 39.3 years (14.6)    		3 months;
    3 months    		Anxiety and depressionPCL-Cc and PHQ-9d
    MoodMissionBakker et al [50]Intervention group (a): n=56
    Intervention group (b): n=56
    Intervention group (c): n=50
    Control group: n=64
    Mean age (SD): 34.2 years (12.1)    		30 days;
    no follow-up    		DepressionPHQ-9
    Thought ChallengerStiles-Shields et al [51]Intervention group (a): n=10
    Intervention group (b): n=10
    Control group: n=10
    Mean age (SD): not given    		6 weeks;
    4 weeks    		DepressionPHQ-9
    HeadspaceFlett et al [52]Intervention group: n=67
    Control group: n=67
    Mean age (SD): 20.1 years (2.8)    		40 days;
    no follow-up    		DepressionCES-De
    Smiling MindFlett et al [52]Intervention group: n=58
    Control group: n=67
    Mean age (SD): 20.1 years (2.8)    		40 days;
    no follow-up    		DepressionCES-D
    DeStressifyLee and Jung [53]Intervention group: n=77
    Control group: n=86
    Mean age (SD): 20.6 years (SD not given)    		4 weeks;
    no follow-up    		Trait anxietySTAIf
    SuperBetterRoepke et al [54]Intervention group (a): n=93
    Intervention group (b): n=97
    Control group: n=93
    Mean age (SD): 40.2 years (12.4)    		4 weeks;
    6 weeks    		DepressionCES-D
    MindSurfCarey et al [55]Intervention group: n=23
    Control group: none
    Mean age (SD): not given    		2 weeks;
    no follow-up    		Anxiety and depression (not statistically significant)DASS-21g

    aPAS: Panic and Agoraphobia Scale.

    bUsed direct data input from apps based on words used by users to describe mood.

    cPCL-C: PTSD (posttraumatic stress disorder) Checklist – Civilian.

    dPHQ-9: Patient Health Questionnaire – Depression Scale.

    eCES-D: Center for Epidemiologic Studies – Depression Scale.

    fSTAI: State Trait Anxiety Inventory.

    gDASS-21: Depression Anxiety Stress Scales – 21-Item Version.


    Principal Findings

    The purpose of this study was to locate mobile mental health apps for treating anxiety and/or depression that contained recognized theoretical frameworks underpinning their development. A search of the Apple App Store and Google Play store revealed a total of 293 apps claiming to offer a therapeutic treatment for anxiety and/or depression, with just over half claiming to have been developed using an evidence-based theoretical framework. Of these, CBT was the most quoted framework in app store descriptions. The method used to identify these apps mimicked that of how a consumer in the general population would ordinarily locate an app to treat anxiety and/or depression; that is, by using the search function of an app store, then reading the description of each app.

    CBT has a research history of over 60 years, longer than any of the other theoretical frameworks contained in the shortlisted apps [56]. It is widely known as being effective for a range of conditions, including cessation of smoking [57] and other drugs [58], managing pain [59], and relieving symptoms of mental ill-health across a range of psychological disorders [56]. CBT is also a term widely known across health-related settings and professions and is often used in information directed at the general community for public campaigns of ways to manage mental ill-health and stress [60]. It perhaps comes as no surprise, then, that CBT is the most widely used theoretical framework in apps for anxiety and depression. However, new research is accelerating in many of the other theoretical frameworks listed in this review [61]; it may be that the proportion of CBT-based apps lessens over time as a result.

    The popularity of the term CBT, as well as the growing popularity of the terms used to identify the other evidence-based frameworks in this review, may be contributing to apps being developed by nonexperts who incorrectly quote these terms in app store descriptions as a means of attempting to gain legitimacy. There are no known safeguards in place anywhere in the world to stop this from happening. While government agencies have started to regulate the health app space, this regulation has thus far focused on apps that only pose a risk of harm to users [62]. While this is important and a welcome addition to the oversight of mental health apps, it does not provide checks on accuracy of information in app store descriptions of apps that may not fall into the category of posing a risk of harm. Such apps may provide fake or incorrect information that may, at worst, be ineffective at reducing anxiety or depression, but they may continue to be available in app marketplaces because they do not meet criteria that would identify them as posing a risk of harm to users. If an app claims to wrongly categorize its interventions as CBT, DBT, ACT, mindfulness, or positive psychology, one of the dangers is that such misuse of these terms may lead users to believe that such theoretical frameworks do not work if the user does not get any benefit from the app.

    A detailed analysis of the research that accompanied 10 of the shortlisted apps is outside the scope of this paper, as this review is focused on the theoretical frameworks that underpin mental health apps. The quality and quantity of research into mental health apps has been detailed elsewhere [63-66]. However, the research found to accompany the apps listed here does appear to vary greatly in methodology; this reaffirms the claims of heterogeneity made in those literature reviews. These reviews all call for more research and ongoing evaluation of research methodology into studying apps, as the current methodologies may not be the most appropriate [62,67,68].

    By examining evidence-based frameworks in mental health apps, this review has highlighted the high proportion (131/293, 44.7%) of apps that claim to offer a therapeutic treatment for anxiety and/or depression that do not rely on validated techniques. It is useful to think of an evidence-based theoretical framework as being like a map that guides clinicians in their therapeutic practices. This is at the heart of a clinician being effective in their treatment and, at the very least, not doing harm to their client or patient. While many mental health apps claim to be using an evidence-based theoretical framework, as many as 44.7% may not be; this leaves open the possibility, therefore, that a large proportion of these apps may be ineffective and possibly run the risk of doing the user harm.

    Limitations

    This review of app stores suffers from the same limitation as other reviews of app stores: the restricted way that searches are conducted. The Apple App Store and Google Play store search results can be challenging because important information may be absent. How developers have completed online questionnaires prior to registering their app for public download, as well as the interaction of these with algorithms developed by the app store, determine the outcomes of a search. Consequently, there are differences in the order in which apps are presented when specific search terms are employed, and there are limited search functions compared to those available when performing a literature search. The results of an app store search are not necessarily presented in a logical order to users because they are unable to choose to display results according to multiple criteria, such as being able to filter from most recent to oldest, as one can do in a literature search. The outcome can lead to considerable ambiguity about the order of displayed results.

    Another difference between a search of app stores and a search of the literature is that research may be found on a particular app, but that app may not be available for download to the general public. For example, Torous et al [27] discovered peer-reviewed publications on the efficacy of four CBT-based apps—excluding apps that were based on DBT or ACT—but when they searched the app stores for these apps, none could be found. While research is welcomed and encouraged in the development of mental health apps, in the end what matters most is how many of these efficacious apps are available to the general population.

    Another limitation of this research is that results were based on the contents of descriptions in the app stores. None of the shortlisted apps were trialed to confirm that their description corresponded to actual content. For example, a nonexpert may have developed an app and claimed in the description that it used a CBT framework, but the developer may not have incorporated any genuine CBT interventions into the app functions. If such pseudo-CBT apps exist, it is likely that they will fail to assist users to manage anxiety and/or depression and may, therefore, lead consumers to believe that CBT is ineffective [27].

    Conclusions

    This review has highlighted difficulties faced by clinicians and consumers when searching the app stores for an app that offers a therapeutic treatment for anxiety and/or depression. The limited search capabilities of the app stores make it difficult to find the most appropriate app for one’s needs. If an individual wants to find a mental health app based on an evidence-based framework, it is difficult to sort through the many other apps that do not have that guiding framework.

    Just as mental health clinicians are trained to follow evidence-based frameworks in their practice, it is reasonable to assume that mental health apps should do the same in their functioning. This review found that little more than half do, according to their app store descriptions. Just as successful therapeutic outcomes of face-to-face therapy can be attributed to more than the theoretical framework—factors such as rapport with the therapist, therapist skills, and an individual’s motivation to change—so too are there other elements of a mental health app that may contribute to a successful therapeutic outcome, such as usability and ease of use, aesthetics, level of gamification, etc [65,69]. There would appear to be much research that still needs to be done on all these factors and how they interrelate with theoretical frameworks as well as whether certain factors mediate or moderate others.

    Another suggested area of future research is to compare apps developed with evidence-based theoretical frameworks with their face-to-face equivalents. Clinicians and consumers need to know about the effectiveness and limitations of apps and where they sit alongside traditional evidence-based approaches. If clinicians and consumers become more confident in understanding how mental health apps can assist in reducing symptoms of anxiety and/or depression, it may increase the take-up of this new treatment modality and turn the potential advantages of using mental health apps into a reality.

    Acknowledgments

    JMM was supported by an Australian Government Research Training Program Stipend Scholarship for this research. The provider of this funding had no role in the study’s design; the collection, analysis, or interpretation of data; the writing of the manuscript; or the decision to submit the manuscript for publication. The authors wish to thank Elizabeth Beaumont, Sarah Dean, and Brett Kerr for their assistance in conducting the app store searches.

    Authors' Contributions

    JMM wrote the manuscript drafts and supervised the app store review. DAD and WB proofread and edited the manuscript drafts and supervised the overall research project. All authors read and approved the final submitted version.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Protocol for App Store Systematic Reviews (PASSR).

    PDF File (Adobe PDF File), 147 KB
    1. Freier A. Business of Apps. Staines-upon-Thames, UK. Soko Media; Sep 13, 2018. URL: https:/​/www.​businessofapps.com/​news/​app-revenue-reaches-92-1-billion-in-2018-driven-by-mobile-gaming-apps/​ [accessed 2020-06-09]
    2. Accenture. New York, NY. Accenture; 2018. URL: https:/​/www.​accenture.com/​us-en/​insight-new-2018-consumer-survey-digital-health?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_ axiosvitals&stream=top-stories [accessed 2019-07-12]
    3. Colder Carras M, Mojtabai R, Furr-Holden CD, Eaton W, Cullen BAM. Use of mobile phones, computers and internet among clients of an inner-city community psychiatric clinic. J Psychiatr Pract. Mar 2014;20(2):94-103. [FREE Full text] [CrossRef] [Medline]
    4. Sydow L. App Annie. New York, NY. App Annie; Jan 16, 2019. URL: https://www.appannie.com/en/insights/market-data/the-state-of-mobile-2019/ [accessed 2019-08-20]
    5. IQVIA Institute. The Growing Value of Digital Health: Evidence and Impact on Human Health and the Healthcare System. Parsippany, NJ. IQVIA Institute for Human Data Science; Nov 07, 2017. URL: https://www.iqvia.com/institute/reports/the-growing-value-of-digital-health [accessed 2019-01-25]
    6. Torous J, Firth J, Huckvale K, Larsen ME, Cosco TD, Carney R, et al. The emerging imperative for a consensus approach toward the rating and clinical recommendation of mental health apps. J Nerv Ment Dis. Aug 2018;206(8):662-666. [CrossRef] [Medline]
    7. Torous J, Powell AC. Current research and trends in the use of smartphone applications for mood disorders. Internet Interv. May 2015;2(2):169-173. [CrossRef]
    8. Hendrikoff L, Kambeitz-Ilankovic L, Pryss R, Senner F, Falkai P, Pogarell O, et al. Prospective acceptance of distinct mobile mental health features in psychiatric patients and mental health professionals. J Psychiatr Res. Feb 2019;109:126-132. [CrossRef] [Medline]
    9. Carpenter-Song E, Noel VA, Acquilano SC, Drake RE. Real-world technology use among people with mental illnesses: Qualitative study. JMIR Ment Health. Nov 23, 2018;5(4):e10652. [FREE Full text] [CrossRef] [Medline]
    10. Huang H, Bashir M. Users' adoption of mental health apps: Examining the impact of information cues. JMIR Mhealth Uhealth. Jun 28, 2017;5(6):e83. [FREE Full text] [CrossRef] [Medline]
    11. Rubanovich CK, Mohr DC, Schueller SM. Health app use among individuals with symptoms of depression and anxiety: A survey study with thematic coding. JMIR Ment Health. Jun 23, 2017;4(2):e22. [FREE Full text] [CrossRef] [Medline]
    12. O'Dea S. Statista. New York, NY. Statista; Feb 28, 2019. URL: https://www.statista.com/statistics/330695/numberof-smartphone-users-worldwide/ [accessed 2019-10-19]
    13. Wang K, Varma DS, Prosperi M. A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. J Psychiatr Res. Dec 2018;107:73-78. [CrossRef] [Medline]
    14. Jones N, Moffitt M. Ethical guidelines for mobile app development within health and mental health fields. Prof Psychol Res Pr. Apr 2016;47(2):155-162. [CrossRef]
    15. Smith A. Pew Research Center. Washington, DC. Pew Internet & American Life Project; Apr 01, 2015. URL: http://www.pewinternet.org/2015/04/01/chapter-one-a-portrait-of-smartphone-ownership/ [accessed 2019-01-20]
    16. Newman MG, Szkodny LE, Llera SJ, Przeworski A. A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: Is human contact necessary for therapeutic efficacy? Clin Psychol Rev. Feb 2011;31(1):89-103. [CrossRef] [Medline]
    17. Stawarz K, Preist C, Tallon D, Wiles N, Coyle D. User experience of cognitive behavioral therapy apps for depression: An analysis of app functionality and user reviews. J Med Internet Res. Jun 06, 2018;20(6):e10120. [FREE Full text] [CrossRef] [Medline]
    18. Moshi MR, Tooher R, Merlin T. Suitability of current evaluation frameworks for use in the health technology assessment of mobile medical applications: A systematic review. Int J Technol Assess Health Care. Jan 2018;34(5):464-475. [CrossRef] [Medline]
    19. Ebert DD, Zarski A, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: A meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10(3):e0119895. [FREE Full text] [CrossRef] [Medline]
    20. Richards D, Richardson T. Computer-based psychological treatments for depression: A systematic review and meta-analysis. Clin Psychol Rev. Jun 2012;32(4):329-342. [CrossRef] [Medline]
    21. Davies EB, Morriss R, Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: A systematic review and meta-analysis. J Med Internet Res. May 16, 2014;16(5):e130. [FREE Full text] [CrossRef] [Medline]
    22. Manicavasagar V, Horswood D, Burckhardt R, Lum A, Hadzi-Pavlovic D, Parker G. Feasibility and effectiveness of a web-based positive psychology program for youth mental health: Randomized controlled trial. J Med Internet Res. Jun 04, 2014;16(6):e140. [FREE Full text] [CrossRef] [Medline]
    23. Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, et al. Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in the general population: Randomized controlled trial. J Med Internet Res. Dec 31, 2012;15(1):e2. [FREE Full text] [CrossRef] [Medline]
    24. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: A meta-analysis. PLoS One. Oct 13, 2010;5(10):e13196. [FREE Full text] [CrossRef] [Medline]
    25. Richardson T, Stallard P, Velleman S. Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: A systematic review. Clin Child Fam Psychol Rev. Sep 2010;13(3):275-290. [CrossRef] [Medline]
    26. Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MFG. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: Randomised controlled non-inferiority trial. BMJ. Apr 18, 2012;344:e2598. [FREE Full text] [CrossRef] [Medline]
    27. Torous J, Levin ME, Ahern DK, Oser ML. Cognitive behavioral mobile applications: Clinical studies, marketplace overview, and research agenda. Cogn Behav Pract. May 2017;24(2):215-225. [CrossRef]
    28. Beck AT, Rush AJ, Shaw BF, Emery G. Cognitive Therapy of Depression. New York, NY. Guilford Press; 1979.
    29. Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clin Psychol Rev. Jan 2006;26(1):17-31. [CrossRef] [Medline]
    30. McDermut W, Miller IW, Brown RA. The efficacy of group psychotherapy for depression: A meta-analysis and review of the empirical research. Clin Psychol Sci Pract. 2001;8(1):98-116. [CrossRef]
    31. Bennett-Levy J, Richards D, Farrand P, Christensen H, Griffiths K, Kavanagh D, et al, editors. Oxford Guide To Low Intensity CBT Interventions. New York, NY. Oxford University Press; 2010.
    32. Seligman MEP, Csikszentmihalyi M. Positive psychology: An introduction. Am Psychol. 2000;55(1):5-14. [CrossRef]
    33. Freedman SR, Enright RD. Forgiveness as an intervention goal with incest survivors. J Consult Clin Psychol. 1996;64(5):983-992. [CrossRef]
    34. Froh JJ, Sefick WJ, Emmons RA. Counting blessings in early adolescents: An experimental study of gratitude and subjective well-being. J Sch Psychol. Apr 2008;46(2):213-233. [CrossRef] [Medline]
    35. Lyubomirsky S, Sheldon KM, Schkade D. Pursuing happiness: The architecture of sustainable change. Rev Gen Psychol. Jun 2005;9(2):111-131. [CrossRef]
    36. Seligman MEP, Steen TA, Park N, Peterson C. Positive psychology progress: Empirical validation of interventions. Am Psychol. 2005;60(5):410-421. [CrossRef] [Medline]
    37. Hayes SC, Hofmann SG. The third wave of cognitive behavioral therapy and the rise of process-based care. World Psychiatry. Oct 2017;16(3):245-246. [FREE Full text] [CrossRef] [Medline]
    38. Hofmann SG, Sawyer AT, Witt AA, Oh D. The effect of mindfulness-based therapy on anxiety and depression: A meta-analytic review. J Consult Clin Psychol. Apr 2010;78(2):169-183. [FREE Full text] [CrossRef] [Medline]
    39. Lynch TR, Chapman AL, Rosenthal MZ, Kuo JR, Linehan MM. Mechanisms of change in dialectical behavior therapy: Theoretical and empirical observations. J Clin Psychol. Apr 2006;62(4):459-480. [CrossRef] [Medline]
    40. de Mello MF, de Jesus Mari J, Bacaltchuk J, Verdeli H, Neugebauer R. A systematic review of research findings on the efficacy of interpersonal therapy for depressive disorders. Eur Arch Psychiatry Clin Neurosci. Apr 2005;255(2):75-82. [CrossRef] [Medline]
    41. Vøllestad J, Nielsen MB, Nielsen GH. Mindfulness- and acceptance-based interventions for anxiety disorders: A systematic review and meta-analysis. Br J Clin Psychol. Sep 2012;51(3):239-260. [CrossRef] [Medline]
    42. A-Tjak JGL, Davis ML, Morina N, Powers MB, Smits JAJ, Emmelkamp PMG. A meta-analysis of the efficacy of acceptance and commitment therapy for clinically relevant mental and physical health problems. Psychother Psychosom. 2015;84(1):30-36. [CrossRef] [Medline]
    43. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. Sep 21, 2017;358:j4008. [FREE Full text] [CrossRef] [Medline]
    44. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. Jul 21, 2009;6(7):e1000097. [FREE Full text] [CrossRef] [Medline]
    45. Statista. New York, NY. Statista; 2019. URL: https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/ [accessed 2019-07-12]
    46. Shen N, Levitan M, Johnson A, Bender JL, Hamilton-Page M, Jadad AAR, et al. Finding a depression app: A review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth. Feb 16, 2015;3(1):e16. [FREE Full text] [CrossRef] [Medline]
    47. Christoforou M, Sáez Fonseca JA, Tsakanikos E. Two novel cognitive behavioral therapy-based mobile apps for agoraphobia: Randomized controlled trial. J Med Internet Res. Nov 24, 2017;19(11):e398. [FREE Full text] [CrossRef] [Medline]
    48. Kinderman P, Hagan P, King S, Bowman J, Chahal J, Gan L, et al. The feasibility and effectiveness of Catch It, an innovative CBT smartphone app. BJPsych Open. May 2016;2(3):204-209. [FREE Full text] [CrossRef] [Medline]
    49. Kuhn E, Kanuri N, Hoffman JE, Garvert DW, Ruzek JI, Taylor CB. A randomized controlled trial of a smartphone app for posttraumatic stress disorder symptoms. J Consult Clin Psychol. Mar 2017;85(3):267-273. [CrossRef] [Medline]
    50. Bakker D, Kazantzis N, Rickwood D, Rickard N. A randomized controlled trial of three smartphone apps for enhancing public mental health. Behav Res Ther. Oct 2018;109:75-83. [CrossRef] [Medline]
    51. Stiles-Shields C, Montague E, Kwasny MJ, Mohr DC. Behavioral and cognitive intervention strategies delivered via coached apps for depression: Pilot trial. Psychol Serv. May 2019;16(2):233-238. [FREE Full text] [CrossRef] [Medline]
    52. Flett JAM, Hayne H, Riordan BC, Thompson LM, Conner TS. Mobile mindfulness meditation: A randomised controlled trial of the effect of two popular apps on mental health. Mindfulness. 2019;10(5):863-876. [CrossRef]
    53. Lee RA, Jung ME. Evaluation of an mHealth app (DeStressify) on university students' mental health: Pilot trial. JMIR Ment Health. Jan 23, 2018;5(1):e2. [FREE Full text] [CrossRef] [Medline]
    54. Roepke AM, Jaffee SR, Riffle OM, McGonigal J, Broome R, Maxwell B. Randomized controlled trial of SuperBetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J. Jun 2015;4(3):235-246. [CrossRef] [Medline]
    55. Carey TA, Haviland J, Tai SJ, Vanags T, Mansell W. MindSurf: A pilot study to assess the usability and acceptability of a smartphone app designed to promote contentment, wellbeing, and goal achievement. BMC Psychiatry. Dec 12, 2016;16(1):442. [FREE Full text] [CrossRef] [Medline]
    56. Thoma N, Pilecki B, McKay D. Contemporary cognitive behavior therapy: A review of theory, history, and evidence. Psychodyn Psychiatry. Sep 2015;43(3):423-461. [CrossRef] [Medline]
    57. Webb M, de Ybarra DR, Baker E, Reis I, Carey M. Cognitive-behavioral therapy to promote smoking cessation among African American smokers: A randomized clinical trial. J Consult Clin Psychol. Feb 2010;78(1):24-33. [CrossRef] [Medline]
    58. Magill M, Ray LA. Cognitive-behavioral treatment with adult alcohol and illicit drug users: A meta-analysis of randomized controlled trials. J Stud Alcohol Drugs. Jul 2009;70(4):516-527. [FREE Full text] [CrossRef] [Medline]
    59. Ehde DM, Dillworth TM, Turner JA. Cognitive-behavioral therapy for individuals with chronic pain: Efficacy, innovations, and directions for research. Am Psychol. 2014;69(2):153-166. [CrossRef] [Medline]
    60. World Health Organization. Scalable Psychological Interventions for People in Communities Affected by Adversity: A New Area of Mental Health and Psychosocial Work at WHO. Geneva, Switzerland. World Health Organization; 2017. URL: https:/​/apps.​who.int/​iris/​bitstream/​handle/​10665/​254581/​WHO-MSD-MER-17.​1-eng.​pdf;jsessionid=3E3A6AD80F948549B0FD8C4BB4B14E5C?sequence [accessed 2020-06-05]
    61. Dimidjian S, Arch JJ, Schneider RL, Desormeau P, Felder JN, Segal ZV. Considering meta-analysis, meaning, and metaphor: A systematic review and critical examination of "third wave" cognitive and behavioral therapies. Behav Ther. Nov 2016;47(6):886-905. [CrossRef] [Medline]
    62. Marshall JM, Dunstan DA, Bartik W. Smartphone psychology: New approaches towards safe and efficacious mobile mental health apps. Prof Psychol Res Pr. Jun 2020;51(3):214-222. [FREE Full text] [CrossRef]
    63. Marshall JM, Dunstan DA, Bartik W. Clinical or gimmickal: The use and effectiveness of mobile mental health apps for treating anxiety and depression. Aust N Z J Psychiatry. Jan 2020;54(1):20-28. [FREE Full text] [CrossRef] [Medline]
    64. Alyami M, Giri B, Alyami H, Sundram F. Social anxiety apps: A systematic review and assessment of app descriptors across mobile store platforms. Evid Based Ment Health. Aug 2017;20(3):65-70. [CrossRef] [Medline]
    65. Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: Review and evidence-based recommendations for future developments. JMIR Ment Health. Mar 01, 2016;3(1):e7. [FREE Full text] [CrossRef] [Medline]
    66. Orman J, O'Dea B. E-Therapy in primary care mental health. Aust J Gen Pract. Apr 2018;47(4):168-172. [FREE Full text] [CrossRef] [Medline]
    67. Clough BA, Casey LM. Smart designs for smart technologies: Research challenges and emerging solutions for scientist-practitioners within e-mental health. Prof Psychol Res Pr. Dec 2015;46(6):429-436. [CrossRef]
    68. Broussard JD, Teng EJ. Models for enhancing the development of experiential learning approaches within mobile health technologies. Prof Psychol Res Pr. Jun 2019;50(3):195-203. [CrossRef]
    69. Jessen S, Mirkovic J, Ruland CM. Creating gameful design in mHealth: A participatory co-design approach. JMIR Mhealth Uhealth. Dec 14, 2018;6(12):e11579. [FREE Full text] [CrossRef] [Medline]

    ‎
    ACT: acceptance and commitment therapy
    AMSTAR: A MeaSurement Tool to Assess systematic Reviews
    CBT: cognitive behavioral therapy
    DBT: dialectical behavior therapy
    IPT: interpersonal therapy
    mHealth: mobile health
    PASSR: Protocol for App Store Systematic Reviews
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    RCT: randomized controlled trial

    Edited by A Tal; submitted 06.10.19; peer-reviewed by B Greer, D Bakker; comments to author 10.03.20; revised version received 27.03.20; accepted 12.04.20; published 24.06.20.

    Copyright

    ©Jamie M Marshall, Debra A Dunstan, Warren Bartik. Originally published in JMIR Mental Health (http://mental.jmir.org), 24.06.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.