This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Mobile mental health apps have become ubiquitous tools to assist people in managing symptoms of anxiety and depression. However, due to the lack of research and expert input that has accompanied the development of most apps, concerns have been raised by clinicians, researchers, and government authorities about their efficacy.
This review aimed to estimate the proportion of mental health apps offering comprehensive therapeutic treatments for anxiety and/or depression available in the app stores that have been developed using evidence-based frameworks. It also aimed to estimate the proportions of specific frameworks being used in an effort to understand which frameworks are having the most influence on app developers in this area.
A systematic review of the Apple App Store and Google Play store was performed to identify apps offering comprehensive therapeutic interventions that targeted anxiety and/or depression. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist was adapted to guide this approach.
Of the 293 apps shortlisted as offering a therapeutic treatment for anxiety and/or depression, 162 (55.3%) mentioned an evidence-based framework in their app store descriptions. Of the 293 apps, 88 (30.0%) claimed to use cognitive behavioral therapy techniques, 46 (15.7%) claimed to use mindfulness, 27 (9.2%) claimed to use positive psychology, 10 (3.4%) claimed to use dialectical behavior therapy, 5 (1.7%) claimed to use acceptance and commitment therapy, and 20 (6.8%) claimed to use other techniques. Of the 162 apps that claimed to use a theoretical framework, only 10 (6.2%) had published evidence for their efficacy.
The current proportion of apps developed using evidence-based frameworks is unacceptably low, and those without tested frameworks may be ineffective, or worse, pose a risk of harm to users. Future research should establish what other factors work in conjunction with evidence-based frameworks to produce efficacious mental health apps.
The practice of psychotherapy is underpinned by evidence-based therapies and interventions. These techniques are used when they are proven to be efficacious via thorough experimental methods. In most developed countries, mental health clinicians are discouraged by professional associations and government regulations from using therapies without such evidence.
New technology is changing the way therapy can be delivered. Therapies previously only delivered face-to-face can now be accessed electronically, and the way that many people do this is via their smartphone. Smartphone apps are software programs that can be downloaded from specialist websites, known as app stores. The two biggest app stores are the Apple App Store and Google Play. Global expenditure on apps in 2018 was approximately US $92.1 billion [
Health, including mental health, apps are emerging as one of the most important categories of apps. A total of 75% of consumers believe that technology is important in managing their health, and 88% are willing to share mobile health (mHealth) data with their health care provider [
People with mental illness are attracted to the possibility of utilizing apps to manage their mental health [
There are many reasons why research on the efficacy of apps for anxiety and/or depression should be prioritized. Firstly, many people are already using mental health apps [
For a mental health app to be efficacious, it is fundamental that it be underpinned by an evidence-based framework; that is, an established therapy technique for reducing psychological distress, such as cognitive behavioral therapy (CBT). An evidence-based framework provides a road map or blueprint for an app’s functions and performance. However, even when an app claims to offer CBT, the contents and functioning of the app may not align with CBT principles [
To date, there has been little research that addresses the potential harm a mental health app may do to an individual. It seems conceivable, however, that an app developed without a proven theoretical framework and method of intervention could have the potential to do harm to a user [
Over the last two decades, e-mental health programs with a CBT framework and designed for use on computers have been found to be effective for both adults and children in reducing anxiety and depression [
CBT proposes that emotions can be more effectively managed by adjusting thinking and behavior and recognizing physiological responses [
Other therapies that have been developed from traditional CBT have also been used successfully to treat anxiety and depression. Positive psychology, for instance, has become a more recent framework that is, under a strict definition, a type of CBT but has a different focus. Rather than simply
Similarly, other frameworks, often referred to as third-wave therapies [
This study involved a systematic search of app stores and concentrated on apps that offered a comprehensive therapeutic treatment for anxiety and/or depression, as opposed to apps that may offer singular or novel interventions (see
This systematic review used the AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2 [
Four researchers, including the lead author (JMM), systematically searched the Apple App Store and Google Play store in December 2018, and then again in July 2019, to ensure newly available apps were included for review. These two marketplaces attract more than three times as much revenue from downloads as their next competitor, the Windows Store [
Searches in each marketplace were made with the following 19 keywords across all categories: mental health, depression, anxiety, wellbeing, happiness, psychological distress, positive psychology, suicide, mental illness, CBT, cognitive behaviour therapy, cognitive behavior therapy, ACT, acceptance and commitment therapy, DBT, dialectical behaviour therapy, dialectical behavior therapy, IPT, and interpersonal therapy.
Apps were shortlisted based on their app store descriptions, and apps that were available in both stores were only counted once.
Inclusion criteria were as follows:
The app language and store description are in English.
The app offers a therapeutic treatment for anxiety and/or depression, not just singular elements of a therapy, such as monitoring symptoms, recording thoughts, or diagnosing the disorder, although apps could have any of these as part of a therapeutic treatment. In this way, apps were excluded if they used only singular elements. This approach is in contrast to other similar reviews [
A search of the Apple App Store and Google Play store uncovered a shortlist of 293 apps whose app store descriptions inferred that they offered a therapeutic treatment for anxiety and/or depression; the full list of these apps is available from the corresponding author (JMM). Of these 293 apps, a total of 162 (55.3%) claimed to have an evidence-based theoretical framework informing the app’s development. Differences between the Apple App Store (112/197, 56.9%) and the Google Play store (50/96, 52%) were negligible.
The evidence-based frameworks found in the apps represented the following proportions: CBT, 30.0% (88/293); mindfulness, 15.7% (46/293); positive psychology, 9.2% (27/293); DBT, 3.4% (10/293); ACT, 1.7% (5/293); and others, 6.8% (20/293). Note, some apps claimed to use multiple frameworks—each time a framework was mentioned, it was counted.
When including only the 162 apps with a theoretical framework in the analysis, the breakdown was as follows: CBT, 54.3% (88/162); mindfulness, 28.4% (46/162); positive psychology, 16.7% (27/162); DBT, 6.2% (10/162); ACT, 3.1% (5/162); and others, 12.3% (20/162).
Of those 162 apps with evidence-based frameworks, 10 (6.2%) were found to have published evidence for their effectiveness (see
Coding used for app store search, and the search results.
Code | Description (ie, framework) | Apps with this framework (N=293), n (%) | Apps with published research, n (%) | Name of the app (or apps) with published research |
CBT | Cognitive behavioral therapy | 88 (30.0) | 5/88 (6) | Agoraphobia Free |
MIND | Mindfulness | 46 (15.7) | 3/46 (7) | Headspace |
POS | Positive psychology | 27 (9.2) | 1/27 (4) | SuperBetter |
DBT | Dialectical behavior therapy | 10 (3.4) | 0 (0) | N/Aa |
ACT | Acceptance and commitment therapy | 5 (1.7) | 0 (0) | N/A |
OTH | Other recognized framework | 20 (6.8) | 1/20 (5) | MindSurf |
NONE | No theoretical framework | 131 (44.7) | 0 (0) | N/A |
aN/A: not applicable.
Summary of published research for shortlisted apps.
App name | Reference | Sample characteristics | Intervention period; |
Statistically significant improvements | Outcome measure used |
Agoraphobia Free | Christoforou et al [ |
Intervention group: n=73 |
12 weeks; |
Anxiety | PASa |
Catch It | Kinderman et al [ |
Intervention group: n=285 |
6 weeks (but varied among participants); |
Noneb | |
PTSD Coach | Kuhn et al [ |
Intervention group: n=62 |
3 months; |
Anxiety and depression | PCL-Cc and PHQ-9d |
MoodMission | Bakker et al [ |
Intervention group (a): n=56 |
30 days; |
Depression | PHQ-9 |
Thought Challenger | Stiles-Shields et al [ |
Intervention group (a): n=10 |
6 weeks; |
Depression | PHQ-9 |
Headspace | Flett et al [ |
Intervention group: n=67 |
40 days; |
Depression | CES-De |
Smiling Mind | Flett et al [ |
Intervention group: n=58 |
40 days; |
Depression | CES-D |
DeStressify | Lee and Jung [ |
Intervention group: n=77 |
4 weeks; |
Trait anxiety | STAIf |
SuperBetter | Roepke et al [ |
Intervention group (a): n=93 |
4 weeks; |
Depression | CES-D |
MindSurf | Carey et al [ |
Intervention group: n=23 |
2 weeks; |
Anxiety and depression (not statistically significant) | DASS-21g |
aPAS: Panic and Agoraphobia Scale.
bUsed direct data input from apps based on words used by users to describe mood.
cPCL-C: PTSD (posttraumatic stress disorder) Checklist – Civilian.
dPHQ-9: Patient Health Questionnaire – Depression Scale.
eCES-D: Center for Epidemiologic Studies – Depression Scale.
fSTAI: State Trait Anxiety Inventory.
gDASS-21: Depression Anxiety Stress Scales – 21-Item Version.
The purpose of this study was to locate mobile mental health apps for treating anxiety and/or depression that contained recognized theoretical frameworks underpinning their development. A search of the Apple App Store and Google Play store revealed a total of 293 apps claiming to offer a therapeutic treatment for anxiety and/or depression, with just over half claiming to have been developed using an evidence-based theoretical framework. Of these, CBT was the most quoted framework in app store descriptions. The method used to identify these apps mimicked that of how a consumer in the general population would ordinarily locate an app to treat anxiety and/or depression; that is, by using the search function of an app store, then reading the description of each app.
CBT has a research history of over 60 years, longer than any of the other theoretical frameworks contained in the shortlisted apps [
The popularity of the term CBT, as well as the growing popularity of the terms used to identify the other evidence-based frameworks in this review, may be contributing to apps being developed by nonexperts who incorrectly quote these terms in app store descriptions as a means of attempting to gain legitimacy. There are no known safeguards in place anywhere in the world to stop this from happening. While government agencies have started to regulate the health app space, this regulation has thus far focused on apps that only pose a risk of harm to users [
A detailed analysis of the research that accompanied 10 of the shortlisted apps is outside the scope of this paper, as this review is focused on the theoretical frameworks that underpin mental health apps. The quality and quantity of research into mental health apps has been detailed elsewhere [
By examining evidence-based frameworks in mental health apps, this review has highlighted the high proportion (131/293, 44.7%) of apps that claim to offer a therapeutic treatment for anxiety and/or depression that do not rely on validated techniques. It is useful to think of an evidence-based theoretical framework as being like a map that guides clinicians in their therapeutic practices. This is at the heart of a clinician being effective in their treatment and, at the very least, not doing harm to their client or patient. While many mental health apps claim to be using an evidence-based theoretical framework, as many as 44.7% may not be; this leaves open the possibility, therefore, that a large proportion of these apps may be ineffective and possibly run the risk of doing the user harm.
This review of app stores suffers from the same limitation as other reviews of app stores: the restricted way that searches are conducted. The Apple App Store and Google Play store search results can be challenging because important information may be absent. How developers have completed online questionnaires prior to registering their app for public download, as well as the interaction of these with algorithms developed by the app store, determine the outcomes of a search. Consequently, there are differences in the order in which apps are presented when specific search terms are employed, and there are limited search functions compared to those available when performing a literature search. The results of an app store search are not necessarily presented in a logical order to users because they are unable to choose to display results according to multiple criteria, such as being able to filter from most recent to oldest, as one can do in a literature search. The outcome can lead to considerable ambiguity about the order of displayed results.
Another difference between a search of app stores and a search of the literature is that research may be found on a particular app, but that app may not be available for download to the general public. For example, Torous et al [
Another limitation of this research is that results were based on the contents of descriptions in the app stores. None of the shortlisted apps were trialed to confirm that their description corresponded to actual content. For example, a nonexpert may have developed an app and claimed in the description that it used a CBT framework, but the developer may not have incorporated any genuine CBT interventions into the app functions. If such pseudo-CBT apps exist, it is likely that they will fail to assist users to manage anxiety and/or depression and may, therefore, lead consumers to believe that CBT is ineffective [
This review has highlighted difficulties faced by clinicians and consumers when searching the app stores for an app that offers a therapeutic treatment for anxiety and/or depression. The limited search capabilities of the app stores make it difficult to find the most appropriate app for one’s needs. If an individual wants to find a mental health app based on an evidence-based framework, it is difficult to sort through the many other apps that do not have that guiding framework.
Just as mental health clinicians are trained to follow evidence-based frameworks in their practice, it is reasonable to assume that mental health apps should do the same in their functioning. This review found that little more than half do, according to their app store descriptions. Just as successful therapeutic outcomes of face-to-face therapy can be attributed to more than the theoretical framework—factors such as rapport with the therapist, therapist skills, and an individual’s motivation to change—so too are there other elements of a mental health app that may contribute to a successful therapeutic outcome, such as usability and ease of use, aesthetics, level of gamification, etc [
Another suggested area of future research is to compare apps developed with evidence-based theoretical frameworks with their face-to-face equivalents. Clinicians and consumers need to know about the effectiveness and limitations of apps and where they sit alongside traditional evidence-based approaches. If clinicians and consumers become more confident in understanding how mental health apps can assist in reducing symptoms of anxiety and/or depression, it may increase the take-up of this new treatment modality and turn the potential advantages of using mental health apps into a reality.
Protocol for App Store Systematic Reviews (PASSR).
acceptance and commitment therapy
A MeaSurement Tool to Assess systematic Reviews
cognitive behavioral therapy
dialectical behavior therapy
interpersonal therapy
mobile health
Protocol for App Store Systematic Reviews
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
randomized controlled trial
JMM was supported by an Australian Government Research Training Program Stipend Scholarship for this research. The provider of this funding had no role in the study’s design; the collection, analysis, or interpretation of data; the writing of the manuscript; or the decision to submit the manuscript for publication. The authors wish to thank Elizabeth Beaumont, Sarah Dean, and Brett Kerr for their assistance in conducting the app store searches.
JMM wrote the manuscript drafts and supervised the app store review. DAD and WB proofread and edited the manuscript drafts and supervised the overall research project. All authors read and approved the final submitted version.
None declared.