Background: Smartphone apps could constitute a cost-effective strategy to overcome health care system access barriers to mental health services for people in low- and middle-income countries.
Objective: The aim of this paper was to explore the patients’ perspectives of CONEMO (Emotional Control, in Spanish: Control Emocional), a technology-driven, psychoeducational, and nurse-supported intervention delivered via a smartphone app aimed at reducing depressive symptoms in people with diabetes, hypertension or both who attend public health care centers, as well as the nurses’ feedback about their role and its feasibility to be scaled up.
Methods: This study combines data from 2 pilot studies performed in Lima, Peru, between 2015 and 2016, to test the feasibility of CONEMO. Interviews were conducted with 29 patients with diabetes, hypertension or both with comorbid depressive symptoms who used CONEMO and 6 staff nurses who accompanied the intervention. Using a content analysis approach, interview notes from patient interviews were transferred to a digital format, coded, and categorized into 6 main domains: the perceived health benefit, usability, adherence, user satisfaction with the app, nurse’s support, and suggestions to improve the intervention. Interviews with nurses were analyzed by the same approach and categorized into 4 domains: general feedback, evaluation of training, evaluation of study activities, and feasibility of implementing this intervention within the existing structures of health system.
Results: Patients perceived improvement in their emotional health because of CONEMO, whereas some also reported better physical health. Many encountered some difficulties with using CONEMO, but resolved them with time and practice. However, the interactive elements of the app, such as short message service, android notifications, and pop-up messages were mostly perceived as challenging. Satisfaction with CONEMO was high, as was the self-reported adherence. Overall, patients evaluated the nurse accompaniment positively, but they suggested improvements in the technological training and an increase in the amount of contact. Nurses reported some difficulties in completing their tasks and explained that the CONEMO intervention activities competed with their everyday work routine.
Conclusions: Using a nurse-supported smartphone app to reduce depressive symptoms among people with chronic diseases is possible and mostly perceived beneficial by the patients, but it requires context-specific adaptations regarding the implementation of a task shifting approach within the public health care system. These results provide valuable information about user feedback for those building mobile health interventions for depression.
Mental health disorders are one of the main health burdens worldwide. Of the 20 leading causes of years lived with disability in the world, 9 of them are mental, neurological or substance use disorders ; neuropsychiatric disorders make a contribution of 13% to 14% to the global burden of disease, measured by the disability-adjusted life years (DALYs), because of the mortality and years of life lost derived from the time lived with compromised health [ - ]. In Peru, neuropsychiatric disorders are even considered as the ones that cause the highest burden of disease in the country, with a contribution of 16% to the DALYs [ ]. Furthermore, unipolar depression is the main cause of burden of disease among this group [ ] and is even higher among those with comorbid chronic noncommunicable diseases (NCDs), such as diabetes or hypertension [ - ]. Considering this high comorbidity, the great incidence of NCDs in low- and middle-income countries (LMICs) [ , ], as well as the important possible negative consequences of this relationship on health outcomes [ , , , - ] and adherence to treatment [ , ], interventions specifically designed for individuals with depression and comorbid diabetes, hypertension or both are needed.
Access to mental health services is poor across LMICs. For example, in Peru, most health insurance does not cover mental health care . Furthermore, there are far fewer psychologists and psychiatrists per inhabitant than in other upper- and middle-income countries, and of those who are available, 85% are located in Lima, the Peruvian capital [ ]. Only 24.3% of the population in Lima with mental health disorders receive any mental health care [ ], which is even less than 10.4% in rural Andean regions [ ]. One approach to confront system-related barriers for mental health care is task shifting [ - ]. Shifting tasks within health care from a specialized professional to a person not specialized in the same field, for example, from a psychiatrist to a comprehensively trained nurse or other health worker not specialized in mental health, is a cost-effective [ , ] and cost-saving approach [ - ]. Similarly, it has shown promising results for a wide variety of health outcomes, including mental health conditions [ , ] such as depression [ , ], and could therefore be an ideal option for LMIC settings to amplify the access to health care [ ] where human resources are scarce.
Other strategies to overcome health system barriers and make mental health care more accessible to the community [, ] are mobile health (mHealth) [ , - ] and other digital self-help interventions [ - ], which have been shown to improve mental health–related outcomes. There are several mobile depression apps already on the market, but in addition to stand-alone, unguided self-management apps, linking interventions to existing resources from local health care systems merits exploration. Furthermore, many of the depression apps currently on the market often lack a solid theoretical background or do not follow clinical guidelines that have already proven to be effective [ , ], and besides, most of them are not available in Spanish. Evidence from high-income countries indicates that the effectiveness of and adherence to digital interventions is higher when the health app is easy to use [ , , ], is individually tailored [ ], and includes human interaction [ , - ]. Hence, it is crucial to evaluate the users’ perspective and feasibility of using mHealth apps in the LMIC context. Although they are not ubiquitous yet, as pervasiveness of smartphones is continuously increasing in developing countries [ , ], mHealth apps will become more needed in the near future. In Peru, the access to smartphones increased from 14.8% to 30.8% among people aged between 46 and 50 years and from 6.8% to 18.0% in those aged 51 years or older in only 1 year (2014-2015) [ ], and it is expected that these percentages will continue to grow.
The Latin America Treatment and Innovation Network in Mental Health (LATIN-MH)  integrated these strategies and developed a new smartphone app, CONEMO (Emotional Control, in Spanish: Control Emocional), a technology-driven, psychoeducational, and nurse-supported intervention aimed at reducing depressive symptoms in people with diabetes, hypertension or both. This study is derived from the qualitative formative research conducted in Lima, Peru, in preparation for a randomized controlled trial testing the effectiveness of CONEMO. The aims of this study were to explore (1) the experience of patients using CONEMO with regard to perceived health benefit, usability, adherence to CONEMO, satisfaction, suggestions for CONEMO, as well as the evaluation of the nurse-support received and (2) the experience of nurses supporting CONEMO users regarding general nurse feedback, evaluation of training received, evaluation of activities related to the study, and their perception of the feasibility of using this type of self-management-plus-coaching approach within the Peruvian public health care system. The quantitative outcomes of the pilot studies will be reported elsewhere [ ].
Study Design and Theoretical Framework
This study analyzes the qualitative information acquired from interviews with patients and nurses about their experiences in 2 pilot studies. Interviews were selected as the most suitable technique, which were applied after their participation in these studies. The data were analyzed using qualitative content analysis, as, for example, described by Bengtsson , considering it as “a research method that provides a systematic and objective means to make valid inferences from verbal, visual, or written data in order to describe and quantify specific phenomena” [ ].
The 2 pilot studies were conducted between 2015 and 2016 in Lima; the first pilot study was implemented in 1 general hospital of the Ministry of Health (MINSA) and the second in 2 primary health care centers of EsSalud, Peru’s Social Security System. Those are 2 distinct public health care systems in Peru, which differ in their organization and functioning. The reason for conducting 2 pilot studies was, first, to pilot test the implementation in the 2 different national health care systems, and second, to test the feasibility of working with staff nurses from the health care system within this study instead of hired nurses, which was not possible in the first pilot study in the MINSA hospital.
The study integrated data from 2 groups of participants: the patient’s experience with CONEMO and the nurses’ experience of conducting the nurse-support of CONEMO within the health care system.
Using a convenience sampling strategy, patients were selected based on the following criteria: having a diagnosis of diabetes, hypertension or both by a physician or receiving treatment for it, being aged at least 21 years, being able to read and write in Spanish (all of the above were self-reported), and experiencing clinically significant depressive symptoms, as measured by the Patient Health Questionnaire-9 (cutoff score ≥10) . Patients who were pregnant or showed cognitive impairment (measured with the Brief Community Screening Instrument for Dementia [ ]) or psychotic symptoms (measured with the Psychosis Screening Questionnaire [ ]) were excluded.
In the first pilot study, implemented in a hospital of the MINSA, 1 nurse was hired by the project. In the second pilot study, nurses from the Social Security System were selected and assigned to the study by the health care centers’ administration. In 1 health care center, all 5 nurses who were working there at the time were assigned to participate in the study, whereas in the other health care center, 1 nurse who was working in the elderly adult program was selected.
Method of Approach
In the MINSA hospital, recruitment took place in the endocrinology and cardiology outpatient clinics, whereas in the 2 primary health care centers, patients were recruited in the waiting areas or in the elderly adult consultation unit that monitors people with NCDs. Some patients were approached before or after their regular consultations, whereas others were referred to the fieldwork team by health care providers. Patients were screened by 1 of 4 fieldworkers who were all psychologists. If the inclusion criteria were fulfilled, they were invited to participate in the 6-week study and to sign a consent form.
Afterward, patients were invited to return to the health care center to complete a baseline questionnaire. Subsequently, they were assigned to a nurse who then scheduled an individual appointment with each patient to train them and provide the study materials. As, at the time of the pilot, smartphones were not yet ubiquitous among low-income Peruvians, all patients were given an Android smartphone with CONEMO installed as well as 2 guidebooks: one about how to use the smartphone and CONEMO and one about the research project. The nurses trained the patients in the use of the technology and informed them that their activities could be followed on a Web-based platform and that they would receive at least 2 monitoring phone calls throughout the intervention period to address difficulties if applicable.
The nurses were approached via the centers’ administration. After the managers of the centers accepted to participate, they selected the nurses and facilitated the contact to the research team. Afterward, all nurses underwent a 3-month theoretical and practical training course with 1 to 2 hours of weekly training sessions to perform their activities before starting to receive patients.
Sample Size and Nonparticipation
Over the 2 pilot studies, 45 patients signed the consent form to participate, but 12 of them did not return to the health care center to receive CONEMO. Of the 33 patients who finally received CONEMO, 29 responded to the interviews (15 of pilot study 1, monitored by a hired nurse, and 14 of pilot study 2, monitored by staff nurses from the health care system).
With regard to the nurses, 6 staff nurses, who were working within the Social Security System and who participated in the second pilot study, were interviewed after finalizing the pilot study activities. The nurse hired in the first pilot study was not considered in the interviews, considering that she would not represent the perspectives of someone working within the health care system.
Intervention: CONEMO (Emotional Control)
CONEMO is a technology-driven, psychoeducational, and nurse-supported intervention that was delivered via a smartphone app (seefor a screenshot of CONEMO). A literature review was conducted to develop both the content, which was based on the theoretical framework of behavioral activation [ - ], as well as the design and structure of the CONEMO app.
Behavioral activation aims at reducing depression by motivating patients to identify and complete activities to obtain a sense of pleasure and accomplishment . In that sense, the content of the intervention included psychoeducation about depression, various lists of potential activities that patients could be interested in, motivational content to stimulate activity completion, as well as strategies to help patients complete these activities. These content elements were delivered in the form of both text and videos. The videos featured a middle-aged woman representing a health practitioner who did not embody any specific social class or background.
Providing a list of suggested activities to the participant was also used in other apps based on behavioral activation to lower depressive symptoms [, ]. Previous behavioral activation research suggested a list of 11 activity categories in which patients could start doing activities [ ]. Owing to the length of our intervention, those 11 domains were regrouped into 3 categories: pleasant activities (eg, based on hobbies and entertainment among others, such as reading a book or meeting friends), healthy activities (based on sports and health, such as following their indicated diet and exercising), and tasks (based on domestic activities, which included activities such as cleaning the house). In this process, activities were discussed and adapted by the research team to ensure the appropriateness of the listed activities, considering the target population’s health characteristics and culture. Finally, CONEMO included interactive elements, such as Android notifications and dialogue pop-ups to remind patients to complete their sessions and activities and let them give feedback, short message service (SMS) text messaging to remind them of their appointments with the nurse, as well as an option to request help from the nurse.
For the development of the structure of the CONEMO app, other behavioral activation intervention outlines, as well as existing mobile intervention designs, were considered. Although in-person interventions based on behavioral activation usually have a duration of 5 to 12 weekly sessions, with each session lasting approximately 45 min [, ], interventions using technology need to deliver the core elements of the treatment by adapting them to the interface used and not as a mere copy of the content [ ]. Other mobile interventions using behavioral activation were typically designed for a duration of 8 to 10 weekly sessions but were also accompanied with weekly psychotherapy sessions [ , ] or pharmacotherapy [ ]. Considering the nurse accompanying the CONEMO intervention would only give technical support and that app developers recommend designing interventions of “high frequency, low intensity, and shorter time commitment” [ ], it was decided to include 18 sessions in total, delivered 3 times a week over a period of 6 weeks. The completion time for each session (excluding the suggested outside-app activity) was estimated to take between 5 and 10 min.
The role of the nurses was to train all patients in the use of the app and the smartphone, followed by regular monitoring of the patients’ participation throughout the 6 weeks of intervention via a Web-based platform, the nurse dashboard. The nurse-support protocol was based on the principles of supportive accountability , with the aim of maintaining patient engagement. Nurses were instructed to contact the patient periodically to positively reinforce use, inquire if they had any difficulties, and resolve possible problems with using CONEMO. In addition, the nurse dashboard served as an activity and progress record, allowing nurses to log their activities, such as the completed phone calls or other contacts they had with the patients. The roughly estimated time expected to be invested per patient was around 2.5 hours in total over the 6-week period (45 to 60 min for the first appointment used for training, 15 to 30 min for the last appointment, and 15 min per remaining week for follow-up calls). Furthermore, during the intervention, all nurses took part in weekly supervision meetings with a psychologist, in which particular cases, tasks, and difficulties were addressed. These supervision meetings were expected to take around 15 to 45 min weekly.
The technology and intervention used in this study was created in collaboration with representatives from the Universidad Peruana Cayetano Heredia in Lima, Peru, and the University of São Paulo in São Paulo, Brazil. Northwestern University’s Center for Behavioral Intervention Technologies in Chicago, United States, supported the design work and provided all software programming. CONEMO was developed to be adaptable linguistically and culturally, providing versions in Spanish, Portuguese, and English.
After the 6 weeks of intervention, the trial manager (LRB) and the clinical coordinator (LH)—both female psychologists—conducted the interviews with the patients and nurses. During both the patient and nurse interviews, the interviewers aimed to transcribe as closely as possible the literal content of the participants’ responses and later transferred these notes to the computer to reduce information loss. During some of the patient interviews and all of the nurse interviews, both interviewers took notes simultaneously, compared them afterward, and found high literal consistency between the notes. Subsequently, the digitized notes were integrated into 1 document.
The patients’ semistructured interviews were applied, consisting of 30 questions addressing the 6 areas of interest, which were considered as most important indicators in view of the randomized controlled trial to be implemented subsequently: perceived health benefit, usability, adherence to CONEMO, satisfaction and suggestions for CONEMO, as well as the evaluation of the nurse-support received (seefor the interview guide for patients). Satisfaction refers here to the patients’ evaluation of things they liked and disliked about CONEMO. Each interview took between 45 and 60 min.
With the nurses, semistructured interviews consisting of 22 questions to retrieve data about general nurse feedback, evaluation of training received, evaluation of activities related to the study, as well as their perception of feasibility of incorporating CONEMO within primary care (seefor the interview guide for nurses) were conducted. Feasibility refers here to the nurses’ perceptions of the viability to scale up the nurse-supported CONEMO within similar health care centers and to be implemented by staff nurses. Each interview took around 40 min. All interviews were conducted in 1 session, without conducting repeat interviews.
The 4 stages of content analysis— decontextualization, recontextualization, categorization, and compilation —as described by Bengtsson , were conducted by 2 researchers—the ones who also conducted the interviews—to increase validity and decrease bias, as suggested by various authors [ , , ]. First, the digitized interview notes for patients, as well as for nurses, were reviewed, and thereby, codes were generated (decontextualization). Both researchers discussed their comprehension of each code and adapted them accordingly before actually coding the data. Then, they compared the codes with the original data to control for completeness of the codes (recontextualization). Codes were created for the complete dataset, so that every statement made by a person would be assigned to at least 1 code. The aim was to have at least 1 code for every statement, which would not overlap in its meaning with other codes. Therefore, initially, no information was considered dross [ , ]. After the codification process, the researchers revised the codification of the other person and discussed and adapted discrepancies to control for different interpretations and increase compatibility.
On the basis of this principle, the qualitative data from patients were first divided into 14 principal codes and 39 subcodes (seefor complete patient codebook). After a thorough review, data from 11 principal codes and 27 subcodes were assembled to the 6 domains of interest (categorization). The data from the remaining codes were considered as not relevant to the research aim and therefore as dross (recontextualization). provides a description of the principal codes and subcodes used to describe each domain. The qualitative data from the nurses were coded into 9 principal codes and 18 subcodes (see for complete nurse codebook), of which data from 16 subcodes over all of the 9 principal codes were selected for this analysis and subsequently categorized into the 4 domains of interest, which are described in .
Afterward, the data were analyzed within each of the domains (compilation). A manifest analysis technique [, , ] was chosen to describe the experience narrated by the informants. Although our analysis was based on qualitative content analysis, in order to draw conclusions from the information retrieved, elements of quantitative content analysis were also applied to receive information about how common certain perspectives were within this population, as suggested by other authors [ , , ].
The formative research and its materials were approved by the Institutional Ethics Committee of the Universidad Peruana Cayetano Heredia in Lima, Peru, as well as by the Data and Safety Monitoring Board of the National Institute of Mental Health in the United States, in accordance with applicable regulations. Informed consent was obtained from all the participants before participating in the study, after screening for fulfillment of inclusion criteria, and having conducted a thorough explanation of the study and all implications of participation. Patients were not paid, but they were reimbursed for transportation costs related to this study.
|Perceived health benefit of CONEMO||Perceived health benefit of CONEMO||Perceived benefit on psychological health, perceived benefit on physical health, and other benefits perceived|
|Usability||Usability of CONEMO||Difficulties with CONEMO usage, and help from others|
|Evaluation of smartphone||Difficulties with smartphone usage|
|Guidebooks||Revision of guidebooks, and utility of guidebooks|
|Interactive tools (notifications, dialogue pop-ups and SMS)||—a|
|Adherence to CONEMO||Adherence to CONEMO||Revision of sessions, performing activities, and difficulties (to perform activities)|
|Satisfaction with CONEMO||Satisfaction with CONEMO||General feedback, things most liked about CONEMO, things least liked about CONEMO, things most liked about the sessions, things least liked about the sessions, and using smartphones for intervention delivery|
|Suggestions for CONEMO||Design||Design of CONEMO|
|Duration and frequency||Duration of intervention, and frequency of sessions|
|Suggestions for CONEMO||Suggestions to improve CONEMO, and preference of how to receive information|
|Evaluation of nurse component||Evaluation of nurse component||Evaluation of training, quantity of contacts, evaluation of contacts, help requests, and suggestions for the nurse component|
aFor some principal codes, further subdivision did not seem to be beneficial; therefore, no subcodes were created.
|General nurse feedback||General experience||—a|
|Expectations||Expectations preimplementation, and fulfillment of expectations|
|Evaluation of study activities||Satisfaction with tasks|
|Benefits||Perceived personal benefits|
|Evaluation of training received||Evaluation of training received||Level of preparedness posttraining, missing subjects, and suggestions for training|
|Evaluation of activities related to the study||Evaluation of study activities||Initial appointments, monitoring calls, nonadherence calls, help requests, revision of nurse dashboard, registering tasks in nurse dashboard, supervision meetings, and difficulties|
|Feasibility of incorporating CONEMO in primary care||Feasibility of scaling up CONEMO||—|
|Incentives||Types of incentives desired|
|Suggestions for the project||—|
aFor some principal codes, further subdivision did not seem to be beneficial, therefore no subcodes were created.
Among the 29 patients, who responded to the interviews, the mean age was 60 years (SD 9.6, range: 47-85 years) and 69% (N=29) were women. Almost half of the patients were diagnosed with both diabetes and hypertension (45%, N=29), whereas 31% were diagnosed with solely diabetes and 24% with solely hypertension. Less than half of the patients, 45% (N=29), reported having experience in using a smartphone. For an overview of the demographic variables of the patients, see.
All nurses who participated were women, the mean age was 38 years (SD 6.2, range: 31-46 years) and the mean time working in the health care center where they were currently employed was 6.6 years (SD 5.3, range: 3-16 years). All nurses had higher education and were licensed in nursing, 1 additionally had a PhD degree, and most had previous experience with using smartphones (83%, N=6) or tablets (67%, N=6). In 1 of the 2 health care centers, 1 nurse monitored 7 patients, whereas in the other center, 5 nurses monitored 10 patients, 2 patients each, who participated in the second pilot study.
Patients’ Experience With CONEMO
Perceived Health Benefit of CONEMO
Almost all patients (93%; if not mentioned otherwise, the denominator for all frequencies in this paper is the total number of patients, N=29) said that CONEMO helped them emotionally. Most patients felt that after having used CONEMO, they were calmer, more cheerful, enthusiastic, motivated, and less worried and stressed out. Although more than half of the patients (59%) mentioned doing new activities or having picked up activities they have not been doing in a while, one-third (34%) attributed their improvement in emotional well-being to being more active. People did not only increase healthy activities, but also pleasant activities, such as reading a book, visiting friends and family, playing soccer with other people, or walking the dog. As 1 patient expressed it:
[CONEMO] has given me the opportunity to rediscover what we have in life.
[Patient 32474, female, 65 years, quote #1; see for original quotes in Spanish and previous smartphone experience.]
One patient explained that being more active distracts oneself from the negative thoughts and therefore improves psychological health.
Another important factor related to the perceived improvement was that CONEMO gave them the feeling of not being alone (34%). CONEMO was viewed as a conscious friend, as a personal psychologist, or just as someone that gave them advice, someone they did not have in their lives before, which made them feel not being forgotten about or abandoned, as they had previously assumed. Interestingly, 2 people (7%) also mentioned to have more self-esteem after having used CONEMO, as the following quote describes:
CONEMO, yes, it improved my emotional health because I felt that I mattered, that I have to allocate some time to myself and that no one matters more than me. I have been feeling better emotionally, freer. I felt that I did not have to depend on anyone and that I can do things for myself.
[Patient 12406, female, 51 years, quote #2, .]
Some patients also valued positively the study safety procedures—which were not part of the intervention itself—to motivate patients to look for professional help, transfer them to mental health professionals within the health care system, and in cases of severe depressive symptoms, even helping them to receive treatment faster than usual (14%). Some patients felt that this also contributed positively to their psychological health. However, 2 people (7%) thought that the time of the intervention was not sufficient to accomplish groundbreaking improvements and other 2 did not feel that CONEMO improved their psychological health.
As many activities suggested by the intervention were healthy activities that people with NCDs could complete, some of the patients (34%) also felt that CONEMO helped them improve their physical health. For example, some mentioned having lost weight because of their healthier diet or others to have gained mobility because of increased exercising.
In addition, the majority of the patients mentioned changing their daily habits toward a healthier routine (62%). Most of them reported exercising more, eating healthier, taking the bike to buy groceries, walking or running more, and taking their medication more regularly. One person also described that CONEMO motivated her to go the doctor, whereas she had previously tried to avoid all doctor consultations if possible.
In terms of other benefits perceived, 6 patients (24% of 25) described CONEMO as helping them to organize their everyday routine better to do different activities. Some (20% of 25) also mentioned that through CONEMO they were more conscious about their chronic condition and paid more attention to their health and their body. CONEMO has also changed some of the patients’ relationships with their families and friends (16% of 25), for example, by implementing CONEMO’s suggestion of exercising in form of weekly family volleyball games.
Most patients encountered some kind of difficulty in using CONEMO or the smartphone (72%) and would have liked to receive more training. Some patients had problems remembering the explanations received from the nurse (14%). Although about half of the patients admitted that they did not know how to handle the technology in the beginning (48%), most patients with difficulties (71% of 21) explained that those generally occurred in the beginning and that, with time and practice, it got easier to use CONEMO and the smartphone. Furthermore, 14 patients (48%) mentioned having received help from others, such as family members, in using CONEMO. The great majority of the patients reported reviewing the guidebooks provided by the research team (90%) and found them helpful (74% of 23).
The specific difficulties with CONEMO and the smartphone mentioned by the patients are described inand , respectively. Regarding CONEMO, most difficulties were experienced when selecting a time and date for scheduling an activity by changing the numbers using up and down arrows. Concerning difficulties using the smartphone itself, patients mostly mentioned the fear of getting it stolen.
Furthermore, many people experienced difficulties with the interactive tools used in this intervention, such as SMS, Android notifications, and dialogue pop-ups. The SMS served as a reminder for the appointments within this study, Android notifications informed patients that a new session had arrived, and dialogue pop-ups were designed to retrieve feedback about the completion of the activities through messages appearing on the screen soliciting a response from the patient. There were some technological and connectivity problems, for example, people did not receive the SMS in time. However, most people did not view or use the Android notifications (51% of 24) and opened CONEMO directly from the home screen (71% of 24) to see if new sessions had arrived, and some people did not know how to open the SMS or could only read them with the help of others (21% of 24). The idea behind the dialogue pop-ups—to give feedback about activity completion—was evaluated positively; however, some did not recall them (25% of 24) when asked about them.
Adherence to CONEMO
This section refers to the self-reported adherence to CONEMO in the interviews. Another publication by the LATIN-MH team addresses the quantitative adherence data of the pilot studies . Adherence to CONEMO is defined here as (1) the amount of CONEMO sessions reviewed and (2) the amount of activities completed that were suggested by CONEMO. Regarding adherence to session review, around half of the patients who talked about it said they not only read all of the sessions of CONEMO (65% of 20) but also re-read numerous sessions (50% of 20). In addition, 2 people even re-read the whole intervention again before returning the smartphone to the research team. Some explained that they re-read sessions to better remember the content or because it served them as a reminder to do certain activities, especially in between sessions or at the end, when there were no new sessions available. On the contrary, some people also disclosed that they did not complete all the sessions (30% of 20). In terms of the adherence of activity performance, 9 people (39% of 23) indicated to have done all activities they selected during this intervention and other 9 (39% of 23) completed most of the activities. Thus, the great majority seemed to have been very adherent to the intervention. The activities with the highest completion rate were pleasant activities, such as going out meeting friends and family, and healthy activities, for example, going for a walk, whereas pending tasks, such as organizing the bills or paying them, seemed less frequent to be done. However, 5 (22% of 23) people did few or none of the activities.
The main barrier reported to performing the activities suggested by CONEMO was the users’ health status (36% of 14), such as limited mobility or memory problems. Time (29% of 14) and economic constraints (21% of 14) were also important barriers to doing activities. Other difficulties mentioned were the low motivation to do activities (9% of 23) as well as sudden changes in their plans because of external factors (14% of 14), for example, receiving a phone call or someone else needing assistance.
Satisfaction With CONEMO
Satisfaction refers to the statements that patients made about what they liked and disliked about the app and the sessions, comments that could infer satisfaction, as well as the patients’ perception about using technology for intervention delivery.
The majority of patients expressed great satisfaction with CONEMO (96% of 24), some did not make any statements regarding their satisfaction (17% of 29), and 1 was unsatisfied (4% of 24) because he felt that his health status was too compromised to follow all of the instructions. When the patients were asked what they liked most about the CONEMO app and its sessions, most appreciated the information and advice received (37% of 27), for example:
[…] its important guidelines [to do things one step at a time] instead of doing everything at once.
[Patient 32433, female, 60 years, quote #3, ]
Furthermore, 26% (of 27) of patients most liked the videos incorporated in CONEMO, 15% (of 27) most appreciated that they were reminded by CONEMO to follow their diet, to work out or follow their medication, and 11% (of 27) valued most the types of activities offered, as described by one of the patients:
In the [list of] activities there were things that caught my attention and that I would be able to do. This opened a whole new field for me of other activities I could do and that I had not done before.
[Patient 32444, female, 64 years, quote #4, ]
Furthermore, 2 patients even transcribed the whole intervention on their computer to be able to re-read it again after giving back the smartphone. Of note, one-third (33% of 27) of the patients especially valued being accompanied by CONEMO and the nurse and receiving monitoring phone calls from the research team, which were implemented as a safety procedure within the research rather than a treatment activity. These contacts transmitted a feeling of being cared for, of someone taking a special interest in them and their health, and of trusting them with a technological device:
I didn’t have anyone to talk to, so I watched the woman in the video, […].
[Patient 12266, female, 76 years, quote #5, ]
I was surprised and thought “what did I do to deserve something like this?”. And in addition, they gave me 10 Soles (Peruvian currency) to cover my transport. [...] This is the first time in my life that I have received this kind of attention. [...] No one has ever been concerned about me, but now there was someone there, who was worried about my health [...].
[Patient 22131, male, 70 years, quote #6, ]
Many people (54% of 28) did not spontaneously express negative aspects about CONEMO, although it was specifically asked for. The statements that were made were very diverse, and there was not much consensus (see).
Most of the patients (95% of 21) expressed a positive attitude toward using smartphones to deliver the intervention, and 48% (of 21) of people mentioned explicitly the advantage regarding the mobility, whereby the use of the smartphone replaced having to go to the health care center to make an appointment or to see a specialist. This potentially reduces the time spent on health consultation and could “unclog the health care center’s environment ” (Patient 22116, female, 52 years,, quote #7), as 1 patient explained. In addition, 4 patients (19% of 21) also highlighted the practicality of being able to take the intervention with them, to re-do the sessions, and especially being able to choose the place and time to follow the intervention:
It is convenient, because we are not as close to the doctor or psychologist… Here, it does not matter, where you are, you can still use it.
[Patient 32433, female, 60 years, quote #8, ]
Suggestions for CONEMO
The suggestions that patients expressed to improve CONEMO were wide ranged and could be grouped into 2 categories: (1) suggestions for the content and design of the app and (2) communication with the research team.
Regarding suggestions for the content and design of the app, most patients suggested increasing the duration of the intervention (68% of 28). The other suggestions are displayed in. In terms of preference of information delivery, if given the option, most people would prefer to be able to hear the content read to them via an audio (43% of 28) or to have both audio and text (36% of 28), whereas others prefer to only read the text (21% of 28).
In terms of the communication with the research team, patients also desired more. Some patients suggested adding group activities with all the patients and trainings or lectures of different topics (14% of 28). Others would have liked to receive personal face-to-face appointments with the research team for depression monitoring instead of phone calls or specialized psychological care via telephone or audio messages (11% of 28).
Evaluation of the Nurse Support
Most patients were satisfied with the nurses’ explanations of how to use CONEMO (59%); however, some patients mentioned that it was still difficult to understand (14%) or that they were not able to use CONEMO at first, directly after the training (17%). Various people said something similar to the following:
The nurse explained well, it is just that maybe I am stupid or something, because when I got home, I did not know how to use it.
[Patient 12073, male, 66 years, quote #9, ]
However, some of the patients who were accompanied by a nurse from the health system perceived the nurse as being in a hurry during the training (10%) or without interest in explaining the intervention (7%). Comparing both pilot studies, 80% (of 15) of the patients who were accompanied by the hired nurse viewed the training session as useful, with a positive knowledge transfer, whereas only 50% (of 14) of patients monitored by a staff nurse expressed either positive or neutral comments about the quality of the training.
On the basis of the study protocol, all patients should have received at least 2 phone calls by the nurse to see if there are any difficulties with CONEMO and additional calls depending on their adherence and difficulties. Evaluating the quantity of contacts, when working with a hired nurse, most of the patients reported having received between 3 and 4 phone calls from the nurse (53% of 15) during the intervention, whereas most patients accompanied by staff nurses reported to have received 1 to 2 calls (36% of 14; see).
Most of the patients in both pilot studies viewed the phone calls as positive (81% of 26) because the nurses helped them to be adherent to the intervention (19% of 26), helped them with their difficulties using CONEMO (19% of 26), paid attention to their health (15% of 26), helped with other problems (8% of 26), and showed the patients that she cared (15% of 26).
Through help requests patients could solicit tech support from the nurse. Most of the patients stated they did not use the help request button within CONEMO (68% of 28), some because they claimed to not have needed it and others because they felt embarrassed, did not remember this function, or thought the nurse would be busy anyway. Some patients also asked for help by error (7% of 28). In addition, 4 people (14% of 28) expressed that they did ask for help and received it, whereas other 3 people stated that they had pressed the button but that no one had called them.
Regarding suggestions for improving the nurse-support, almost half of the patients would have liked to receive more phone calls (44% of 27), and ideally a space to talk about their emotional state, instead of only receiving technical support from the nurse. Others suggested amplifying or improving the quality or depth of the training to feel more confident in using CONEMO (11% of 27).
A summary of the main results retrieved from the patients’ interviews can be found in.
|Perceived health benefit of CONEMO||Almost all patients perceived improvements in their mental health and some also in their physical health after using CONEMO.|
|The majority felt more active after using CONEMO, having mostly increased pleasant and healthy activities.|
|CONEMO was also viewed as a companion, which made them feel less alone.|
|Usability||Initially, most patients encountered difficulties in using CONEMO or the smartphone and with using SMS, Android notifications, and dialogue pop-ups; however, many technological issues also emerged around those elements, which is why the distinction of those two is not completely clear.|
|Some patients received help from family members or reviewed the guidebooks, and most patients felt that these difficulties subsided with time and practice.|
|Adherence to CONEMO||Self-reported adherence was high, most patients completed all sessions and the majority completed most or all activities advised by CONEMO.|
|The most important barriers to completing the outside-app activities were the patients’ health status, time, and economic constraints.|
|Satisfaction with CONEMO||Satisfaction with CONEMO was high, as it was with using smartphones for intervention delivery.|
|Features most appreciated were the advice provided, the videos and types of activities suggested, as well as the monitoring calls received by the nurse.|
|The few critiques were directed at the type of activities suggested by CONEMO and that the sessions were repetitive.|
|Suggestions for CONEMO||Patients suggested increasing the duration of the intervention, the amount of videos and frequency of sessions and improving the in-build training session.|
|Many patients also desired a more frequent interaction with the nurses.|
|Evaluation of nurse component||Although most praised the explanation from the nurses, some still experienced difficulties afterward, and therefore, they suggested improving the training.|
|The monitoring calls were viewed as positive and helpful to increase adherence and resolve difficulties.|
Nurses’ Experience Supporting CONEMO Users
General Nurse Feedback
Most nurses (5/6) felt that CONEMO was an innovative and useful intervention. They thought, it was a good idea, something different, an opportunity to talk more to the patients and something beneficial to them. Some nurses reported that participating in this project benefited themselves as well because they took part in improving the patients’ health (3/6) and gained experience participating in a research project (1/6). However, most nurses (5/6) mentioned having had difficulties in consolidating their CONEMO activities with their workflow and felt that this increased their workload. The difficulties mentioned were related to the fact that before implementation, the health care centers’ management agreed to provide the nurses with reserved hours to participate in the intervention; however, because of logistical and administrative constraints, once the study started, the nurses had to accommodate these additional activities in their daily routine without adjustments to their existing workload, as 1 nurse describes:
At the beginning, I found it tedious, because we did not have protected hours for this task. I took on more [tasks], because I had more free time. [...] When the [chief] doctor saw that we were having trouble, she said that when we were in the clinic, we should ask the auxiliary nurse to cover for us for a moment [to do our tasks related to CONEMO].
[Nurse 306, female, 38 years, quote #10, ]
All nurses felt strongly that the implementation of CONEMO including nurse-support must imply strictly reserved work hours to be executed as expected.
Evaluation of Training Received
Most nurses (5/6) stated that the training they had received was appropriate. However, 2 of them felt they were not able to remember all the procedures when the study started because of a lack of practice or a lack of concentration during the training sessions.
[I learned] about 80% [of the procedures] because it was difficult to stay focused during the training, due to the number of tasks that we had [to do] in the health care center.
[Nurse 304, female, 33 years, quote #11, ]
Regarding the study technology, 3 of the 6 nurses reported that it was more difficult to use the nurse dashboard in the tablet to monitor their patients than to learn and explain how to use CONEMO, and they recommended increasing the time to practice during the training. Of note, the training of the hired nurse was a lot less time intensive compared with the training of the nurses from the public health care system because attendance fluctuated and it was difficult to gather everyone at the same time, maintain their concentration and motivation, and have sufficient time per training session to conduct complete practice runs before the nurses had to leave again.
Evaluation of Activities Related to the Study
To complete their tasks, nurses were supposed to use the nurse dashboard to view and log their tasks as well as monitor their patients. Only 2 nurses revised the dashboard regularly, every other day. The other 4 only revised the dashboard 2 or 3 times during the 2 to 3 months they were monitoring patients because they did not have enough time (3/6), they forgot (1/4), or because the tablet was not always accessible to them (1/4) as it was stored in the director’s office.
One of the most important responsibilities of the nurses was the initial appointments in which they trained patients on how to use CONEMO. Most nurses (4/6) emphasized that their patients had some difficulties in understanding the technology, but that they were able to explain it again afterward or that the patients received help from their family members. Some nurses (2/6) had difficulties during the training because they themselves forgot how certain things worked, and another 1 reported time constraints that forced her to complete it as fast as possible.
Nurses were also supposed to complete monitoring calls; ask their patients if they had difficulties in using CONEMO; motivate them, if needed, to increase adherence; and answer help requests. Half of the nurses made the calls by their own initiative, whereas the other 3 did not complete the calls unless they were reminded to do so by the clinical supervisor. In addition, some of the nurses had great difficulties in reaching some of their patients (2/3).
Supervision meetings led by a psychologist were supposed to be conducted on a weekly basis. In 1 health care center, the nurse was very comfortable with the supervision because she felt the research team addressed her doubts. However, at the other health care center, nurses felt pressured to have those supervision meetings (4/5) mainly because they felt overwhelmed because of their workload:
You, [the research team], were not the problem, but the work burden we have here. We just could not do it, it was pretty uncomfortable because we did not have the time to receive you as we should have. [...] It felt like “another thing” that I have to do.
[Nurse 304, female, 33 years, quote #12, ]
Feasibility of Incorporating CONEMO in Primary Care
Considering the possibility of scaling up CONEMO plus nurse-support, all nurses identified the lack of time to perform the related activities as the main problem for its implementation within the health care system. All nurses considered it necessary that the activities of the nurse-support would be included as part of the monthly schedule and paid work hours, a decision that depends on the administration of the Social Security System:
You would have to separate [CONEMO] as an independent program because here each nurse is responsible for one program. It cannot interfere with other activities.
[Nurse 301, female, 38 years, quote #13, ]
|General nurse feedback||The intervention was viewed as useful, beneficial, and an opportunity to talk more to their patients. However, most nurses had difficulties to consolidate the study activities with their daily work routine.|
|Evaluation of training received||The training was viewed as appropriate, although most nurses had some difficulties because of lack of concentration and fluctuating attendance.|
|Evaluation of activities related to the study||Most nurses experienced some difficulties conducting their tasks reliably, especially concerning the review and registry of activities in the dashboard, conducting monitoring calls, and participating in supervision meetings.|
|Feasibility of incorporating CONEMO in primary care||The activities related to CONEMO would have to be part of their paid work hours instead of being additional tasks. Some nurses also considered incentives as a possibility to increase the feasibility to scale up.|
One factor related to this issue, which inhibited the nurses from being adherent to the intervention, was their job insecurity (5/6): Each nurse is required to meet a productivity goal and if it is not achieved, their employment contract may not be renewed again.
When nurses were asked if incentives could improve their performance regarding to CONEMO activities, 3 of 6 nurses stated that they would consider incentives, such as a course or regular meals; however, one of them explained that her main interest in participating was based on the perception that CONEMO benefits her patients. Other 2 nurses said that an incentive might motivate their colleagues but they themselves are uncertain if they would be part of the CONEMO nurse-support again because of their personal time constraints, and 1 stated she would not participate in the intervention again, even if she would get paid.
The results for the nurses are summarized in.
In this study, we aimed to explore the experience of primary care patients and nurses with CONEMO, a technology-driven, nurse-supported intervention to reduce depressive symptoms in people with diabetes, hypertension or both, to derive lessons informing future adaptations, and to assess the feasibility of using this technology in LMIC settings. Almost all patients felt that CONEMO helped them to improve their mental health by being more active, increasing their self-esteem, and making them feel valuable. Many patients also experienced improvements in their physical health related to healthier diets and more physical activity. The self-reported adherence to this intervention was relatively high. Although most people reported some difficulties with the technology in the beginning and suggested amplifying the training, the majority felt that it got easier with time. Using smartphones for intervention delivery was generally seen as practical and time saving. The main recommendations were to include more sessions and videos, to increase personal contact, and to make it more personalized. Most patients viewed the nurse-support as positive, although the evaluation of the support of the hired nurse was more frequently positive than of the staff nurses. The majority of staff nurses participating in the pilot study felt that the intervention was valuable for patients but that the activities of CONEMO conflicted with their daily work routine.
Reflections and Comparison With Previous Work
A growing body of research is aimed at capitalizing on the expansion of technology and its potential to introduce newer vehicles to target mental health, suggesting potential effectiveness of Web-based, text-messaging, and telephone support interventions . However, few studies are being conducted in LMIC settings [ ], which makes our studies highly relevant.
Multimorbidities, defined as having 2 or more diseases, including NCDs and mental conditions, constitute a growing health threat, especially in LMICs. However, most available treatments usually only focus on 1 health condition instead of approaching multimorbidities in a more integrative way [, ]. Therefore, cost-effective treatments that address multiple conditions are highly needed, and future research should be aimed at testing the effectiveness of mHealth interventions targeting multiple conditions. Nevertheless, in technological interventions designed for people with multiple health conditions, such as CONEMO, special consideration needs to be put on the specific target population. For example, one challenge, especially in behavioral activation interventions, could be balancing the activity suggestions, considering the demand of physical mobility to complete them, so that people with more compromised health statuses can still participate, without feeling frustrated. In general, but especially in LMIC settings, economic constraints also have to be considered when creating activity selections. Other studies have also recommended that researchers be aware of additional challenges of people with chronic conditions, such as limited mobile literacy [ ], which is in line with our findings. Although our study has shown satisfactory results, usability could be optimized by improving and amplifying training, considering the large proportion of people in this population, who are unfamiliar with the devices.
Although most people found the guidebooks useful and argued that the nurses generally explained well how to handle the technology, the majority reported some kind of difficulty in the beginning and suggested extending or improving the training. Although many difficulties will most likely subside as smartphones become ubiquitous, some changes to the design of CONEMO could also improve usability. Although most patients did not provide more information about how the training or design could be improved, one hypothesis was that a more realistic demonstration (using an in-build training session close to the actual CONEMO sessions) as well as a training video could be helpful to both nurses and patients. This way, the training would be more structured and more standardized over the nurses and, furthermore, would give the patient the opportunity to practice twice, while still being with the nurse: first, when conducting the training session and second, when completing the first real session. Nevertheless, in this study, as well as in others [- ], there seems to be a natural learning curve, which for most patients resulted in the ability to use CONEMO. For some people, this learnability curve was intensified by receiving help from other family members. This suggests, on one hand, that reading about how to use the smartphone app seems to be less effective in this population than hearing the explanations from other people and receiving demonstrations of how to do it and, on the other hand, that friends or family members frequently become involved to some extent in the intervention delivery.
Although this intervention was created specifically for people with diabetes, hypertension or both, some of the results could also be relevant for other mental health interventions with different populations. Other studies suggest that key features of acceptable smartphone health apps seem to be self-help suggestions and feedback to the patients’ responses, they should be easy to use (foolproof), and not take too much time (maximum 10 min per single usage) [, ]. A study by Baker et al [ ] established 16 recommendations for developing mental health smartphone apps, of which 12 seem to apply to CONEMO, for example, encouraging non-technology-based activities, recording the patients’ behaviors, having a log of patients’ app use, applying reminders to engage them, and using a simple and intuitive interface. Other studies from high-income countries also indicate that the effectiveness, motivation, or adherence is higher when the health app is easy to use [ , , ], is individually tailored [ ], and includes human interaction [ , - ]. Although most studies regarding depression apps where completed in high-income countries [ ], our results appear to be in line with such previous findings.
Nevertheless, some of our interactive features showed low utility, and some technological problems were encountered. For example, most people did not check the Android notifications to see if a new session was available. It is plausible that those notification symbols were too small for this population, considering their average age. Another hypothesis could be that considering most people were novice users, who had not yet integrated the smartphone into their everyday lives and, hence, only used the smartphone for one app—CONEMO—they would not be interested in other Android notifications of the default apps already installed on the smartphone. Therefore, it might have appeared easier to open the one app of interest directly from the home screen. The utility of the other interactive elements, such as SMS and dialogue pop-ups, was difficult to evaluate given the technological problems experienced. Considering that those difficulties were not encountered during the testing phase and some could be related to signal changes in the field, formative research for large studies including technological platforms appears to be of high importance.
Another important consideration about health apps is related to the setting with special emphasis on crime rate. Given the perception of insecurity and fearing a possible robbery, many patients were reluctant to borrow a smartphone from the research team. This fear could be reduced by explaining that in case of a robbery patients would not be charged for the phone. However, as a consequence, many people did not use it as a mobile device, outside of their homes. This suggests that the socioeconomic conditions related to security can have an effect on the user’s experience with mHealth interventions. Nevertheless, after concluding the pilot studies, only 1 smartphone was stolen and 1 was not returned, constituting a loss of 6%, which was a lot lower than expected. Considering scalability, installing CONEMO on the patients’ own phones if available and, hence, reducing the perceived responsibility for a borrowed phone could be one way to reduce this reluctance besides not holding patients accountable for a possible robbery or loss of the smartphone and covering those costs.
Evidence shows that in technology-only interventions, where people are left alone with mobile self-help interventions, participants are less adherent [, - ] and less motivated to engage in the program [ ] than people who are accompanied by professionals or coaches or have some other kind of face-to-face interaction as part of the intervention. In this formative research, the nurse embodied this role. The majority of patients reported being highly satisfied with CONEMO, they valued the advice and information provided, but some patients especially appreciated the attention received within the study. This was not anticipated, considering that the nurses’ role was based on giving technological support, but apparently, their accompaniment was perceived as beyond that role. In line with this finding, other studies emphasized that supportive accountability increases the adherence to any intervention and the extent of influence is characterized by “the bond between the user and supporting individual, the sense of accountability, and the user’s perception of the supporting individual as possessing legitimacy and expertise” [ ]. It is plausible to assume that the combination of technology with an in-person monitoring in these studies also influenced the slightly higher self-reported adherence rates of 65% to 78% and the low dropout rate of 12%, when compared with other similar studies, which found average adherence rates of 50% to 70% and dropout rates of 1% to 57% [ ].
In addition to the nurse-support, CONEMO contained an intermediate feature between technology and a personal component, which is relevant to consider: the videos in the CONEMO sessions were narrated by a real person, who was maintained throughout the intervention. Although patients never met this person, some people felt a connection to her and imagined she was the face behind the technology, who talked directly to them, who monitored them, and who had a leading role steering this intervention. Therefore, some people felt a personal relationship to the woman in the videos.
The importance of the videos was also highlighted by the patients’ suggestion to incorporate more of them. However, when designing this type of intervention and considering the number of videos, developers need to prioritize and contemplate the size of the videos (especially in a fully functioning offline app), which influences the speed of installation versus, for example, relying on a stable internet connection to watch videos on the Web. Those technological considerations need to be in line with the research priorities.
Feasibility of working with nurses within the health system to monitor the patients was low because of the conflicting use of time with their daily activities. Especially given their job uncertainty, many nurses felt pressured. Other studies in a very similar context also identified workload and time constraints as the most important barriers to implementation, especially taking into account that the studies’ activities were additional to their existing tasks, without considering more paid time . Therefore, when scaling up these types of task shifting interventions within the public health sector, specific resource allocation, such as workflow adaptations or incentives, have to be considered to overcome those barriers. It is also crucial to conduct cost-effectiveness studies and evaluate potential cost-savings to address organizational-level interests in implementing technology-based interventions and make correlated resource allocation more appealing to health care systems.
Adherence to the study activities can also be influenced by other factors specific to the health care center settings. For example, in our study, adherence seemed to be to some extent related to the attitudes of the nurses toward the study (positive attitude–higher adherence, and vice versa). This, in turn, could have been influenced by the management style of the health care center; the 5 nurses from 1 health care center who monitored 2 patients each were all assigned by their superiors to this study, although most were initially reluctant to participate. The one nurse with the highest workload—monitoring 7 patients—who had experience participating in different health programs, was asked by her superior beforehand, and she had the most positive attitude toward the intervention, was highly motivated, and ultimately was less affected by the workload of CONEMO. Therefore, although some health care center managers were initially enthusiastic about the study, important factors, distinct for each health care center, such as staff management, work climate, and conditions, could also influence implementation success. Furthermore, although it is ideal to find motivated nurses enthusiastic with this intervention, this may not be feasible across all of Lima or Peru, presenting a challenge to scalability; thus, structured resources should be put in place.
LATIN-MH benefited from this detailed formative phase conducted across 2 distinct health care provider systems in Lima, a large Latin American capital. Conducting qualitative formative research stages to shape mental health interventions has also been beneficial for other studies . In our case, this approach set the foundation for the modifications and adaptations of the intervention to build an app tailored to the patients’ needs before deploying a complex intervention such as CONEMO in a large randomized controlled trial, as recommended by other studies [ , , ].
Although this study has provided important insights in the evaluation of an mHealth intervention for people with NCDs and depressive symptoms, this study also had some limitations. For example, although all patients were provided with the same intervention and number of sessions, the intervention uptake might not have been the same for all of the patients, depending on their adherence to review the sessions. However, the number of patients included in this study was sufficient to signal positive and negative experiences with the CONEMO intervention . Therefore, we feel confident that we have sufficient information to improve the future design of CONEMO for the clinical trial.
Another limitation of this study is related to the fact that for the first pilot study in the MINSA hospital, a nurse was hired, whereas in the second pilot study in the Social Security System, nurses from the health system accompanied the patients. Therefore, although the perspectives of patients from both public health systems were obtained, only the perspectives of the nurses within the Social Security System were analyzed. Furthermore, it is plausible that the patients’ experience differed depending on what type of nurse accompanied them throughout the intervention beyond what has been identified and described here. However, conducting this second pilot study with staff nurses was extremely beneficial for the study purposes to explore feasibility of working with staff nurses and understanding the challenges they encounter, which might be similar in other public health systems. Therefore, after the first experience, it was crucial to conduct a second pilot study with this aim.
Furthermore, the interviews were not recorded and, therefore, there is a possibility that some information from the participants was lost. However, the 2 people who conducted the interviews aimed at writing down the verbal statements literally and took notes simultaneously during some of the interviews, which were contrasted afterward and which showed high literal consistency. This makes us confident that the notes represent the participants’ statements reliably.
Although generalizability is limited, all of these findings are relevant and crucial to inform the development of future complex interventions for mental health conditions, multimorbidity, and specifically for further designing and testing the efficacy of CONEMO under more controlled designs; indeed, CONEMO will be tested in 2 parallel randomized controlled trials. Other studies have demonstrated that mobile apps can effectively change behavior; however, considering the limited evidence, uncertainty of long-term effects and the predominance of those studies in high-income countries so far , the randomized controlled trial will give important insights regarding this type of interventions in LMIC settings.
Furthermore, this study is especially relevant considering the current mental health reform in Peru, where for the last 12 years, several efforts have been made aimed at improving the availability of free and universal access to mental health care for Peruvian citizens through a community-based approach, including primary health care centers [, ]. In this effort, tasks are already being shifted toward personnel not specialized in the same field, integrating mental health care in a variety of different medical settings. This development creates an opportunity for scaling up cost-effective mHealth interventions with task shifting components in the public health care system.
Smartphone apps constitute a potentially cost-effective opportunity in LMIC settings to overcome health system barriers and extend mental health care to large populations. On the basis of the experiences and opinions of the patients, it seems feasible to use this nurse-supported mHealth intervention for people with chronic diseases and comorbid depressive symptoms in Peru. CONEMO is perceived as helpful in improving mental health, but it requires context-specific adaptations, especially regarding the implementation of a task shifting approach within the public health care system; working with staff nurses using a task shifting approach within the public health system in Lima will only be feasible if the nurses’ time is protected for the program. Findings from this study will provide important information to develop a larger study focused on testing the effectiveness of this program on patients’ health and mental health outcomes.
This study was supported by the National Institute of Mental Health (1U19MH098780).
PRM, RA, JJM, and DM conceptualized the LATIN-MH program of research, for which this was a substudy of the formative phase. FDC and LRB designed this substudy and LRB and LH conducted it (LRB as trial manager and LH as clinical coordinator), under supervision of FDC. LRB and LH created the codebook and codified the interviews. LRB analyzed the data and wrote this paper with guidance from JJM and FDC and contributions from LH. After the first draft, PRM, RA, JJM, DM, and FDC reviewed the paper and contributed to the final version submitted.
Conflicts of Interest
Multimedia Appendix 1
Screenshot of CONEMO.PNG File, 56KB
Multimedia Appendix 2
Interview guide—patients.DOCX File, 16KB
Multimedia Appendix 3
Interview guide—nurses.DOCX File, 15KB
Multimedia Appendix 4
Codebook—patients.DOCX File, 16KB
Multimedia Appendix 5
Codebook—nurses.DOCX File, 14KB
Multimedia Appendix 6
Demographic variables—patients.DOCX File, 16KB
Multimedia Appendix 7
Original quotes in Spanish with demographic information.DOCX File, 17KB
- World Health Organization.: WHO; 2013. Investing in mental health: evidence for action URL: http://www.who.int/iris/handle/10665/87232 [accessed 2019-05-22] [WebCite Cache]
- Prince M, Patel V, Saxena S, Maj M, Maselko J, Phillips MR, et al. No health without mental health. Lancet 2007 Sep 8;370(9590):859-877. [CrossRef] [Medline]
- Collins PY, Patel V, Joestl SS, March D, Insel TR, Daar AS, Scientific Advisory Board and the Executive Committee of the Grand Challenges on Global Mental Health, et al. Grand challenges in global mental health. Nature 2011 Jul 6;475(7354):27-30 [FREE Full text] [CrossRef] [Medline]
- Raviola G, Becker AE, Farmer P. A global scope for global health—including mental health. Lancet 2011 Nov;378(9803):1613-1615. [CrossRef]
- Velásquez A. [The burden of disease and injuries in Peru and the priorities of the essential plan of universal insurance]. Rev Peru Med Exp Salud Publica 2009;26(2):221-231 [FREE Full text]
- Moussavi S, Chatterji S, Verdes E, Tandon A, Patel V, Ustun B. Depression, chronic diseases, and decrements in health: results from the World Health Surveys. Lancet 2007 Sep 8;370(9590):851-858. [CrossRef] [Medline]
- Egede LE. Major depression in individuals with chronic medical disorders: prevalence, correlates and association with health resource utilization, lost productivity and functional disability. Gen Hosp Psychiatry 2007 Sep;29(5):409-416. [CrossRef] [Medline]
- Pitt B, Deldin PJ. Depression and cardiovascular disease: have a happy day--just smile!. Eur Heart J 2010 May;31(9):1036-1037. [CrossRef] [Medline]
- Golden SH, Lazo M, Carnethon M, Bertoni AG, Schreiner PJ, Diez RA, et al. Examining a bidirectional association between depressive symptoms and diabetes. J Am Med Assoc 2008 Jun 18;299(23):2751-2759 [FREE Full text] [CrossRef] [Medline]
- Rotella F, Mannucci E. Depression as a risk factor for diabetes: a meta-analysis of longitudinal studies. J Clin Psychiatry 2013 Jan;74(1):31-37. [CrossRef] [Medline]
- Anderson RJ, Freedland KE, Clouse RE, Lustman PJ. The prevalence of comorbid depression in adults with diabetes: a meta-analysis. Diabetes Care 2001 Jun;24(6):1069-1078. [Medline]
- Davidson KW. Depression and coronary heart disease. ISRN Cardiol 2012;2012:743813 [FREE Full text] [CrossRef] [Medline]
- Perel P, Casas JP, Ortiz Z, Miranda JJ. Noncommunicable diseases and injuries in Latin America and the Caribbean: time for action. PLoS Med 2006 Sep;3(9):e344 [FREE Full text] [CrossRef] [Medline]
- World Health Organization. 2011. Global Status Report on Noncommunicable Diseases 2010 URL: http://apps.who.int/iris/bitstream/handle/10665/44579/9789240686458_eng.pdf;jsessionid=ACD8C60841CC669784D12CDC99E0D9C3?sequence=1 [accessed 2019-05-22] [WebCite Cache]
- Huffman JC, Celano CM, Beach SR, Motiwala SR, Januzzi JL. Depression and cardiac disease: epidemiology, mechanisms, and diagnosis. Cardiovasc Psychiatry Neurol 2013;2013:695925 [FREE Full text] [CrossRef] [Medline]
- Andreoulakis E, Hyphantis T, Kandylis D, Iacovides A. Depression in diabetes mellitus: a comprehensive review. Hippokratia 2012 Jul;16(3):205-214 [FREE Full text] [Medline]
- Lin EHB, Rutter CM, Katon W, Heckbert SR, Ciechanowski P, Oliver MM, et al. Depression and advanced complications of diabetes: a prospective cohort study. Diabetes Care 2010 Feb;33(2):264-269 [FREE Full text] [CrossRef] [Medline]
- Barth J, Schumacher M, Herrmann-Lingen C. Depression as a risk factor for mortality in patients with coronary heart disease: a meta-analysis. Psychosom Med 2004;66(6):802-813. [CrossRef] [Medline]
- Toyama M, Castillo H, Galea JT, Brandt LR, Mendoza M, Herrera V, et al. Peruvian mental health reform: a framework for scaling-up mental health services. Int J Health Policy Manag 2017 Jan 22;6(9):501-508 [FREE Full text] [CrossRef] [Medline]
- Instituto Nacional de Salud Mental. 2013. [Epidemiological Study of Mental Health in Metropolitan Lima and Callao - 2012 replication] URL: http://www.insm.gob.pe/ojsinsm/index.php/Revista1/article/view/40 [accessed 2019-05-25] [WebCite Cache]
- Instituto Nacional de Salud Mental. 2009. [Epidemiological Study of Mental Health in the Rural Sierra 2008. General Report] URL: http://www.insm.gob.pe/ojsinsm/index.php/Revista1/article/view/33 [accessed 2019-05-25] [WebCite Cache]
- Laurant M, Reeves D, Hermens R, Braspenning J, Grol R, Sibbald B. Substitution of doctors by nurses in primary care. Cochrane Database Syst Rev 2005 Apr 18(2):CD001271. [CrossRef] [Medline]
- World Health Organization. 2008. Treat train retain. Task shifting: Global recommendations and guidelines URL: https://www.who.int/healthsystems/TTR-TaskShifting.pdf [accessed 2019-05-22] [WebCite Cache]
- World Health Organization. Geneva; 2006. Taking stock: Health worker shortages and the response to AIDS Internet URL: https://apps.who.int/iris/handle/10665/69761 [accessed 2019-05-22] [WebCite Cache]
- Callaghan M, Ford N, Schneider H. A systematic review of task- shifting for HIV treatment and care in Africa. Hum Resour Health 2010 Mar 31;8:8. [CrossRef] [Medline]
- Ogedegbe G, Gyamfi J, Plange-Rhule J, Surkis A, Rosenthal DM, Airhihenbuwa C, et al. Task shifting interventions for cardiovascular risk reduction in low-income and middle-income countries: a systematic review of randomised controlled trials. BMJ Open 2014 Oct 16;4(10):e005983 [FREE Full text] [CrossRef] [Medline]
- Kredo T, Adeniyi F, Bateganya M, Pienaar E. Task shifting from doctors to non-doctors for initiation and maintenance of antiretroviral therapy. Cochrane Database Syst Rev 2014 Jul 1(7):CD007331. [CrossRef] [Medline]
- Fulton BD, Scheffler RM, Sparkes SP, Auh EY, Vujicic M, Soucat A. Health workforce skill mix and task shifting in low income countries: a review of recent evidence. Hum Resour Health 2011 Jan 11;9:1 [FREE Full text] [CrossRef] [Medline]
- Joshi R, Alim M, Kengne AP, Jan S, Maulik PK, Peiris D, et al. Task shifting for non-communicable disease management in low and middle income countries--a systematic review. PLoS One 2014;9(8):e103754 [FREE Full text] [CrossRef] [Medline]
- Buttorff C, Hock RS, Weiss HA, Naik S, Araya R, Kirkwood BR, et al. Economic evaluation of a task-shifting intervention for common mental disorders in India. Bull World Health Organ 2012 Nov 1;90(11):813-821 [FREE Full text] [CrossRef] [Medline]
- Petersen I, Lund C, Bhana A, Flisher AJ, Mental Health and Poverty Research Programme Consortium. A task shifting approach to primary mental health care for adults in South Africa: human resource requirements and costs for rural settings. Health Policy Plan 2012 Jan;27(1):42-51. [CrossRef] [Medline]
- Chatterjee S, Naik S, John S, Dabholkar H, Balaji M, Koschorke M, et al. Effectiveness of a community-based intervention for people with schizophrenia and their caregivers in India (COPSI): a randomised controlled trial. Lancet 2014 Apr 19;383(9926):1385-1394 [FREE Full text] [CrossRef] [Medline]
- van Ginneken N, Tharyan P, Lewin S, Rao GN, Meera SM, Pian J, et al. Non-specialist health worker interventions for the care of mental, neurological and substance-abuse disorders in low- and middle-income countries. Cochrane Database Syst Rev 2013 Nov 19(11). [CrossRef] [Medline]
- Emdin CA, Millson P. A systematic review evaluating the impact of task shifting on access to antiretroviral therapy in sub-Saharan Africa. Afr Health Sci 2012 Sep;12(3):318-324 [FREE Full text] [Medline]
- Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood) 2010 Feb;29(2):244-251 [FREE Full text] [CrossRef] [Medline]
- Krishna S, Boren SA, Balas EA. Healthcare via cell phones: a systematic review. Telemed J E Health 2009 Apr;15(3):231-240. [CrossRef] [Medline]
- Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009;38(4):196-205. [CrossRef] [Medline]
- Van Voorhees BW, Mahoney N, Mazo R, Barrera AZ, Siemer CP, Gladstone TR, et al. Internet-based depression prevention over the life course: a call for behavioral vaccines. Psychiatr Clin North Am 2011 Mar;34(1):167-183 [FREE Full text] [CrossRef] [Medline]
- Zhao J, Freeman B, Li M. Can mobile phone apps influence people's health behavior change? An evidence review. J Med Internet Res 2016 Oct 31;18(11):e287 [FREE Full text] [CrossRef] [Medline]
- Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [CrossRef] [Medline]
- Proudfoot J, Ryden C, Everitt B, Shapiro DA, Goldberg D, Mann A, et al. Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry 2004 Jul;185:46-54 [FREE Full text] [Medline]
- van't Hof E, Cuijpers P, Stein DJ. Self-help and Internet-guided interventions in depression and anxiety disorders: a systematic review of meta-analyses. CNS Spectr 2009 Feb;14(2 Suppl 3):34-40. [Medline]
- Riper H, Kramer J, Smit F, Conijn B, Schippers G, Cuijpers P. Web-based self-help for problem drinkers: a pragmatic randomized trial. Addiction 2008 Feb;103(2):218-227. [CrossRef] [Medline]
- Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res 2009;11(2):e15 [FREE Full text] [CrossRef] [Medline]
- Learmonth D, Trosh J, Rai S, Sewell J, Cavanagh K. The role of computer-aided psychotherapy within an NHS CBT specialist service. Counsel Psychother Res 2008 Jun;8(2):117-123. [CrossRef]
- Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health 2016;3(1):e7 [FREE Full text] [CrossRef] [Medline]
- Shen N, Levitan MJ, Johnson A, Bender JL, Hamilton-Page M, Jadad AA, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth 2015;3(1):e16 [FREE Full text] [CrossRef] [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- Proudfoot J, Parker G, Hadzi PD, Manicavasagar V, Adler E, Whitton A. Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress. J Med Internet Res 2010;12(5):e64 [FREE Full text] [CrossRef] [Medline]
- Schueller SM, Mohr DC. Initial Field Trial of a Coach-Supported Web-Based Depression Treatment. : Institute of Electrical and Electronics Engineers Inc; 2015 Aug Presented at: 9th International Conference on Pervasive Computing Technologies for Healthcare. ICST; April 25-27, 2015; Istanbul, Turkey p. 25-28. [CrossRef]
- Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011;13(1):e30 [FREE Full text] [CrossRef] [Medline]
- Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, et al. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One 2013 Aug;8(8):e70086 [FREE Full text] [CrossRef] [Medline]
- Mechael PN. The case for mHealth in developing countries. Innov Technol Gov Global 2009 Jan;4(1):103-118. [CrossRef]
- Rice RE, Katz JE. Comparing internet and mobile phone usage: digital divides of usage, adoption, and dropouts. Telecomm Policy 2003 Sep;27(8-9):597-623. [CrossRef]
- OSIPTEL. 2016 Jun. [Access to mobile telephony is close to 100% in Peru] URL: http://www.osiptel.gob.pe/Archivos/Publicaciones/reporte_estadistico_junio2016/files/assets/common/downloads/Reporte%20Estad.pdf [accessed 2019-05-23] [WebCite Cache]
- Menezess PR, Araya R, Miranda J, Mohr DC, Price LS. The Latin American treatment and innovation network in mental health h (LATINMH): rationale and scope. Rev Fac Cien Med Univ Nac Cordoba 2015;72(4):321-330. [Medline]
- Menezes P, Quayle J, Claro HG, da Silva S, Brandt LR, Diez-Canseco F, et al. Use of smartphone application to treat comorbidity depression with hypertension or diabetes I: a pilot study in Brazil and Peru. JMIR Mhealth Uhealth 2019 Apr 26;6(4). [CrossRef] [Medline]
- Bengtsson M. How to plan and perform a qualitative study using content analysis. NursingPlus Open 2016;2:8-14. [CrossRef]
- Downe-Wamboldt B. Content analysis: method, applications, and issues. Health Care Women Int 1992;13(3):313-321. [CrossRef] [Medline]
- Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [Medline]
- Prince M, Acosta D, Ferri CP, Guerra M, Huang Y, Jacob KS, 10/66 Dementia Group. A brief dementia screener suitable for use by non-specialists in resource poor settings--the cross-cultural derivation and validation of the brief Community Screening Instrument for Dementia. Int J Geriatr Psychiatry 2011 Sep;26(9):899-907 [FREE Full text] [CrossRef] [Medline]
- Bebbington P, Nayani T. The psychosis screening questionnaire. Int J Methods Psychiatr Res 1995;5(1):11-19 [FREE Full text]
- Ekers D, Webster L, van Straten A, Cuijpers P, Richards D, Gilbody S. Behavioural activation for depression; an update of meta-analysis of effectiveness and sub group analysis. PLoS One 2014;9(6):e100100 [FREE Full text] [CrossRef] [Medline]
- Richards DA, Ekers D, McMillan D, Taylor RS, Byford S, Warren FC, et al. Cost and Outcome of Behavioural Activation versus Cognitive Behavioural Therapy for Depression (COBRA): a randomised, controlled, non-inferiority trial. Lancet 2016 Aug 27;388(10047):871-880 [FREE Full text] [CrossRef] [Medline]
- World Health Organization. Geneva; 2016. mhGAP intervention guide for mental, neurological and substance use disorders in non-specialized health settings: mental health Gap Action Programme (mhGAP) – version 2.0 URL: https://www.who.int/mental_health/mhgap/mhGAP_intervention_guide_02/en/ [accessed 2019-05-23] [WebCite Cache]
- Lejuez CW, Hopko DR, Hopko SD. A brief behavioral activation treatment for depression. Treatment manual. Behav Modif 2001 Apr;25(2):255-286. [CrossRef] [Medline]
- Dahne J, Kustanowitz J, Lejuez CW. Development and preliminary feasibility study of a brief behavioral activation mobile application (Behavioral Apptivation) to be used in conjunction with ongoing therapy. Cogn Behav Pract 2018 Feb;25(1):44-56 [FREE Full text] [CrossRef] [Medline]
- Collado A, Calderón M, MacPherson L, Lejuez C. The efficacy of behavioral activation treatment among depressed Spanish-speaking Latinos. J Consult Clin Psychol 2016 Jul;84(7):651-657 [FREE Full text] [CrossRef] [Medline]
- Abas M, Bowers T, Manda E, Cooper S, Machando D, Verhey R, et al. 'Opening up the mind': problem-solving therapy delivered by female lay health workers to improve access to evidence-based care for depression and other common mental disorders through the Friendship Bench Project in Zimbabwe. Int J Ment Health Syst 2016 May;10:39 [FREE Full text] [CrossRef] [Medline]
- Lejuez CW, Hopko DR, Acierno R, Daughters SB, Pagoto SL. Ten year revision of the brief behavioral activation treatment for depression: revised treatment manual. Behav Modif 2011 Mar;35(2):111-161. [CrossRef] [Medline]
- Ben-Zeev D, Schueller SM, Begale M, Duffecy J, Kane JM, Mohr DC. Strategies for mHealth research: lessons from 3 mobile intervention studies. Adm Policy Ment Health 2015 Mar;42(2):157-167. [CrossRef] [Medline]
- Mantani A, Kato T, Furukawa TA, Horikoshi M, Imai H, Hiroe T, et al. Smartphone cognitive behavioral therapy as an adjunct to pharmacotherapy for refractory depression: randomized controlled trial. J Med Internet Res 2017 Nov 3;19(11):e373 [FREE Full text] [CrossRef] [Medline]
- Burnard P. A method of analysing interview transcripts in qualitative research. Nurse Educ Today 1991 Dec;11(6):461-466. [Medline]
- Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004 Feb;24(2):105-112. [CrossRef] [Medline]
- Berg BL. Qualitative Research Methods For The Social Sciences (Fourth Edition). Needham Heights, MA: Allyn & Bacon; 2019.
- Morgan DL. Qualitative content analysis: a guide to paths not taken. Qual Health Res 1993 Feb;3(1):112-121. [Medline]
- Naslund JA, Aschbrenner KA, Araya R, Marsch LA, Unützer J, Patel V, et al. Digital technology for treating and preventing mental disorders in low-income and middle-income countries: a narrative review of the literature. Lancet Psychiatry 2017 Jun;4(6):486-500 [FREE Full text] [CrossRef] [Medline]
- Abaza H, Marschollek M. mHealth application areas and technology combinations*. A comparison of literature from high and low/middle income countries. Methods Inf Med 2017 Aug 8;56(7):e105-e122. [CrossRef] [Medline]
- The Lancet. Making more of multimorbidity: an emerging priority. The Lancet 2018 Apr;391(10131):1637. [CrossRef] [Medline]
- The Academy of Medical Sciences. 2018 Apr. Multimorbidity: a priority for global health research URL: https://acmedsci.ac.uk/file-download/82222577 [accessed 2019-05-23] [WebCite Cache]
- Rivas-Nieto AC, Málaga G, Ruiz-Grosso P, Huayanay-Espinoza CA, Curioso WH. [Use and perceptions of information and communication technologies in patients with hypertension, dyslipidemia or diabetes in a national hospital in Lima, Peru]. Rev Peru Med Exp Salud Publica 2015;32(2). [CrossRef]
- Sarkar U, Gourley GI, Lyles CR, Tieu L, Clarity C, Newmark L, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 2016 Dec;31(12):1417-1426. [CrossRef] [Medline]
- Ye J, Zuo Y, Xie T, Wu M, Ni P, Kang Y, et al. A telemedicine wound care model using 4G with smart phones or smart glasses: a pilot study. Medicine (Baltimore) 2016 Aug;95(31):e4198 [FREE Full text] [CrossRef] [Medline]
- Martínez-Pérez B, de la Torre-Diez I, López-Coronado M. Experiences and results of applying tools for assessing the quality of a mHealth app named Heartkeeper. J Med Syst 2015 Nov;39(11):142. [CrossRef] [Medline]
- East ML, Havard BC. Mental health mobile apps: from infusion to diffusion in the mental health social system. JMIR Ment Health 2015;2(1):e10 [FREE Full text] [CrossRef] [Medline]
- Zhu Q, Liu C, Holroyd K. From a traditional behavioral management program to an m-health app: Lessons learned in developing m-health apps for existing health care programs. : IEEE; 2012 Jun 28 Presented at: 4th International Workshop on Software Engineering in Health Care (SEHC); June 4-5, 2012; Zurich, Switzerland. [CrossRef]
- Lie SS, Karlsen B, Oord ER, Graue M, Oftedal B. Dropout from an eHealth intervention for adults with type 2 diabetes: a qualitative study. J Med Internet Res 2017 May 30;19(5):e187 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
- Diez-Canseco F, Toyama M, Ipince A, Perez-Leon S, Cavero V, Araya R, et al. Integration of a technology-based mental health screening program into routine practices of primary health care services in Peru (The Allillanchu Project): development and implementation. J Med Internet Res 2018 Mar 15;20(3):e100 [FREE Full text] [CrossRef] [Medline]
- Singla D, Lazarus A, Atif N, Sikander S, Bhatia U, Ahmad I, et al. "Someone like us": delivering maternal mental health through peers in two South Asian contexts. J Affect Disord 2014 Oct;168:452-458 [FREE Full text] [CrossRef] [Medline]
- Miranda JJ, Diez-Canseco F, Araya R, Cutipe Y, Castillo H, Herrera V, et al. Transitioning mental health into primary care. Lancet Psychiatry 2017 Feb;4(2):90-92. [CrossRef] [Medline]
|DALY: disability-adjusted life year|
|LATIN-MH: Latin America Treatment and Innovation Network in Mental Health|
|LMIC: low- and middle-income country|
|mHealth: mobile health|
|MINSA: Ministry of Health|
|NCD: noncommunicable disease|
|SMS: short message service|
Edited by G Eysenbach; submitted 26.07.18; peer-reviewed by P Soderlund, M Lucassen, R Cooper, S Comulada, M Leung; comments to author 23.12.18; revised version received 15.02.19; accepted 30.03.19; published 18.06.19Copyright
©Lena R Brandt, Liliana Hidalgo, Francisco Diez-Canseco, Ricardo Araya, David C Mohr, Paulo R Menezes, J Jaime Miranda. Originally published in JMIR Mental Health (http://mental.jmir.org), 18.06.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.