This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
The development of Web-based interventions for substance abuse in Latin America is a new field of interest with great potential for expansion to other Spanish-speaking countries.
This paper describes a project aimed to develop and evaluate the usability of the Web-based Help Program for Drug Abuse and Depression (Programa de Ayuda para Abuso de Drogas y Depresión, PAADD, in Spanish) and also to construct a systematic frame of reference for the development of future Web-based programs.
The PAADD aims to reduce substance use and depressive symptoms with cognitive behavioral techniques translated into Web applications, aided by the participation of a counselor to provide support and guidance. This Web-based intervention includes 4 steps: (1) My Starting Point, (2) Where Do I Want to Be? (3) Strategies for Change, and (4) Maintaining Change. The development of the program was an interactive multistage process. The first stage defined the core structure and contents, which were validated in stage 2 by a group of 8 experts in addiction treatment. Programming of the applications took place in stage 3, taking into account 3 types of end users: administrators, counselors, and substance users. Stage 4 consisted of functionality testing. In stage 5, a total of 9 health professionals and 20 drug users currently in treatment voluntarily interacted with the program in a usability test, providing feedback about adjustments needed to improve users’ experience.
The main finding of stage 2 was the consensus of the health professionals about the cognitive behavioral strategies and techniques included in PAADD being appropriate for changing substance use behaviors. In stage 5, the health professionals found the functionalities easy to learn; their suggestions were related to the page layout, inclusion of confirmation messages at the end of activities, avoiding “read more” links, and providing feedback about every activity. On the other hand, the users said the information presented within the modules was easy to follow and suggested more dynamic features with concrete instructions and feedback.
The resulting Web-based program may have advantages over traditional face-to-face therapies owing to its low cost, wide accessibility, anonymity, and independence of time and distance factors. The detailed description of the process of designing a Web-based program is an important contribution to others interested in this field. The potential benefits must be verified in specific studies.
International Standard Randomized Controlled Trial Number (ISRCTN): 25429892; http://www.controlled-trials.com/ISRCTN25429892 (Archived by WebCite at http://www.webcitation.org/6ko1Fsvym)
The drug treatments with the greatest empirical validity are those involving cognitive behavioral interventions (CBIs) [
A recent meta-analysis of the effectiveness of Internet-based interventions based on cognitive behavioral therapy demonstrated a large effect size (0.83, n=3960) as compared with other modalities, including psychoeducation (0.46, n=6796) [
Web-based interventions allow complex treatments to be delivered with consistency and minimal demands on staff time and training resources. Moreover, computerized programs may be less threatening, provide greater anonymity [
During the last decade, several Web-based interventions have been developed and validated in the mental health field [
Web-based interventions developed in Latin America are very limited and focus on smoking [
Approximately 3.3 billion people around the world use the Internet, among them 345 million in Latin America, with a growth of 1808.4% in the past 15 years [
There are few publications that describe the process of developing a Web-based intervention, the selection of strategies required, or the limitations involved in starting it up [
The purposes of this paper are therefore to describe the development process of the Web-based Help Program for Drug Abuse and Depression (Programa de Ayuda para Abuso de Drogas y Depresión, PAADD, in Spanish) [
The PAADD was developed in an interactive multistage process that involved design, testing, and redesign tasks, following international recommendations for the development of eHealth strategies and ethical standards for Web-based interventions [
The aims of this phase were to define the structure and contents of the program. After a search of the literature on treatment in Mexico for problems of substance use and depression, 3 sources were chosen: (1) self-help manual ¿Cómo dejar de consumir drogas? (How to Stop Using Drugs) [
The basic structure of the intervention and the specific techniques of behavior modification were taken from the self-help manual How to Stop Using Drugs [
The Web-based program ADep [
The Web-based program Beber Menos [
On the basis of the literature review of Web-based interventions, the specific characteristics of the PAADD were defined as follows:
1. The PAADD is a stand-alone intervention directed at persons with risky levels of drug use and not at those with drug dependence.
2. It offers strategies to address substance abuse and depressive symptoms together.
3. It includes 4 steps to establish a baseline and a treatment goal, in addition to strategies to change the pattern of use, stay focused on the goal, and prevent relapses.
4. It offers contact from a counselor, which previous studies have shown produces favorable results [
Once the outline and the core contents for each step were planned, these were shared with a focus group of 8 experts in addiction treatment from the public mental health care system in the Mexico City metropolitan area. Participants were asked to comment on the proposed techniques and the possible impact of the program and to provide suggestions for improvement and implementation.
These experts expressed the opinion that the CBI strategies and techniques for changing substance use behaviors were appropriate. Their comments were useful for fine-tuning the inclusion criteria: (1) medium risk level of drug use according to the Alcohol, Smoking and Substance Involvement Screening Test [
The PAADD was structured into 4 successive steps. The home page is shown in
Overall structure of the Web-Based Help Program for Drug Abuse and Depression (Programa de Ayuda para Abuso de Drogas y Depresión, PAADD, in Spanish).
Step | Cognitive behavioral strategies |
Step 1: My Starting Point | Establishment of baseline |
Step 2: Where Do I Want to Be? | Goal setting |
Step 3: Strategies for Change | Self-monitoring |
Step 4: Maintaining Change | Social skills for resisting pressure |
aADep: Ayuda para Depresión (Help for Depression).
Help Program for Drug Abuse and Depression (Programa de Ayuda para Abuso de Drogas y Depresión, PAADD)'’s welcome page. Illustration of the stages and tools comprised in the program.
Timeline followback in step 1. Tool used to record drug use during the previous week and the amount of money expended.
Decisional balance in step 1. Tool to identify advantages and disadvantages of drug use.
Goal setting in step 2. Tool to set a consumption goal for the following week.
Cognitive restructuring in step 3. Tool to analyze negative thoughts associated with drug use.
The aim of this module is to promote the awareness of personal drug use and its risks in order to increase the motivation to change. This module includes assessment instruments and behavioral strategies to establish a baseline. At the end, users receive a printable report and are asked if they are committed to participating in a process of change. A positive answer directs them to step 2; a negative answer redirects them to a feedback page to prompt reflection about the consequences of drug use.
The purpose of this step is to set treatment goals. Users are provided with various tools, such as the recommendation to define a goal (simple, specific, realistic), to establish a reduction plan (to focus on 1 drug at a time, to plan ahead, to remember that any reduction in use is a step forward), and to set a time limit to achieve the goal. At the end of this step the user receives a printable feedback page.
Step 3 is aimed at achieving behavioral change in accordance with the goals established in step 2. Users are advised to log in every day, record daily drug use, review the progress graph, and do an activity to achieve the change. They are offered tools to perform a functional analysis of substance use behavior, to develop action plans to face risk situations, to apply emotional control techniques to cope with anxiety, to generate positive self-reinforcement, and for cognitive restructuring of negative thoughts.
The objective of step 4 is to reinforce changes in drug use behavior and avoid relapses. It offers strategies for recognition of success (self-reinforcement), for facing pressure to use drugs (assertiveness), and for seeking social support, among others. The counselor makes the decision to enable this step in accordance with the results from step 3.
The program is designed to be completed in 8 weeks, but this period can be extended if necessary depending on the evaluation of the counselor, who might suggest specific activities that the participant may have skipped, according to the progress and needs of each user. The role of the counselor also includes monitoring the completion of the activities as well as providing written feedback and encouragement within; all these functions are performed through a messaging system within the program within 24 hours after a user logs in. This is a minimal contact scheme that differs from conducting therapy sessions via the Internet and represents an opportunity to optimize time because one counselor can monitor 2 or more patients simultaneously.
The structure of the Web-based intervention was outlined as a diagram in order to establish the sequence of the steps involved. The PAADD was constructed by a commercial programming company as a Web application accessible through computers, laptops, or tablets with any browser; although it is not a mobile application, it can also be accessed with a mobile phone. It took into account 3 types of end users: administrators, counselors, and substance users. Functionality and usability tests were performed on the completed application.
The purpose of this process was to ensure that the PAADD met the technical requirements set out in the functional design and to identify programming glitches. The test specifically verified the following:
The program worked according to specifications.
The database responded to the input and output specifications of expected information.
The system followed the sequence identified in the functional design.
These functionalities were tested by members of the research team (n=7), who interacted with the program and verified each one, following a checklist. Errors found were corrected and a second test was performed. This functionality testing was then followed by a usability test.
Usability is defined as the ability of a software or program to be understood, learned, used, and be attractive to the end user under specific conditions of use [
A total of 9 health professionals from Mexico City, State of Mexico, and the states Morelos, and Baja California agreed to participate. Inclusion criteria were that (1) they worked in an addiction treatment center; (2) they had at least a year of experience working in substance abuse treatment using cognitive behavioral strategies; (3) they had experience using the Internet; and (4) they were regular email users. Each was asked to invite patients under treatment (n=20) to give their opinion of the PAADD. The inclusion criteria for the patient group were that (1) they were adults; (2) they had received a minimum of 4 treatment sessions for abuse of psychoactive substances during the previous 6 months; (3) they could read and write; (4) they had experience using the Internet; and (5) they were regular email users.
All participants (both health professionals and patients) responded to a questionnaire about their Internet usage and received an instruction guide for usability test. The checklist for the health professionals focused on evaluating the role of the counselor (usefulness of strategies, contents, and message system), whereas that for patients focused on their perspective about language and ease of use.
This procedure was approved by the Research Ethics Committee of the Ramón de la Fuente Muñiz National Institute of Psychiatry. All participants were volunteers and signed an informed consent form with the understanding that the data might be used in research. All information obtained in this program is confidential and available only to members of the research team. The questionnaires, forms, and other documents were identified by a numeric code that avoids the identification of participants.
The usability test was based on a selection of tasks and pages according to the guidelines proposed by Dumas and Loring [
The tests carried out by the health professionals consisted of navigating the PAADD, following a list of specific actions, and reporting on their experience. The tests by patients were carried out at the centers where they were receiving treatment. A member of the research team accompanied them and recorded their commentaries and suggestions.
Results of the usability test are presented here separately by type of participant.
The group included 5 women and 3 men with an average age of 30.25 years (SD 5.33). All participants had a master's degree in psychology of addiction and had an average of 5 years of experience using the CBI approach to treat persons with problems of substance abuse. All participants were regular users of the Internet, with an average daily use of 6.1 hours, the majority from their home and workplace.
The main suggestions from the health professionals focused on the content and format but not on the structure of the program, for instance, recommendations to change the color scheme, page layout, font size, arrangement of functions, inclusion of confirmation messages at the end of activities, avoiding the use of technical language and “read more” links, providing short introductory texts for every task, and creating more visible help buttons with specific explanations and tips. They found the functionalities easy to learn, although they indicated that some were not intuitive at first glance. They suggested providing feedback about every activity and including more activities and exercises within the program, as well as more tips on how to end substance abuse. They also suggested the possibility of including testimonials from persons who had benefited from the program (see
The user group included 16 men and 4 women, 60% of them single, with an average age of 29.6 (SD 9.1) years. In this group 70% (14/20) had completed high school, 15% (3/20) an undergraduate degree, and 15% (3/20) elementary school; 65% (13/20) were employed, 20% (4/20) did not work, and 15% (3/20) were students. All participants were regular users of the Internet, with an average daily use of 3 hours, the majority from home (85%) and the rest using their cell phone and/or at Internet cafes.
Users suggested a more dynamic design with images and colors, clearer, more concrete instructions, inclusion of empathetic, reflective feedback, and also a chart showing the names and appearance of different drugs. The majority said it was important to be able to modify the weekly goal and clarify the sequence of activities in the program (see
The observations and suggestions of users and health professionals were compiled into a document that served as a guide for development of a new version of the PAADD, which was in turn put through an evaluation process.
Observations and suggestions from health professionals.
Step | Positive aspects | Negative aspects | Suggestions |
Step 1 | Includes counselor from first contact. |
It is necessary to clarify some instructions and the sequence of the instruments. |
Indicate what each instrument evaluates. |
Step 2 | Gives users the responsibility to set their own goals. |
The sequence of activities is not clear. |
Provide simpler instructions. |
Step 3 | The graph of use allows users to observe their changes. |
The sequence of activities is not clear. |
Make the sequence of activities explicit. |
Step 4 | The exercises included are strategically important to maintain the change in drug use. |
Needs automatic feedback on finishing exercises. |
Add a phrase to show when an activity is finished. |
Observations and suggestions from drug users in treatment.
Step | Positive aspects | Negative aspects | Suggestions |
Step 1 | The initial information and the informed consent help to understand what the program is about. |
The design of the program is very serious. |
Use a dynamic design that includes images, vignettes, and more color. |
Step 2 | Feedback on setting goals helps to remember them. | The instructions are complex. |
Include more than 1 drug in the log of goals. |
Step 3 | The initial screen shows a general overview of the activities of this step. |
The instructions are complex. |
Provide clear and concise instructions and interactive examples of the activity log. |
Step 4 | The activities are easy to do. | Activities do not provide feedback, and there is no indicator of when they are finished. | Give automatic reminders in case the weekly goal is not met. |
This paper describes the development and results of the usability test of PAADD, a Web-based intervention with the participation of a counselor, the first of its kind in Latin America that is designed to reduce substance use and depressive symptoms. The development of the program was systematic and rigorous. It began with a review of the literature on brief interventions and incorporated the opinions of experts in addiction treatment as well as 2 types of end users, following a procedure similar to that reported by Morrison et al [
The translation of cognitive behavioral strategies to the concrete activities of the PAADD was complex and time consuming. Van Voorhees et al [
An important characteristic of the PAADD is that it includes the participation of counselors who provide feedback according to the motivational interviewing techniques. This element is intended to strengthen adherence, as it has been shown that the inclusion of health professionals has a positive influence on retention and motivation of participants in such programs [
In the usability test, drug users commented that the PAADD was professional and clear. Many felt that the information presented within the modules was relatively easy to follow, comprehensive, and of good quality. A few participants reported at times feeling impatient to receive the next module. The majority of users commented positively on the content of the program and felt that they could benefit from it. In their view, it helped principally with improved self-awareness of their pattern of drug use. They reported that the PAADD encouraged them to think about self-management techniques, how to monitor their thoughts and feelings, and how to regulate their behavior and consumption patterns. These observations suggest that the design process was successful; it resulted in a program that is easily usable by persons who wish to end their substance use and who present depressive symptomatology.
The PAADD may have advantages over traditional face-to-face therapies because of its low cost, wide accessibility, anonymity, and independence of time and distance factors. It may be a good alternative for persons not receiving treatment owing to factors such as physical distance from available services, lack of qualified providers, socioeconomic condition, and stigma, although these potential benefits will have to be verified in specific studies.
In future efforts it will also be important to consider that the initial investment for the design of a Web-based intervention can be high and that it is necessary to perform a cost-benefit analysis to document the viability of the project. It should also be kept in mind that technologies are developing rapidly; in order to provide increased availability and accessibility, it is necessary to plan from the beginning the possibility of using the program on diverse types of devices, such as mobile phones and tablets, and not just on computers.
Because PAADD is not an open-access program, it will also be necessary to design a dissemination strategy that would include training of counselors and to promote the use of PAADD in specialized treatment centers. This will allow to document adoption and implementation in real-life scenarios.
The lack of forums or chat functionalities for users to interact with each other may be a limitation in this program because having contact with others in the same situation can be a source of support. It is also possible that the small number of users participating in the usability test limited the number of opinions regarding improvements to the program. Such limitations have been reported in similar studies [
Data from the usability test indicate that the PAADD has the necessary features to support users in their process of reducing drug use. If this is corroborated in effectiveness studies, the program could be used as an alternative to treatment as usual.
Ayuda para Depresión (Help for Depression)
cognitive behavioral intervention
Programa de Ayuda para Abuso de Drogas y Depresión (Web-based Help Program for Drug Abuse and Depression)
Patient Health Questionnaire
This study was supported by a grant from the US Department of State (SINLEC11GR0015/A001/A002).
None declared.