Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 11.01.17 in Vol 4, No 1 (2017): Jan-Mar

This paper is in the following e-collection/theme issue:

    Original Paper

    Tips and Traps: Lessons From Codesigning a Clinician E-Monitoring Tool for Computerized Cognitive Behavioral Therapy

    1Department of Psychological Medicine, Faculty of Medical and Health Sciences, The University of Auckland, Auckland, New Zealand

    2Department of General Practice and Primary Health Care, The University of Auckland, Auckland, New Zealand

    3School of Health, Wellbeing and Social Care, The Open University, Milton Keynes, United Kingdom

    4Department of Paediatrics, Child and Youth Health, The University of Auckland, Auckland, New Zealand

    5School of Counselling, Human Services and Social Work, The University of Auckland, Auckland, New Zealand

    6Department of Clinical Psychology, The University of Auckland, Auckland, New Zealand

    7Kapiti Youth Support, Paraparaumu, New Zealand

    Corresponding Author:

    Frederick Sundram, MB, BCh, BAO, FRCPsych, MA, MSc, PhD

    Department of Psychological Medicine

    Faculty of Medical and Health Sciences

    The University of Auckland

    Private Bag 92019

    Auckland Mail Centre

    Auckland, 1142

    New Zealand

    Phone: 64 9 923 7521

    Fax:64 9 373 7013



    Background: Computerized cognitive behavioral therapy (cCBT) is an acceptable and promising treatment modality for adolescents with mild-to-moderate depression. Many cCBT programs are standalone packages with no way for clinicians to monitor progress or outcomes. We sought to develop an electronic monitoring (e-monitoring) tool in consultation with clinicians and adolescents to allow clinicians to monitor mood, risk, and treatment adherence of adolescents completing a cCBT program called SPARX (Smart, Positive, Active, Realistic, X-factor thoughts).

    Objective: The objectives of our study were as follows: (1) assess clinicians’ and adolescents’ views on using an e-monitoring tool and to use this information to help shape the development of the tool and (2) assess clinician experiences with a fully developed version of the tool that was implemented in their clinical service.

    Methods: A descriptive qualitative study using semistructured focus groups was conducted in New Zealand. In total, 7 focus groups included clinicians (n=50) who worked in primary care, and 3 separate groups included adolescents (n=29). Clinicians were general practitioners (GPs), school guidance counselors, clinical psychologists, youth workers, and nurses. Adolescents were recruited from health services and a high school. Focus groups were run to enable feedback at 3 phases that corresponded to the consultation, development, and postimplementation stages. Thematic analysis was applied to transcribed responses.

    Results: Focus groups during the consultation and development phases revealed the need for a simple e-monitoring registration process with guides for end users. Common concerns were raised in relation to clinical burden, monitoring risk (and effects on the therapeutic relationship), alongside confidentiality or privacy and technical considerations. Adolescents did not want to use their social media login credentials for e-monitoring, as they valued their privacy. However, adolescents did want information on seeking help and personalized monitoring and communication arrangements. Postimplementation, clinicians who had used the tool in practice revealed no adverse impact on the therapeutic relationship, and adolescents were not concerned about being e-monitored. Clinicians did need additional time to monitor adolescents, and the e-monitoring tool was used in a different way than was originally anticipated. Also, it was suggested that the registration process could be further streamlined and integrated with existing clinical data management systems, and the use of clinician alerts could be expanded beyond the scope of simply flagging adolescents of concern.

    Conclusions: An e-monitoring tool was developed in consultation with clinicians and adolescents. However, the study revealed the complexity of implementing the tool in clinical practice. Of salience were privacy, parallel monitoring systems, integration with existing electronic medical record systems, customization of the e-monitor, and preagreed monitoring arrangements between clinicians and adolescents.

    JMIR Ment Health 2017;4(1):e3




    Depression in adolescence is a major cause of disability that is often underreported, unrecognized [1,2], and not addressed by timely and effective treatments [3-5]. Psychological therapies are recommended as a first-line treatment for mild-to-moderate forms of depression [6,7] but are difficult to access. Computer-delivered therapies, especially those based on cognitive behavioral therapy (CBT), have been developed to help address this [8]. Computerized cognitive behavioral therapy (cCBT) can be offered as pure self-help or guided by a clinician [9,10]. The advantages of pure self-help tools include improved cost-effectiveness and nonreliance on clinicians to guide or support adolescents, and ultimately it may be some adolescents’ preference to utilize programs independently. However, there is some evidence that clinician-guided interventions or blended therapies (where face-to-face therapy and computerized approaches are used side by side) have better completion rates and are more effective than unsupported self-help therapy [11-15]. An example of a supported program is the virtual MindSpot Clinic in Australia that allows remote screening assessments (phone- or Web-based) and clinician-guided treatment for anxiety and depression [16]. On the contrary, “Beating the Blues” is an unguided Web-based depression treatment program completed by a user at home that enables the prescribing general practitioner (GP) to receive risk alerts and progress updates [17,18].

    To date most cCBT programs have been designed for adults, although more recently a number of programs have been created for depressed or anxious children and adolescents. They too range from self-help tools to supported programs. For example, “Think, Feel, Do” [19] relies on a facilitator (who is not required to be a clinician) to be available during the entire delivery of the program to discuss the content covered. BRAVE-ONLINE is a clinician supported anxiety program in which young people receive weekly emails from a clinician designed to give them feedback and encourage their program progression [20]. However, BRAVE-ONLINE does not include the ability to monitor symptoms. SPARX (Smart, Positive, Active, Realistic, X-factor thoughts) is a gamified cCBT program for adolescents, which was shown to be effective in 12- to 19-year-olds presenting to primary care with mild-to-moderate symptoms of depression [21]. SPARX was designed as a pure self-help program in response to young people wanting to maintain privacy and access the program independently. A freely available self-help resource such as SPARX, which is accessible to all young people in New Zealand, can help bring treatment to the estimated three quarters of young people with depression who do not seek help [22]. In the original trial of SPARX, the referring clinicians provided a minimal degree of oversight with only brief contact after a month to ensure adequate progress. Our experience from the trial suggested that while most clinicians endorsed the use of a self-help program, some wanted to have a more formal means of monitoring adolescent users’ progress while using SPARX (Merry, personal communication).

    Therefore, our study aimed to develop and implement an electronic monitoring (e-monitoring) tool relevant to adolescents with depression who attend youth-oriented primary care settings. The monitoring tool was envisaged to work in settings such as school health and counseling services because this is where many adolescents are likely to seek help [23]. To achieve this, we (1) assessed clinicians’ and adolescents’ views on using an e-monitoring tool and used this information to help shape development of the tool and (2) assessed clinician experiences with a fully developed version of the tool that was implemented in their clinical service. Of note, the tool was not expected to replace face-to-face contact but rather to offer another means of clinician oversight.


    Study Design and Setting

    A qualitative study was undertaken in New Zealand using focus groups with adolescents (aged 12-19 years) who were in contact with mental health services and also clinicians in primary care settings from the services where they worked (Table 1). We approached local secondary schools, a youth health center, and a nongovernmental organization using a snowball approach to recruit English-speaking participants. The clinicians approached were known to the researchers, and had an interest in adolescent health and in using the tool. The clinicians were GPs and allied health staff (school guidance counselors, clinical psychologists, youth workers, and nurses).

    We set out to embed within SPARX an e-monitoring tool that would allow a clinician to “prescribe” SPARX and remotely oversee users’ progress and mood scores. As a minimum, the tool would have the following features:

    1. A registration system for clinicians so that they could use the SPARX e-monitoring tool

    2. Electronic linking of SPARX user data to the prescribing clinician

    3. A dashboard accessible on the Internet to allow clinicians to track all users they had referred to SPARX

    4. An algorithm to automatically generate alerts at specific and concerning levels of depression, or self-harm, and a system to encourage users to access more help

    5. A system to deliver alerts or “flags” via email to the clinician

    In order to track mood in SPARX, we aimed to use the Patient Health Questionnaire for Adolescents (PHQ-A) [24,25] at 3 time points: module 1 (baseline), module 4 (mid-point), and module 7 (end of intervention). We also consulted a software developer to investigate potential solutions, and schematic blueprints (wireframes) were then developed to visually guide participants through the flow of a given system and illustrate what an e-monitoring system or dashboard might look like. These were then presented at focus groups with the clinicians and adolescents, who provided feedback during the various phases of development of the tool.

    Ethics Approval and Consent

    Ethics approval was obtained from the Health and Disability Ethics Committee, New Zealand (Reference: 12/CEN/62), and written informed consent was gained from each clinician, adolescent, and parents of any adolescent aged less than 16 years.

    Table 1. Details of focus group participants during various phases of the study.
    View this table

    Focus Groups and Data Collection

    Focus groups (Table 1) were conducted during 3 phases. Phase 1 was the consultation stage and was carried out prior to the creation of the e-monitor to canvass ideas that would help the design of the e-monitoring system, based on early wireframe designs (December 2012 to March 2013). Phase 2 was the development stage and was conducted after preliminary wireframes had been revised, and it included a broader discussion of how e-monitoring could be used clinically (June to July 2013). Phase 3 was the postimplementation stage and was conducted after clinicians trialed the e-monitoring tool in day-to-day practice (August 2013 to February 2014). A semistructured schedule was used for all the focus groups (Multimedia Appendix 1), and these were digitally recorded and professionally transcribed. Each group was run by 2 of 6 experienced facilitators (FS, KS, ML, TF, MS, and SM) and the duration, while ultimately determined by the participants, was usually 60-90 minutes. FS checked fidelity of the Phases 1 and 2 clinician transcripts, and AG reviewed the transcripts of the adolescent groups and Phase 3 clinician groups.

    Data Analysis

    A theoretical thematic analysis framework was used for examining the data from focus groups [26]. Transcripts were imported into NVivo before thematic analysis [27], and more specifically, the general inductive approach [28] was used to organize the dataset into multiple coded blocks. A higher level of data interpretation was then performed, and responses were assessed for what might be implied or inferred. FS checked and coded the transcripts of the Phases 1 and 2 clinician focus groups, and AG similarly reviewed and coded the transcripts of the adolescent focus groups and Phase 3 clinician focus groups. Each dataset was then independently coded (SH, Phases 1 and 2 clinician groups; MS, all adolescent groups; KS, Phase 3 clinician groups). The independent coding by the 2 researchers was then discussed until consensus was reached to determine the final themes. Quotations from focus groups, where relevant, are provided verbatim.


    Common themes were identified from both the clinician and adolescent focus groups, as well as from issues reported upon as unique to either group. During Phases 1 and 2, themes were organized into 3 broad categories: clinical progress; confidentiality and privacy; and technical issues. Clinicians who were involved in clinical testing (Phase 3) reflected on some concepts (for instance, the initial themes) and drew on their “hands-on” experiences of using the actual tool postimplementation. Refinements to improve the tool were also suggested (Textboxes 1 and 2).

    Development of the E-Monitor (Phases 1 and 2)

    Clinical Progress
    Engagement, Adherence, and Offering Help

    Most participants were positive and supportive of the idea of adding e-monitoring to enhance the effectiveness of cCBT. Some adolescents noted that it might be hard to initiate contact with a clinician if they received a message from SPARX saying “you should seek help,” and that the e-monitoring system would help clinicians to make contact with adolescents.

    I think the monitoring is definitely quite key for like if you’re you know making sure you get like consistently doing it and stuff. It could be quite easier if no one was checking up on you and stuff to be like just oh that’s enough that kind of thing.
    Clinical Burden and Effects on Therapeutic Relationship

    E-monitoring was seen as a positive step by clinicians and adolescents, but there was also concern that it could potentially contribute to increased clinician burden. Some clinicians felt that the clinician-user therapeutic relationship may be impacted upon by a tool used in place of an ongoing face-to-face relationship, while others were concerned at the possible risk of rapid change in depression severity in adolescents. Similarly, some adolescents were concerned that clinicians might become over-burdened with the monitoring, with potential negative impacts on the clinical relationship and rapport. Clinicians and adolescents suggested the need for a discussion around e-monitoring, so that the adolescent was aware that a clinician had oversight of their progress, and could maintain the therapeutic relationship while simultaneously enhancing the adolescent’s autonomy and sense of control.

    I personally think this is great, so you can give the child a bit more power if you like, to take control of how they think and what they’re going to do, because it seems often that it’s one of the parents driving what’s going to happen next and how quickly they are able to get better.
    Responsibility of Monitoring Risk, Parallel Monitoring, and Backup Systems

    Clinicians were concerned about how they could discharge the clinical responsibility of monitoring an adolescent who was using a self-help therapy and might not be seeing the clinician regularly. In particular, there was concern about managing self-harm or suicide if the clinician was unable to monitor progress regularly due to after-hours cover, being part-time, or on annual leave. A potential solution proposed was for other people in the service to check alerts. Having a robust system in place with cross-cover arrangements would allow monitoring to continue in their absence. Similarly, having e-monitoring alerts sent to more than one clinician would allow more staff to better assess and manage risk.

    I mean if I was working here and I saw it come through and I glance at it out of interest because I’m seeing the patient, I would be very happy and comfortable that the nurses had full ability to follow-up on that and someone else was doing it. It would be great. It would take too much time so that would be good.

    Adolescents realized that mood monitoring was based on honest responses by the user of the program, which may not always happen. They suggested that there should be “check-ins” alongside SPARX to ensure that the adolescent user was truly getting benefits from the treatment.

    Confidentiality and Privacy

    In keeping with the literature [29], confidentiality was an important consideration for both adolescents and clinicians. There were concerns about privacy, information use, and who had access to the information collected during the process of e-monitoring. Clinicians also had the impression that adolescents would prefer if their parents were not involved and that adolescents preferred to retain anonymity.

    There was some discussion about social media, and this was in the context of adolescents possibly using their Facebook login credentials to access the cCBT program, an idea posed as a possible means of reducing registration overheads. Another idea raised was whether mental health programs should have a social media presence, given the potential stigma associated with mental illness. Adolescents were worried that others could see that they had signed up for SPARX through a Facebook announcement, and consequently asked that the option to register using a social media account be removed.

    It’s like when you are depressed or something, you don’t like want to tell the whole world about it really.

    Textbox 1. Key points arising during each phase.
    View this box

    Textbox 2. Proposed solutions to issues identified.
    View this box

    Some clinicians believed there would be reduced adolescent uptake should their details be identifiable via social media. However, many clinicians did not have privacy concerns especially if details were provided on the e-monitoring arrangements to the adolescent beforehand. Both adolescents and clinicians generally agreed that SPARX users should be given the option of selecting what information they wanted to share and how to be contacted.

    Technical Considerations

    Both adolescents and clinicians agreed that the registration process for e-monitoring should not be time-consuming. Ideally, clinicians would like to do it with the individual still present in their office, time permitting. Adolescents may change their contact details frequently, and therefore discussing the right communication medium is necessary. Technical aids such as information booklets with pictorial guides could be provided. Some adolescents suggested that the registration process should be “short and sweet” and there should be shortcuts, for example, a “Play Now” button that takes you straight to the program. Others said it would be easier for their clinician to complete the registration on their behalf and then send them the details.

    Some clinicians wondered who should have access to the tool and whether it should be restricted to those with appropriate clinical skills and experience. They suggested that before a clinician was given access to e-monitoring (or the ability to “prescribe” SPARX), their credentials should be verified with the relevant regulatory agencies that maintain the professional registration of clinicians. A minority of clinicians thought that the positive and supportive material within the cCBT program may cause some adolescents to avoid being e-monitored. This is because mood may improve while undergoing cCBT, and consequently the adolescent may view engagement with the clinician as unnecessary. Despite concerns about the security of emails and the way alerts would be sent, overall clinicians were supportive of e-monitoring.

    Many clinicians also suggested integrating the e-monitor and alerts with existing patient management systems (including electronic mailboxes that receive laboratory results). Clinicians supported the use of the Web-based tools built into the e-monitoring system for assessing mood, for example, PHQ-A, as these would potentially allow the “teasing out” of mood symptoms in greater detail so that energy, sleep, concentration, and other dimensions could be clarified. When adolescents describe low-mood symptoms, e-monitoring would potentially allow the assessment of the pattern of change over time. Clinicians felt that the acquisition of such finer detail is currently not possible with many face-to-face meetings. However, internet connectivity is key to e-monitoring, and adolescents experiencing sociodemographic deprivation may be particularly disadvantaged in this regard.

    Adolescents had a number of suggestions concerning the technical aspects of e-monitoring, but perhaps unsurprisingly, they were different from those expressed by the clinicians. Adolescents wanted to select a communication method (eg, email, texting) and agree this with the clinician. Adolescents were cognizant that the severity of symptoms varies between individuals and wanted alerts to the clinician set at a threshold high enough to warrant attention. Some also wanted to include a function that would allow the users to determine the urgency of the situation. Adolescents suggested that the following information be included when informing potential SPARX users about the ways to get help: “A recommended doctor,” “What to do if you’re depressed and you got no one,” “And if you don’t trust anyone to talk to.” Information where and how to access help (including after-hours help) should be prominent, clear, and available to all users (including those who may not need it immediately). Adolescents did not wish to receive many “unnecessary” alerts (via email or text). Therefore, the alerts going back to users should be sent at milestones (eg, completion of levels) and should not be sent too frequently. Personalized messages were favored over automated ones as they would “make you want to read it more.”

    Clinician Feedback on the E-Monitoring Tool Postimplementation (Phase 3)

    Following the use of the e-monitor in clinical practice, clinicians provided feedback that was often consistent with their prior responses during the consultation and development phases of the e-monitoring tool.

    Clinical Progress, Confidentiality, and Privacy

    Clinicians noted that e-monitoring was “another thing to remember,” requiring additional work, and following up with adolescents took more time than they allocated. For many, although the e-monitor was available, it was not used. During use of the tool, one primary contact was allocated to receive all the alerts in a clinic and then notify clinicians as appropriate. This process worked well but required dedicated time. The use of e-monitoring in this way was different from how it was originally conceptualized by the clinicians.

    Overall, clinicians explained that e-monitoring was helpful in checking whether adolescents had completed SPARX. They also described that e-monitoring provided another form of communication between them and the adolescent. Clinicians who used the tool believed that the quality of their relationship with adolescents was unaffected by the use of e-monitoring. Some clinicians, contrary to their earlier expectations, reported that it strengthened the therapeutic relationship as the adolescent felt more supported. Clinicians reported that e-monitoring was not on adolescents’ “radar” when they were doing it. So, it did not impact the therapeutic relationship, and clinicians continued to engage and talk with adolescents as they normally would. Overall, clinicians thought that the process worked well and shared similar views in terms of privacy considerations described earlier. They clarified with adolescents the purpose of e-monitoring and what information they could access from the e-monitor.

    Technical Considerations

    Despite attempts to keep the registration process simple, the log-on process was a barrier and “could be simpler.” Clinicians suggested an e-monitoring icon that could be integrated onto their current desktop to simplify access and serve as a reminder or prompt to check adolescents’ progress. Email alerts were widely discussed, and clinicians thought that these could serve a variety of purposes, including alerting of at-risk adolescents, reminding them to log in, and providing regular summaries in terms of adolescents’ engagement and progress with SPARX.

    Final Version of the E-Monitor

    In order to minimize burden on users and increase engagement with the program, mood was tracked in SPARX at the 3 time points originally proposed. Other suggestions from adolescents and clinicians are found in Textboxes 1 and 2, alongside considerations for an e-monitoring system to work effectively. Following the development and postimplementation phases, the e-monitor created (Figures 1 and 2) had the following features in (Textbox 3)

    Textbox 3. Key features of the final version of the e-monitor.
    View this box
    Figure 1. A final version of the e-monitoring dashboard following consultation with clinicians depicting fictional data of adolescents prescribed computerized cognitive behavioral therapy (cCBT) (SPARX [Smart, Positive, Active, Realistic, X-factor]), with an example of an alert. PHQ-A: Patient Health Questionnaire for Adolescents.
    View this figure
    Figure 2. Clinician view of progress, mood, and risks for an individual prescribed computerized cognitive behavioral therapy (cCBT) (fictional data). PHQ-A: Patient Health Questionnaire for Adolescents.
    View this figure


    Principal Findings

    Our study focused on the development and use of an e-monitoring tool in clinical practice. It is therefore a timely contribution, as the area of electronically supported psychotherapy is very much in its infancy [30,31]. Using a codesign process, we developed an e-monitoring tool that would enable clinicians to electronically supervise adolescents with depression completing Web-based cCBT. There are various options for providing supported therapy, and the proposed e-monitoring tool for SPARX not only provides a mechanism for clinicians to monitor an adolescent’s progress, but also affords them an opportunity to gain regular depression assessment data via the Internet in the absence of face-to-face therapy.

    In line with recent treatment preferences, clinicians were in favor of a system of monitoring adolescents with mild-to-moderate depression that embraced technology, various communication modalities, and a clinician-adolescent shared decision-making approach [32,33]. E-monitoring in tandem with face-to-face check-ins could potentially address clinician and adolescent concerns of no one providing oversight of a cCBT program such as SPARX, and this is a solution that has been previously identified in the literature [34]. While computer-based clinician support systems are increasingly gaining traction, it is also important that these do not place additional demands on the clinician’s workload [35]. Providing e-monitoring as well as face-to-face supervision may result in duplication of assessments and an increased clinician workload, but the need for such additional face-to-face assessment may reflect the reluctance of clinicians in depending on systems other than their own clinical judgment gathered from direct clinical contact.

    Clinicians in our study also recognized the potential difficulties associated with after-hours monitoring and responding to concerns. Therefore, monitoring arrangements should be preagreed with adolescents in relation to checking alerts and also with regard to their preferred communication methods (phone, text alerts, emails). As has been recommended by others, relevant contact numbers can be provided by a GP [36] should a user or client experience a crisis, or these details could be embedded in the cCBT program, or alternatively, integrated between hotlines or emergency services and the cCBT program. Whereas the latter would be a novel and useful contribution (because most programs do not have such integration built-in), there may be additional challenges to implement this, as it involves a wider group of stakeholders. Additionally, clinicians would like e-monitoring integrated with the existing patient data management systems they are familiar with for receiving and monitoring alerts. Logistically this is a challenging prospect, as clinicians in primary care utilize different systems, and there may be little motivation on the part of the developers of patient data management systems to include e-monitors (like the one we have developed) into their systems.

    There have been e-monitoring studies on various health conditions such as asthma and human immunodeficiency virus infection focusing on treatment adherence [37,38]. However, there are only a limited number of studies that have directly involved clinicians in a codesign process. For example, a Swiss qualitative study conducted with pharmacists on the implementation of an e-monitoring system for supporting treatment adherence similarly highlighted some of the difficulties we encountered, specifically the challenges of integrating an e-monitor into existing systems, but preexisting collaboration with physicians on codesigning the system helped [39]. Understandably clinicians would like to use an e-monitoring system that is simple to access, but also ensures privacy and confidentiality. Future developments should strive to fully integrate e-monitoring into existing data management systems, as challenging as this may be, so that e-monitoring is maximally useful to clinicians and acceptable to users [40].

    During the development phase of the e-monitor in our study, clinicians noted several concerns. However, when they were followed up after using the tool postimplementation, they found that some of the initially identified concerns did not eventuate. For example, clinicians expected a negative impact on the therapeutic relationship but this did not arise. Also, during the development phase, alerts were envisaged to be sent to clinicians to highlight adolescents who might be critical in terms of mood severity or risks. However, they used the tool somewhat differently from what had been proposed during the postimplementation phase, because clinicians flagged that it required additional time for e-monitoring to occur. So, a dedicated clinician at the primary care service reviewed the dashboard and alerts regularly, and informed the other monitoring clinicians about the progress of their adolescent users. This was not how the e-monitor was originally designed to work, but it demonstrated a practical adaptation at this particular service. As clinicians are also concerned about the practicalities of e-monitoring, when a GP becomes the main crisis-intervention provider, there have been suggestions of a “zero-risk” approach for GPs and sharing of clinical responsibility with crisis teams [36,41].

    Adolescents valued their privacy and preferred not to use their social networking login details. They also preferred a quick registration process, but one that either they or clinicians could undertake on their behalf. Moreover, adolescents wanted to discuss and customize e-monitoring arrangements and the timing and frequency of feedback they received via the e-monitor. While adolescents preferred their parents not to be involved in the e-monitoring process, they welcomed parallel monitoring with face-to-face check-ins with their clinician, despite reported difficulties with adolescents seeking help or expressing their concerns directly via face-to-face conversations [42]. Adolescents also wanted information on how to access support, especially after-hours when their clinician might be unavailable. As most adolescents have access to the Internet [43], this makes a Web-based e-monitoring system feasible in practice. With regard to SPARX (with or without e-monitoring), it has the capacity to advise adolescents to seek immediate help if they achieve a high score on mood symptoms or risk, but in practice, it may be difficult for the adolescent to ask for help. Information sheets could be provided prior to e-monitoring or during primary care consults, as depressed adolescents often do not recall details of consultations due to impaired concentration [36].


    This qualitative study involved adolescents and clinicians in New Zealand who, respectively, had experienced mental health difficulties and those who had managed adolescents with depression in primary care. The main limitation was that we were unable to take our interpretations of the data back to the participants for verification, which limited the study’s trustworthiness. Additionally, we were not able to undertake a systematic evaluation of the e-monitoring tool at multiple primary care sites or involve adolescents in the postimplementation phase. This was due to time and resource constraints. However, the multidisciplinary research team consisted of experienced clinicians who were directly involved in adolescent mental health and therefore very familiar with the issues.


    Depression often commences during a critical period of early development, so addressing the needs of adolescents yields not only significant improvements in their health care outcomes, but also longer-term benefits over their life span [44,45]. Clinician and adolescent views were acquired on e-monitoring for cCBT, an area with little previous empirical research and even less so in primary care where adolescents are usually seen for depression. Subsequently, an e-monitoring tool was designed in line with clinicians’ wishes for oversight of adolescents using SPARX. The e-monitor was also designed with the “needs and wants” of adolescents and included a system that was easy to use with information and alerts. We are currently exploring ways in which the e-monitor for SPARX can be integrated into patient data management systems. Overall, clinical progress, confidentiality or privacy, and technical issues need to be considered alongside discussion between the clinician and the adolescent prior to commencement of e-monitoring arrangements. The findings from this study are potentially applicable beyond the adolescent population and will be of interest to developers of the various cCBT packages who may be considering a monitoring or feedback mechanism.


    The authors would like to thank the clinicians and adolescents who participated in this study. Additional thanks to the staff and participants from Kapiti Youth Support. The study was funded by the Health Research Council of New Zealand, and the development of SPARX was funded by the New Zealand Ministry of Health. The national distribution of SPARX in New Zealand has been funded by the Prime Minister’s Youth Mental Health Project.

    Conflicts of Interest

    Intellectual property for SPARX is owned by Uniservices Ltd at The University of Auckland. The cocreators of SPARX (SM, KS, TF, MS, and ML) can benefit from its commercialization.

    Multimedia Appendix 1

    Semi-structured schedule used with focus groups.

    PDF File (Adobe PDF File), 33KB


    1. Costello EJ, Erkanli A, Angold A. Is there an epidemic of child or adolescent depression? J Child Psychol Psychiatry 2006 Dec;47(12):1263-1271. [CrossRef] [Medline]
    2. Lewinsohn PM, Pettit JW, Joiner Jr TE, Seeley JR. The symptomatic expression of major depressive disorder in adolescents and young adults. J Abnorm Psychol 2003 May;112(2):244-252. [Medline]
    3. Kessler RC, Avenevoli S, Merikangas KR. Mood disorders in children and adolescents: an epidemiologic perspective. Biol Psychiatry 2001 Jun 15;49(12):1002-1014. [Medline]
    4. Wilkinson P, Dubicka B, Kelvin R, Roberts C, Goodyer I. Treated depression in adolescents: predictors of outcome at 28 weeks. Br J Psychiatry 2009 Apr;194(4):334-341 [FREE Full text] [CrossRef] [Medline]
    5. Mariu KR, Merry SN, Robinson EM, Watson PD. Seeking professional help for mental health problems, among New Zealand secondary school students. Clin Child Psychol Psychiatry 2012 Apr;17(2):284-297. [CrossRef] [Medline]
    6. Computerised Cognitive Behaviour Therapy for the Treatment of Depression and Anxiety. London: National Institute for Health and Clinical Excellence; 2006.
    7. Richardson T, Stallard P, Velleman S. Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: a systematic review. Clin Child Fam Psychol Rev 2010 Sep;13(3):275-290. [CrossRef] [Medline]
    8. Kenardy J, Adams C. Computers in cognitive-behaviour therapy. Aust Psychol 1993;28(3):189-194. [CrossRef]
    9. Naslund JA, Marsch LA, McHugo GJ, Bartels SJ. Emerging mHealth and eHealth interventions for serious mental illness: a review of the literature. J Ment Health 2015;24(5):321-332. [CrossRef] [Medline]
    10. Dowrick C. Computerised self help for depression in primary care. Br Med J 2015;351:h5942. [CrossRef] [Medline]
    11. Perini S, Titov N, Andrews G. Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial. Aust N Z J Psychiatry 2009 Jun;43(6):571-578. [CrossRef] [Medline]
    12. Høifødt RS, Lillevoll KR, Griffiths KM, Wilsgaard T, Eisemann M, Waterloo K, et al. The clinical effectiveness of web-based cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled trial. J Med Internet Res 2013;15(8):e153 [FREE Full text] [CrossRef] [Medline]
    13. van der Vaart R, Witting M, Riper H, Kooistra L, Bohlmeijer ET, van Gemert-Pijnen LJ. Blending online therapy into regular face-to-face therapy for depression: content, ratio and preconditions according to patients and therapists using a Delphi study. BMC Psychiatry 2014;14(1):355 [FREE Full text] [CrossRef] [Medline]
    14. Wentzel J, van der Vaart R, Bohlmeijer ET, van Gemert-Pijnen JE. Mixing online and face-to-face therapy: how to benefit from blended care in mental health care. JMIR Ment Health 2016;3(1):e9 [FREE Full text] [CrossRef] [Medline]
    15. Kleiboer A, Donker T, Seekles W, van Straten A, Riper H, Cuijpers P. A randomized controlled trial on the role of support in Internet-based problem solving therapy for depression and anxiety. Behav Res Ther 2015 Sep;72:63-71. [CrossRef] [Medline]
    16. Nielssen O, Dear BF, Staples LG, Dear R, Ryan K, Purtell C, et al. Procedures for risk management and a review of crisis referrals from the MindSpot Clinic, a national service for the remote assessment and treatment of anxiety and depression. BMC Psychiatry 2015;15:304 [FREE Full text] [CrossRef] [Medline]
    17. Hunt S, Howells E, Stapleton B. The addition of a computerised cognitive behavioural therapy program, to a stepped care, primary care mental health service. The Journal of Primary Care Mental Health 2006;9(2):34.
    18. Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med 2003 Feb;33(2):217-227. [Medline]
    19. Stallard P, Richardson T, Velleman S, Attwood M. Computerized CBT (Think, Feel, Do) for depression and anxiety in children and adolescents: outcomes and feedback from a pilot randomized controlled trial. Behav Cogn Psychother 2011 May;39(3):273-284. [CrossRef] [Medline]
    20. Spence SH, Donovan CL, March S, Gamble A, Anderson R, Prosser S, et al. Online CBT in the treatment of child and adolescent anxiety disorders: issues in the development of BRAVE–ONLINE and Two Case Illustrations. Behav Cogn Psychother 2008;36(4):411-430. [CrossRef]
    21. Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MF. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. Br Med J 2012;344:e2598 [FREE Full text] [CrossRef] [Medline]
    22. Sawyer MG, Arney FM, Baghurst PA, Clark JJ, Graetz BW, Kosky RJ, et al. The mental health of young people in Australia: key findings from the child and adolescent component of the national survey of mental health and well-being. Aust N Z J Psychiatry 2001 Dec;35(6):806-814. [Medline]
    23. National Research Council & Institute of Medicine. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: The National Academies Press; 2009.
    24. Johnson JG, Harris ES, Spitzer RL, Williams JB. The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health 2002 Mar;30(3):196-204. [Medline]
    25. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. J Am Med Assoc 1999 Nov 10;282(18):1737-1744. [Medline]
    26. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357 [FREE Full text] [CrossRef] [Medline]
    27. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    28. Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval 2006 Jun 01;27(2):237-246. [CrossRef]
    29. Ford CA, Millstein SG, Halpern-Felsher BL, Irwin Jr CE. Influence of physician confidentiality assurances on adolescents' willingness to disclose information and seek future health care. A randomized controlled trial. J Am Med Assoc 1997 Sep 24;278(12):1029-1034. [Medline]
    30. Wilhelmsen M, Høifødt RS, Kolstrup N, Waterloo K, Eisemann M, Chenhall R, et al. Norwegian general practitioners' perspectives on implementation of a guided web-based cognitive behavioral therapy for depression: a qualitative study. J Med Internet Res 2014;16(9):e208 [FREE Full text] [CrossRef] [Medline]
    31. Montero-Marín J, Prado-Abril J, Botella C, Mayoral-Cleries F, Baños R, Herrera-Mercadal P, et al. Expectations among patients and health professionals regarding Web-based interventions for depression in primary care: a qualitative study. J Med Internet Res 2015;17(3):e67 [FREE Full text] [CrossRef] [Medline]
    32. Gray JAM. The Resourceful Patient. Oxford: eRosetta Press; 2002.
    33. Powell JA, Darvell M, Gray JAM. The doctor, the patient and the world-wide web: how the internet is changing healthcare. J R Soc Med 2003 Feb;96(2):74-76 [FREE Full text] [Medline]
    34. Rhodes H, Grant S. A review of computerised cognitive behavioural therapy (cCBT) for depression. Cumbria Partnership Journal of Research Practice and Learning 2012;2(1):2-9.
    35. Miller P, Phipps M, Chatterjee S, Rajeevan N, Levin F, Frawley S, et al. Exploring a clinically friendly web-based approach to clinical decision support linked to the electronic health record: design philosophy, prototype implementation, and framework for assessment. JMIR Med Inform 2014 Jul;2(2):e20 [FREE Full text] [CrossRef] [Medline]
    36. Falconer M, Arroll B. Beating the Blues the online way. New Zealand Doctor. 2012. p. 34-35   URL:
    37. Chan AHY, Reddel HK, Apter A, Eakin M, Riekert K, Foster JM. Adherence monitoring and e-health: how clinicians and researchers can use technology to promote inhaler adherence for asthma. J Allergy Clin Immunol Pract 2013;1(5):446-454. [CrossRef] [Medline]
    38. Anderson 3rd WC, Szefler SJ. New and future strategies to improve asthma control in children. J Allergy Clin Immunol 2015 Oct;136(4):848-859. [CrossRef] [Medline]
    39. Marquis J, Schneider MP, Spencer B, Bugnon O, Du Pasquier S. Exploring the implementation of a medication adherence programme by community pharmacists: a qualitative study. Int J Clin Pharm 2014 Oct;36(5):1014-1022. [CrossRef] [Medline]
    40. De Bleser L, De Geest S, Vincke B, Ruppar T, Vanhaecke J, Dobbels F. How to test electronic adherence monitoring devices for use in daily life: a conceptual framework. Comput Inform Nurs 2011 Sep;29(9):489-495. [CrossRef] [Medline]
    41. de Montalk J. 'Zero-risk' e-tool beats the blues. New Zealand Doctor 2012:30.
    42. Coyle D, McGlade N, Doherty G, O'Reilly G. Exploratory evaluations of a computer game supporting cognitive behavioural therapy for adolescents. 2011 Presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 2011; Vancouver p. 2937-2946. [CrossRef]
    43. Madden M, Lenhart A, Duggan M, Cortesi S, Gasser U. Teens and Technology 2013 Pew Research Center. 2013.   URL: [accessed 2016-09-03] [WebCite Cache]
    44. Simon GE, Ludman EJ. It's time for disruptive innovation in psychotherapy. Lancet 2009 Aug 22;374(9690):594-595. [CrossRef] [Medline]
    45. Weller EB, Weller RA. Depression in adolescents growing pains or true morbidity? J Affect Disord 2000 Dec;61(Suppl 1):9-13. [Medline]


    CBT: cognitive behavioral therapy
    cCBT: computerized cognitive behavioral therapy
    GP: general practitioner
    PHQ-A: Patient Health Questionnaire for Adolescents
    SPARX: Smart, Positive, Active, Realistic, X-factor

    Edited by G Eysenbach; submitted 16.04.16; peer-reviewed by C Eichenberg, P Yip, R van der Vaart; comments to author 27.07.16; revised version received 05.09.16; accepted 13.12.16; published 11.01.17

    ©Frederick Sundram, Susan J Hawken, Karolina Stasiak, Mathijs FG Lucassen, Theresa Fleming, Matthew Shepherd, Andrea Greenwood, Raechel Osborne, Sally N Merry. Originally published in JMIR Mental Health (, 11.01.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.