Abstract
Background: Therapist-guided, internet-delivered cognitive behavioral therapy (guided ICBT) appears to be efficacious for depression, panic disorder (PD), and social anxiety disorder (SAD) in routine care clinical settings. However, implementation of guided ICBT in specialist mental health services is limited partly due to low referral rates from general practitioners (GP), which may stem from lack of awareness, limited knowledge of its effectiveness, or negative attitudes toward the treatment format. In response, self-referral systems were introduced in mental health care about a decade ago to improve access to care, yet little is known about how referral pathways may affect treatment outcomes in guided ICBT.
Objective: This study aims to compare the overall treatment effectiveness of GP referral and self-referral to guided ICBT for patients with depression, PD, or SAD in a specialized routine care clinic. This study also explores if the treatment effectiveness varies between referral pathways and the respective diagnoses.
Methods: This naturalistic open effectiveness study compares treatment outcomes from pretreatment to posttreatment and from pretreatment to 6-month follow-up across 2 referral pathways. All patients underwent module-based guided ICBT lasting up to 14 weeks. The modules covered psychoeducation, working with negative or automatic thoughts, exposure training, and relapse prevention. Patients received weekly therapist guidance through asynchronous messaging, with therapists spending an average of 10‐30 minutes per patient per week. Patients self-reported symptoms before, during, immediately after, and 6 months posttreatment. Level and change in symptom severity were measured across all diagnoses.
Results: In total, 460 patients met the inclusion criteria, of which 305 were GP-referred (“GP” group) and 155 were self-referred (“self” group). Across the total sample, about 60% were female, and patients had a mean age of 32 years and average duration of disorder of 10 years. We found no significant differences in pretreatment symptom levels between referral pathways and across the diagnoses. Estimated effect sizes based on linear mixed modeling showed large improvements from pretreatment to posttreatment and from pretreatment to follow-up across all diagnoses, with statistically significant differences between referral pathways (GP: 0.97‐1.22 vs self: 1.34‐1.58, P<.001-.002) and for the diagnoses separately: depression (GP: 0.86‐1.26, self: 1.97‐2.07, P<.001-.02), PD (GP: 1.32‐1.60 vs self: 1.64‐2.08, P=.06-.02) and SAD (GP: 0.80‐0.99 vs self: 0.99‐1.19, P=.18-.22).
Conclusions: Self-referral to guided ICBT for depression and PD appears to yield greater treatment outcomes compared to GP referrals. We found no difference in outcome between referral pathway for SAD. This study underscores the potential of self-referral pathways to enhance access to evidence-based psychological treatment, improve treatment outcomes, and promote sustained engagement in specialist mental health services. Future studies should examine the effect of the self-referral pathway when it is implemented on a larger scale.
doi:10.2196/68165
Keywords
Introduction
Background
Depression and anxiety disorders are recognized as major contributors to global disability, carrying significant societal costs and having high personal impact []. In 2019, nearly 600 million people worldwide were affected by these conditions. Throughout life, depression and anxiety disorders are approximately 50% more common in women than in men []. Broadly accessible treatment is required to reduce this burden [], yet a significant treatment gap remains between the need for and access to adequate care []. This gap is driven by a variety of factors, including limitations in available health care services, financial barriers, avoidance of help-seeking, lack of mental health literacy, and stigma []. The dominant model of treatment delivery—face-to-face treatment with trained mental health professionals in clinical settings—further restricts the widespread dissemination of mental health care []. Even in high-income countries, where access to care is more readily available, only about one-third of those with major depressive disorders receive formal mental health care [].
Pharmacological and psychological therapies have demonstrated equal effects in treating depression and anxiety disorders [,]. However, psychological therapy is often preferred by patients over medication due to having fewer side effects and better long-term outcomes [,]. Cognitive behavioral therapy (CBT) is the psychological treatment with the strongest empirical support [] and it is the recommended first-line treatment for these disorders [,]. Internet-delivered cognitive behavioral therapy (ICBT) delivers evidence-based CBT specifically targeting, but not limited to, depression and anxiety disorders []. ICBT offers several practical advantages that help address the treatment gap, including reduced travel time and expenses, greater flexibility to fit around individuals’ daily schedules, and the potential to overcome stigma-related barriers through increased anonymity [,]. Additionally, the internet-delivered treatment format is less time-consuming for the therapist, thus it is scalable and affordable without compromising the quality of care []. These factors make ICBT an attractive option for expanding access to mental health treatment, particularly in areas with limited resources or where traditional face-to-face therapy is not readily available.
Systematic reviews have found the effect of guided ICBT for depression and anxiety disorders to be no different from that of face-to-face CBT [-]. Guided ICBT for depression and anxiety is found to work well in routine care clinics and tends to replicate results found in efficacy studies in Sweden, Denmark, Norway, Canada, and Australia [-] and to have long-term effects []. However, the implementation of guided ICBT in specialist mental health care has been slow, partly due to lack of knowledge, prejudice, and negative attitudes among health care professionals and general practitioners (GPs) []. This is concerning, as GPs in primary care often serve as gatekeepers and are responsible for initiating referrals to secondary care and specialist clinics.
The lack of referral from GPs to guided ICBT [] has led to efforts enhancing access to care, with self-referral being proposed as a way to improve access to psychological therapies []. Self-referral implies that patients can seek the service from secondary care or specialist clinics, bypassing the need for referrals from GPs []. Self-referral to ICBT opens a pathway to evidence-based psychological care especially for individuals who never reach specialist clinics or mention their problems when consulting their GP []. Some have suggested that self-referral may attract more motivated patients [], which could influence treatment engagement and improve outcomes []. Participants in guided ICBT trials tend to be more educated []. Since higher education is associated with better health literacy and access to and understanding of health information [,], patients self-referred to ICBT may thus be more responsive to treatment, which could help explain potential differences in treatment outcomes.
In addition, self-referral pathways are believed to empower patients by giving them greater control over their health care []. According to Self-Determination Theory (SDT), having greater control over the decision to seek care may foster autonomy, which in turn could enhance motivation and engagement with treatment []. For example, patients who self-refer may feel more motivated, ready, and confident in their choice to pursue ICBT, which can lead to greater engagement in therapy. When autonomy is supported by a sense of competence, such as patients feeling capable of managing their treatment, it can strengthen intrinsic motivation. This, combined with supportive feedback from ICBT therapists and a sense of relatedness, can also strengthen intrinsic motivation and may thus encourage continued participation and adherence to treatment plans [].
However, while self-referral can improve access to psychological therapies, it also presents challenges. One concern is that self-assessment tools for depression and anxiety are not validated for lay self-diagnosis in this context. This is particularly important when accurate diagnosis is crucial for providing appropriate evidence-based treatment []. It has been argued that self-referral in general may result in unnecessary, costly, or even harmful interventions. Conversely, it could also lead to reduced patient responsibility, causing symptoms to be dismissed or action to be delayed, potentially leading to harm []. This reliance on individual health care–seeking behaviors has been identified as a factor that can contribute to widening socioeconomic inequalities []. Additionally, self-referral may lead to oversaturation of specialist health care and potentially widen already existing health inequalities by primarily attracting younger, well-educated woman. However, this concern remains underexplored and may be context-dependent [].
The practice of using self-referral pathways to specialist care varies across countries and clinical domains, with physiotherapy and mental health services being among the most common []. Self-referral is well studied in the field of physiotherapy and is an available pathway to musculoskeletal care in many countries []. Consistent yet limited evidence suggests that self-referral for musculoskeletal care yields clinical outcomes comparable to GP referrals []. Research comparing different referral pathways to mental health services remains limited. In a recent systematic review examining who benefits from guided internet-based interventions across mental health diagnoses, 88 predictors and moderators of treatment outcome were analyzed but referral pathway to treatment was not included []. A recent study recommends investigating referral pathways on patient outcomes []. However, some studies comparing referral pathways to psychological care already exist.
First, in a study on GP referral and self-referral to psychological treatment for patients with severe health anxiety, Hoffmann et al [] examined the accuracy of these referral pathways in recruiting patients with treatment-demanding symptom levels. The accuracy was assessed by comparing the proportion of patients in each referral group who met the treatment criteria, with results significantly favoring self-referral. One reason for this difference was that several GP-referred patients did not attend the clinical diagnostic interview and therefore were excluded from the study. The findings suggest that self-referral may be a more accurate method for recruiting patients with severe health anxiety, as self-referred patients not only meet the criteria for treatment but also appear to be more motivated to participate in it [].
Referral pathway has also been studied in relation to how consistently patients attend psychological therapy sessions within Improving Access to Psychological Therapies (IAPT) services []. When comparing GP referral, GP-initiated self-referral, and true self-referral to IAPT, no significant differences were found between referral pathways and attendance at the subsequent therapy sessions. Moreover, the study examined the patient’s preferred pathway and found that those who had a GP-initiated self-referral later stated a preference for the GP to take full responsibility for the referral process. Accordingly, 60% of the true self-referrers stated that they preferred to self-refer again if they needed additional services from IAPT [].
Although studies on treatment outcomes across referral pathways to psychological therapy are scarce, a notable exception is an observational study comparing GP referral and self-referral to 2 similar ICBT treatments for depression and/or anxiety []. In this study, patients from both referral pathways reported significant symptom reduction; however, those who self-referred showed larger effect sizes both at posttreatment and at the 3-month follow-up compared to those referred by their GPs [].
No studies have yet investigated the role of referral pathway on treatment outcomes for guided ICBT for depression and anxiety disorders to a specialized routine care clinic. Based on the results from the comparison of GP referral and self-referral pathways to ICBT [,], we hypothesize that individuals who self-refer to specialized mental health care services will experience greater treatment effectiveness from pretreatment to posttreatment and for pretreatment to 6-month follow-up compared to those referred by GPs. Additionally, we will explore differences in treatment effectiveness across the specific diagnoses in relation to referral pathways.
Aim
The aim of this study was to compare the overall treatment effectiveness across different referral pathways—GP-referred and self-referred—in guided ICBT for moderate depression, panic disorder (PD), and social anxiety disorder (SAD). We also explore whether differences in treatment effectiveness between the referral pathways vary across the 3 diagnoses.
Methods
Ethical Considerations
This study was approved by the Regional Committee for Medical Research Ethics (REK) 2014/2175. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation as well as with the principles of the Declaration of Helsinki []. The original written informed consent covers secondary analysis without additional consent. Data were deidentified, with direct personal identifiers removed and the key linking IDs stored separately on an inaccessible server. No compensation was provided for participation.
Setting
The data collection for this study was conducted at the eCoping clinic, a specialized routine mental health care clinic at Haukeland University Hospital in Bergen, Norway. All patients were referred to the eCoping clinic either from their GP or by themselves through direct contact with the eCoping team, resulting in both GP referrals and self-referrals. This study presents data from GP-referred patients included between September 2014 and May 2019 and self-referred patients included between September 2016 and May 2019.
Design
This study was a naturalistic open effectiveness study with repeated assessments for primary treatment outcomes and a 6-month follow-up for patients with moderate depression, PD, and SAD undergoing therapist-guided ICBT.
Referral to Treatment
Several approaches were used to increase knowledge about eCoping among the GPs. First, there were face-to-face educational visits conducted by the eCoping team. Second, GPs were provided with test-user accounts for the eCoping program to familiarize themselves with the treatment, however, none of the GPs logged in. Third, information about the treatment was presented in GPs’ waiting rooms through flyers and short messages on information screens. In addition, promotion of eCoping was carried out through the local newspapers to highlight the new referral option. Finally, for a short period, Facebook ads about the possibility to self-refer were targeted at the local population.
GPs evaluated patients using their clinical judgment to assess symptom severity to determine the need for specialized care services; if deemed necessary, the GP authored a referral to the eCoping clinic. Patients who self-referred sent an email with their contact details to an address available on the eCoping website []. Subsequently, an eCoping therapist conducted a clinical interview by telephone to assess symptom severity and the treatment’s relevance. A summary of the interview was generated as a self-referral.
A specialist in clinical psychology reviewed all referrals regardless of referral pathway in accordance with national priority guidelines [] to determine eligibility for specialized care treatment.
As the study was conducted within routine care, the eligibility assessment and the inclusion and exclusion criteria were identical across referral pathways, ensuring that the sample was unbiased with respect to referral source. Inclusion criteria for all study patients were: (1) being 18 years of age or older; (2) diagnosed either with major depressive episode, SAD, or PD; (3) if using antidepressants, being on a stable dosage over the previous four weeks; and (4) fluent in oral and written Norwegian. Exclusion criteria for all study patients were: (1) current suicidal ideation, (2) current psychosis, (3) current substance abuse, (4) using benzodiazepines daily, (5) immediate need of other treatment, and (6) no access to the internet. Written informed consent was obtained from all study patients prior to data collection. We have no data on individuals who were screened out during the eligibility process as these patients did not sign informed consent forms. Patients who met the criteria for treatment received a scheduled appointment for a face-to-face consultation.
Procedure
During the face-to-face consultation, all patients underwent a diagnostic interview with the Mini-International Neuropsychiatric Interview (MINI) []. Based on the MINI, patients deemed unfit for eCoping were excluded and rereferred to a more suitable treatment option. The treatment program allocation was determined based on the MINI assessment.
Training
All therapists at the eCoping clinic were colocated for 1-2 days per week when working with guided ICBT, with an ordinary workload during the rest of the week. In addition to a 1-year continuing education program, the therapists received weekly peer supervision and monthly expert supervision from the Internet Psychiatry Clinic in Stockholm.
Treatment
For depression, the guided ICBT program included 8 text-based modules including psychoeducation, behavioral activation, and cognitive reappraisal and relapse prevention. PD was addressed with 9 text-based modules with psychoeducation, working with automatic thoughts, behavioral experiments, in vivo exposure, and relapse prevention. Similarly, SAD treatment comprised 9 text-based modules including psychoeducation, working with automatic thoughts, behavioral experiments, shifting focus, and relapse prevention. The treatments are described in detail in previous publications [,,]. The therapists adhered to the treatment protocol and provided uniform treatment to all patients regardless of referral pathway. The treatment programs were provided on a secure web platform that was state-of-the-art when data collection started in 2014.
Treatment time for the 3 diagnoses was up to 14 weeks. Irrespective of treatment program, each patient was expected to spend 7‐10 days per module; access to the next module was gained upon finishing the previous one. Each module required approximately 45 minutes to complete.
After each completed module or at least once per week, a therapist gave feedback and guidance tailored to individual patient needs based on their worksheets, symptom assessment, and emails, while also introducing them to the next module. All feedback and communication were enabled asynchronously through a secure email system. Therapists spent an average of 10‐30 minutes per patient per week. Patients not heard from for 1 week were contacted by the therapist via an SMS text message to encourage them to continue to work through the program. When necessary, phone calls could be made to solve problems, discuss motivation, or simply get in touch with an inactive patient.
Primary Outcomes
All self-report measures and questionnaires were administered via the internet and made accessible at the end of each module. Patients completed the measures and questionnaires pretreatment, after each module, posttreatment, and at the 6-month follow-up. The programs for depression, PD, and SAD had the following primary outcome measures:
- Depression: Montgomery Åsberg Depression Rating Scale, Self-rating version (MADRS-S) []. The MADRS-S comprises 9 items rated on a Likert scale from 0‐6 (total score range: 0‐54), where higher scores indicate more severe depression. The scale has been found to be sensitive to change [,] and has shown high correlations between expert ratings and self-reports []. Internal consistency measured with Cronbach α yielded 0.77 for patients with GP referral and 0.82 for self-referred patients.
- Panic disorder: Body Sensation Questionnaire (BSQ) []. The BSQ has been found sensitive to symptom change during treatment []. The BSQ comprises 16 items rated on a 5-point Likert scale (total score range: 16‐80), where higher scores indicate a higher level of fear and sensitivity to bodily sensations commonly experienced during autonomic nervous system arousal. Cronbach α yielded 0.84 for GP-referred patients and 0.88 for self-referred patients and showed good internal consistency reliability.
- Social anxiety disorder: Social Phobia Scale (SPS) []. The SPS measures social phobia and the distress of being observed or watched while performing daily activities in the presence of others []. The SPS entails 20 questions rated on a 5-point Likert scale (total score range: 0‐100), where higher scores indicate higher anxiety of being observed or scrutinized. The scale has shown good reliability and validity [,], as well as discriminant validity in distinguishing individuals diagnosed with SAD from both healthy controls and individuals with other anxiety disorders []. Internal consistency measured with Cronbach α yielded 0.91 for GP-referred patients and 0.93 for self-referred patients.
To address the main aim of this study, we combined the outcome measures for the 3 treatment modalities. We harmonized the outcome total scores and computed a common harmonized outcome measure (see Equation 1), thereby increasing the net sample and the statistical power for the analyses []. To ensure comparability across different measurements, this formula normalizes the total score by first subtracting the minimum possible value, then dividing by the range (maximum possible value minus minimum), and finally multiplying by 100. This transformation ensures that all scores are expressed on a common scale, reducing biases introduced by different measurement units. However, the harmonization removes the original scale’s absolute meaning and assumes that all scales represent the same underlying construct in a comparable way, which may not always be true if the scales have different distributions or nonlinear relationships. Moreover, harmonization was only feasible at the total score level. At the item level, where variables were ordinal, no harmonization was possible since no patients could have information on all 3 outcome variables. Therefore, we were unable to calculate internal consistency for the harmonized outcome measure.
(1)
Statistics
Data preparation and calculation of descriptive statistics and bivariate analyses, including percentages, means, standard deviations, and cross tabulations with χ2 tests, were conducted using IBM SPSS (version 29; IBM Corp) []. Effect sizes from pretreatment to posttreatment and from pretreatment to 6-month follow-up are reported as Cohen d, based on pooled standard deviations []. All measurement points (completed modules) were used for the analyses; however, the focus is the between-group (GP-referred vs self-referred) difference in pretreatment levels and changes from pretreatment to posttreatment and from pretreatment to 6-month follow-up. We performed analyses of the treatment outcome using linear mixed modeling (LMM). LMM is a recommended statistical method for handling missing data under the assumption of missing at random and uses all available data for estimation []. We analyzed levels and changes in data with a random intercept and fixed slope model. First, unconditional models including the time variable were tested. Time was defined as modules, giving changes in outcomes per module. To compare group differences over time, we added the referral group, both as a main effect and in an interaction effect with time. The reliable change index (RCI) was calculated using individual-level changes from pretreatment to posttreatment and from pretreatment to the 6-month follow-up, based on observed data for patients in each referral pathway and diagnosis. The RCI calculates whether changes in symptoms are reliable and not caused by measurement error []. Symptom level was considered to have improved if the outcome measure indicated a reliable change, as defined by the RCI []. The RCI was calculated with the formula 1.96 × SD × √2(1-Rel), where SD is the observed standard deviation and Rel is the internal consistency at pretreatment assessment for each referral pathway and outcome measure. Improvement was defined by a negative RCI change, while deterioration was defined by a positive RCI change. Sensitivity models based on multiple imputation (MI) were analyzed to explore possible differences in levels and changes related to missing data for the 2 groups, assuming missing data were missing at random []. Pretreatment and longitudinal information were used as predictors of plausible values, and 50 data sets were generated. To account for the clustered data structure in the imputation process, data were transformed into wide format and analyzed with latent growth curve models. These models were parameterized identically to the corresponding linear mixed effect models, ensuring the same number of parameters and constraints []. Imputations and analyses were conducted using Mplus 8.10 [].
Results
Patients
A total of 460 patients provided informed consent across the 2 referral pathways. Patient characteristics are shown in 
Table 1. Of the total sample, approximately two-thirds were referred by a GP (GP-referred), while about one-third were self-referred. There were more females in both groups (GP-referred: 176/289, 60.9%; self-referred: 106/151, 70.2%). The mean age across the total sample was approximately 32 years, with an average duration of complaints of about 10 years in both groups. The only statistically significant difference between the 2 groups was that a higher proportion of those who self-referred had obtained a university-level education. The distribution across the diagnosis-specific treatment programs was approximately 22% depression, 38% PD, and 41% SAD. Among patients in the depression group, the referral pathway was approximately equally distributed. In contrast, the pathway distribution for both the PD group and the SAD group was about two-thirds GP referrals and one-third self-referrals.
| Demographics | Total group (N=460a) | Self-referred (n=155) | GP-referredb (n=305) | P value |
| Gender, nc/N (%) | .054 | |||
| Female | 282/440 (64.1) | 106/151 (70.2) | 176/289 (60.9) | |
| Male | 158/440 (35.9) | 45/151 (29.8) | 113/289 (39.1) | |
| Age (years), n/N, mean (SD) | 460/460, 32.5 (11.0) | 155/155, 31.9 (10.3) | 305/305, 32.7 (11.4) | .12 |
| Relationship status, n/N (%) | .32 | |||
| Married/cohabitant | 225/435 (51.7) | 83/151 (55) | 142/284 (50) | |
| Single | 210/435 (48.3) | 68/151 (45) | 142/284 (50) | |
| Education, n/N (%) | <.001 | |||
| Primary level | 60/439 (13.7) | 14/151 (9.3) | 46/288 (16) | |
| Secondary level | 193/439 (44) | 50/151 (33.1) | 143/288 (49.7) | |
| Tertiary level | 186/439 (42.4) | 87/151 (57.6) | 99/288 (34.4) | |
| Years with complaints, n/N mean (SD) | 426/460 10.2 (9.5) | 147/151, 9.80 (9.43) | 279/305, 10.35 (9.62) | .54 |
| Treatment program, n (%) | ||||
| Depression | 101 (21.7) | 48 (31) | 53 (17.4) | |
| Panic disorder | 172 (37.4) | 56 (36.1) | 116 (38) | |
| Social anxiety disorder | 187 (40.7) | 51 (32.9) | 136 (44.6) | |
aN: number of patients.
bGP-referred: referred by general practitioners.
cn: number of patients in that subgroup.
Attrition and Adherence
In the depression group (N=101), 97 patients (96%) completed the MADRS-S assessment pretreatment, 66 (65.3%) completed it at posttreatment, and 41 (40.6%) completed it at the 6-month follow-up. The amount of missing data was found to be equal between the 2 groups (GP-referred: mean 4.9, SD 2.9; self-referred: mean 4.6, SD 3.3; t99=0.50, P=.62). In the PD group (N=172), 156 patients (90.7%) completed the BSQ pretreatment, with 111 (64.5%) and 67 (38.9%) completing the assessment at posttreatment and follow-up, respectively. No difference in the amount of missing data was found between the 2 groups (GP-referred: mean 5.0, SD 3.5; self-referred: mean 3.9, SD 3.1; t170=2.0, P=.051). For the SAD group (N=187), 177 patients (95.2%) completed the SPS pretreatment, followed by 99 (52.9%) at posttreatment and 59 (31.6%) at follow-up, with no difference in the amount of missing data between the groups (GP-referred: mean 5.7, SD 3.6; self-referred: mean 4.9, SD 3.8; t185=1.50, P=.14). Details on the observed diagnosis-specific outcome measures are provided in Table S1 in .
Primary Outcomes
LMM results showed that, when harmonizing the outcome measures for all 3 diagnoses, significant symptom reduction was evident for both referral pathways from pretreatment to posttreatment and pretreatment to 6-month follow-up. Overall, patients who self-referred demonstrated significantly greater estimated symptom reduction from pretreatment to posttreatment and pretreatment to 6-month follow-up compared to those referred by GPs. The estimated scores from the LMM showed that the MADRS-S level decreased over time (Table 2). Self-referred patients showed no significant difference in depression scores from GP-referred patients at the pretreatment assessment, but a statistically greater reduction from pretreatment to posttreatment and pretreatment to 6-month follow-up. More details on the level and change in the estimated outcome measures are provided in Table S2 in . Table 3 shows estimated means at pretreatment, posttreatment, and follow-up together with estimated effect sizes. Overall, we found large effect sizes (>0.8) over time.
Figure 1 depicts the corresponding level and change in the LMM. Overall, the figure shows that the estimated harmonized levels for both referral pathways decreased over time, with statistically significant greater reductions from pretreatment to posttreatment and from pretreatment to 6-month follow-up for those who self-referred. The pattern of changes in MADRS-S showed a somewhat different picture compared to the other results, with a stronger reduction in the self-referred group in the pretreatment to posttreatment interval, but no further reduction from posttreatment to follow-up. BSQ levels for both referral pathways decreased over time. Self-referred patients showed a significantly greater reduction in BSQ scores from pretreatment to 6-month follow-up and a temporarily greater reduction after completing module 2. The LMM scores showed that the estimated SPS levels and changes did not differ between the referral pathways from pretreatment to 6-month follow-up. Self-referred patients showed a temporary significantly greater reduction from pretreatment to module 3.
| Harmonized outcome | MADRS-Sa | BSQb | SPSc | |||||
| b | P value | b | P value | b | P value | b | P value | |
| Pred | 46.13 | <.001 | 23.82 | <.001 | 43.15 | <.001 | 40.12 | <.001 |
| Poste | −15.67 | <.001 | −5.15 | <.001 | −12.57 | <.001 | −11.68 | <.001 |
| Follow-upf | −19.63 | <.001 | −7.54 | <.001 | −15.21 | <.001 | −14.40 | <.001 |
| Group differences | ||||||||
| g | −2.26 | .23 | 0.50 | .76 | −1.42 | .44 | −2.55 | .35 |
| −5.85 | <.001 | −6.60 | <.001 | −3.00 | .06 | −2.75 | .18 | |
| −5.72 | .002 | −4.80 | .02 | −4.62 | .02 | −2.93 | .22 | |
aMADRS-S: Montgomery Åsberg Depression Rating Scale, Self-rating version.
bBSQ: Body Sensation Questionnaire.
cSPS: Social Phobia Scale.
dPre: pretreatment.
ePost: posttreatment.
fFollow-up: 6-month follow-up.
gReference group: general practitioner–referred.
| Prea | Postb | Follow-upc | ||||
| Meand | Mean | Effect size | Mean | Effect size | ||
| Harmonized outcomee | ||||||
| GPf | 46.13 | 30.46 | −0.97 | 26.50 | −1.22 | |
| Selfg | 43.87 | 22.35 | −1.34 | 18.52 | −1.58 | |
| MADRS-Sh | ||||||
| GP | 23.82 | 18.67 | −0.86 | 16.28 | −1.26 | |
| Self | 24.32 | 12.57 | −1.97 | 11.98 | −2.07 | |
| BSQi | ||||||
| GP | 43.15 | 30.58 | −1.32 | 27.94 | −1.60 | |
| Self | 41.73 | 26.16 | −1.64 | 21.90 | −2.08 | |
| SPSj | ||||||
| GP | 40.12 | 28.44 | −0.80 | 25.72 | −0.99 | |
| Self | 37.57 | 23.14 | −0.99 | 20.24 | −1.19 | |
aPre: pretreatment.
bPost: posttreatment.
cFollow-up: 6 months follow-up.
dMean: model-estimated mean values.
eHarmonized outcome: harmonization of MADRS-S, BSQ, and SPS.
fGP: GP-referred.
gSelf: self-referred.
hMADRS-S: Montgomery Åsberg Depression Rating Scale, Self-rating version.
iBSQ: Body Sensation Questionnaire.
jSPS: Social Phobia Scale.
Reliable Change
Overall, of those who self-referred, statistically significantly more patients showed an improvement in reliable change index (RCI) from pretreatment to posttreatment compared to those referred by their GP (Table 4). From pretreatment to follow-up, we found statistically significant differences only for self-referred patients with PD. RCI improvement based on the observed data required a reduction of at least 8 and 9 points on the MADRS-S (GP and self-referred), 12 and 11 points on the BSQ (GP and self-referred), and 13 points on the SPS (both referral pathways). There were statistically significant differences in RCI improvement between referral pathways among patients with depression from pretreatment to posttreatment, as well as among patients with PD from pretreatment to posttreatment and from pretreatment to follow-up. In all cases, symptom improvement favored those who self-referred. Of patients with depression or PD who reported no change in symptom level at posttreatment and at the 6-month follow-up, the majority were GP-referred. In contrast, the reliable change in reported symptom level between the referral pathway among patients with SAD was minor at both posttreatment and the 6-month follow-up. The proportions of patients with symptom improvement and no change in SPS were evenly distributed between the referral pathways.
| Totala | MADRS-Sb | BSQc | SPSd | |||||
| GPe, n (%) | Selff, n (%) | GP, n (%) | Self, n (%) | GP, n (%) | Self, n (%) | GP, n (%) | Self, n (%) | |
| Posttreatment | ||||||||
| 81 (44.8) | 64 (64) | 9 (27.3) | 21 (70) | 37 (51.4) | 27 (71.1) | 35 (46.1) | 16 (50) | |
| 94 (51.9) | 36 (36) | 21 (63.6) | 9 (30) | 35 (48.6) | 11 (28.9) | 38 (50) | 16 (50) | |
| 6 (3.3) | 0 (0) | 3 (9.1) | 0 (0) | 0 (0) | 0 (0) | 3 (3.9) | 0 (0) | |
| g | <.001 | .002 | .047 | .51 | ||||
| Follow-uph | ||||||||
| 56 (57.7) | 50 (72.5) | 9 (45) | 14 (66.7) | 28 (66.7) | 22 (91.7) | 19 (54.3) | 14 (58.3) | |
| 40 (41.2) | 19 (27.5) | 11 (55) | 7 (33.3) | 14 (33.3) | 2 (8.3) | 15 (42.9) | 10 (41.7) | |
| 1 (1) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 1 (2.9) | 0 (0) | |
| .04 | .16 | .02 | .69 | |||||
aTotal: Sum of improved, sum of no change, and sum of deterioration from MADRS-S, BSQ, and SPS.
bMADRS-S: Montgomery Åsberg Depression Rating Scale, Self-rating version.
cBSQ: Body Sensation Questionnaire.
dSPS: Social Phobia Scale.
eGP: GP-referred.
fSelf: self-referred.
gChi-square test for differences in distributions for improved, no change, and deterioration between referral pathways.
hFollow-up: 6-month follow-up.
Missing Data Sensitivity Analyses
The sensitivity analyses using MI indicated that the estimated pretreatment MADRS-S mean score for the GP-referred group was identical to the full-information maximum likelihood (FIML) estimate () . However, the reduction in symptoms from pretreatment to 6-month follow-up was smaller in the MI analysis compared to the original analysis (FIML=–7.54 vs MI=–6.55). Additionally, we also found a difference in the group effect of self-referral at the 6-month follow-up (FIML=–4.80 vs MI=5.49). This suggests less reduction and less difference in reduction in depression symptom levels in the net population based on the sample with the observed information compared to the total population with intact information on all variables and all measurement points. Thus, the main results may have shown somewhat overestimated reductions. A similar pattern was found for BSQ (GP follow-up levels: FIML=–15.21 vs MI=–14.21). For SPS, the difference was smaller but in the opposite direction (GP follow-up levels: FIML=–14.40 vs MI=–15.26). Differences from pretreatment to posttreatment are also reported in ; however, since there were fewer missing data at this assessment point, the results closely matched those of the main analysis.
Discussion
This study compared the overall treatment effectiveness of guided ICBT across GP-referred and self-referred pathways for patients with moderate depression, PD, or SAD. We also explored whether differences in treatment effectiveness between the referral pathways varied across the diagnoses separately. All patients underwent guided ICBT in a specialized routine mental health care clinic.
Principal Findings
Overall, there were large effect sizes of guided ICBT from pretreatment to posttreatment and from pretreatment to 6-month follow-up, aligning with the large effects reported in a systematic review of routine care practice on the effectiveness of guided ICBT for depression and anxiety [], and further aligning with treatment results from previous investigations from the same specialized routine care clinic [,,].
Our overall results support the hypothesis that patients who self-refer have significantly larger treatment effectiveness compared to those referred by their GP. This is evident from the relatively large estimated effect sizes in the harmonized scores in the self-referred group (effect sizes: 1.34‐1.58) compared to the GP-referred group (effect sizes: 0.97‐1.22). Additionally, there was a significant difference in reliable change between the self-referred and GP-referred groups. These results are noteworthy, especially since no difference in pretreatment symptom level was found across referral pathways. Our results favoring self-referral are consistent with those of Staples et al [] where self-referred patients showed greater effect sizes compared to GP-referred patients both from pretreatment to posttreatment and from pretreatment to the 3-month follow-up. Neither our study nor previous studies on referral pathway to ICBT [] investigated the potential mechanisms behind the findings that self-referred patients have better treatment effectiveness, leaving open the possibility that unaccounted-for factors may influence the results.
One such factor could be differences in motivation and autonomy among patients across the referral pathways. According to SDT, internal motivation thrives when individuals experience autonomy, capability, and relatedness []. Self-referred patients may feel a greater sense of empowerment and autonomy by actively choosing to seek help through guided ICBT [], which could enhance their motivation to engage with treatment more effectively compared to GP-referred patients []. The hope of recovery and the desire to gain control over one’s life, identified as internal motivators in a study by Wilhelmsen et al [], may be particularly strong among self-referred patients, who make the decision to undergo therapy independently. This increased autonomy may enhance their motivation to engage with treatment []. In turn, higher motivation may lead to more effective engagement with ICBT tasks, which could boost self-referees’ sense of competence [].
Other potential factors that may influence differences in treatment outcomes include both GPs’ beliefs about treatment and patients’ own beliefs and expectations. The assumption that “gold standard” psychotherapeutic treatment requires long-term, weekly face-to-face sessions with a therapist to be effective may shape how both GPs and patients perceive ICBT []. Patients often rely on their GPs for guidance in navigating the health care system, including for treatment recommendations []. A recent systematic review found that GPs’ negative attitudes toward ICBT can transfer to patients and potentially undermine treatment outcomes []. GP-referred patients who expected or preferred face-to-face therapy may have engaged less with ICBT, influencing their outcomes in this study. In contrast, self-referred participants actively chose ICBT, suggesting they may have had greater knowledge, motivation, and readiness for this treatment. Accordingly, patients’ perceptions of ICBT’s usefulness—and potential lack of awareness of its effectiveness—may impact patients’ engagement and treatment outcomes [].
Guided ICBT relies on the patient’s ability to actively engage with the treatment and implement changes into their everyday life. Since sustained behavior change is more effective when driven by autonomous motivation [], fostering this motivation may enhance treatment engagement. Enhancing patients’ sense of competence and connection may may further improve their engagement with homework, compliance with exposure exercises, and relapse prevention. These factors are crucial to therapy effectiveness, but have also been identified as major challenges in ICBT []. However, because technology is central to ICBT, factors like low computer self-efficacy, lack of basic computer skills, or computer anxiety may undermine perceived competence and hinder engagement with treatment [], although previous studies have found that patients with lower levels of computer anxiety tend to show greater interest in ICBT []. Although these factors were not explicitly analyzed in our study, they may help contextualize the significant differences in treatment outcomes we found between the referral pathways.
Another factor potentially related to increased effectiveness in self-referred patients is educational level, as people with higher education report greater acceptance of digital mental health interventions than those with lower education []. Thus, the difference in education levels between referral pathways may contribute to differences in treatment outcomes. More self-referred patients had higher education, such as university degrees, compared to those referred by GPs. This is a pattern commonly observed across ICBT clinics and studies [,,,]. A key challenge in this context has been demonstrating that ICBT can effectively support individuals with fewer resources and lower education levels [].
As education may develop capacities on many levels including increased sense of personal control, mastery, and self-direction [], education may enhance engagement with ICBT by promoting autonomy and self-determination, as outlined in SDT [,]. Additionally, the treatment format in our study relies on text-heavy modules that guide patients through psychoeducation, working with negative or automatic thoughts and exposure therapy, a format known as bibliotherapy []. Patients with higher education levels may find it easier to process and apply these materials effectively, potentially contributing to their greater improvements.
Beyond comprehension, education is also linked to digital literacy, health literacy, and problem-solving skills, which may further support active engagement with the treatment format [,]. However, a recent systematic review found inconsistent evidence regarding the relationship between higher education and treatment outcomes in guided internet-delivered therapy, including ICBT []. This suggests that, while education may facilitate certain aspects of engagement, other factors such as motivation and readiness for digital interventions may play an equally important role.
Exploring the difference in treatment effectiveness between depression, PD, and SAD revealed large effect sizes of guided ICBT from pretreatment to posttreatment and from pre-treatment to the 6-month follow-up, regardless of referral pathway. These findings align with the large effects reported in systematic reviews for depression [,], PD [,], and SAD []. Our results indicate statistically significant differences in effect sizes between referral pathways across the 3 diagnoses.
In our study, both referral pathways led to significant symptom reduction, consistent with findings from research comparing GP referral and self-referral in psychological care [] and musculoskeletal care []. Considering self-referral in a broader context, the United Kingdom’s IAPT program was introduced in 2007 within primary care to expand access to treatment for common mental health problems. Self-referral within IAPT has improved access to care, particularly for individuals who might not seek help through traditional GP referrals []. However, concerns have been raised about equity, as factors such as education, digital literacy, and ethnicity influence who engages with the program []. Recent research suggests that digital solutions, such as artificial intelligence–driven self-referral chatbots, may help reduce these disparities by increasing referrals from underrepresented groups, including ethnic minorities and nonbinary individuals []. Building on the IAPT, the Norwegian Prompt Access to Mental Health Care program provides adults with quick, low-threshold access to evidence-based treatment for anxiety and depression through self-referral. Situated within primary care, the program offers both short-term face-to-face therapy and guided internet-delivered treatment []. A randomized controlled trial comparing these modalities found that the digital option improves accessibility, requires significantly less therapist time, and has the potential to reach a larger population []. Sweden permits self-referral directly to specialized mental health care. A study of young Swedish adults who self-referred to a specialized mental health clinic found no evidence of overutilization of specialized services. Additionally, most self-referrers had not previously sought professional help for their psychiatric symptoms. This suggests that self-referral lowers the threshold for accessing specialized care especially for those who for various reasons do not contact their GPs for mental health problems [].
Limitations
First, the attrition was high, particularly at the 6-month follow-up, potentially introducing bias into the results, as indicated by the results from the missing data sensitivity analyses. This may, in part, reflect the limitations imposed by conducting the study within routine care, where ethical approval from the Regional Ethical Committee restricted patient contact to what was considered natural for the treatment process, and offering incentives for survey completion was not permitted. Future studies conducted outside these constraints could explore strategies such as using multiple contact methods or offering incentives to improve follow-up rates. Second, since we did not assess patients’ motivation for treatment, we can only speculate whether underlying patient characteristics such as those outlined by SDT [] contributed to variations in effectiveness across referral pathways. Third, as we were only able to assess primary outcome measures for the diagnoses and did not assess secondary outcomes such as comorbidity disorders or quality of life, we cannot document a broader impact of the results. Fourth, although all data in this study were collected through the same secure web platform, technological advancements since data collection began in 2014 have surpassed the platform’s capabilities. Although extracting user data from the platform’s backend to examine engagement patterns and completion speeds would be valuable, this is not supported by the platform.
Implications
Overall, our results showed that both referral pathways led to significant symptom reduction in guided ICBT. However, for patients with SAD, treatment effectiveness did not differ between referral pathways. These results underscore the importance of maintaining both referral pathways, as they attract distinct patient populations and serve complementary roles in facilitating access to mental health care. Although self-referral may be a more effective route for some, GPs remain crucial as gatekeepers, gate-openers, and trusted guides in navigating the health care system for others. Therefore, raising awareness among GPs and other health care providers about the viability and effectiveness of guided ICBT is essential to ensuring more patients benefit from this evidence-based treatment.
Delayed help-seeking remains a major challenge in mental health care [,]; our finding that the patients had been experiencing symptoms for an average of 10 years underscores the urgent need to increase help-seeking behavior. Improving mental health literacy could help reduce stigma and encourage help-seeking behavior [], while public information campaigns may further raise awareness of the benefits of guided ICBT and self-referral. Additionally, our results suggest that motivation plays a key role in treatment success. Addressing potential barriers such as lower motivation in GP-referred patients through targeted engagement strategies could help optimize outcomes.
Beyond increasing accessibility, self-referral offers an essential pathway for individuals who may not seek help through traditional means [,]. Technology-driven solutions, such as artificial intelligence–powered self-referral chatbots, could further enhance access by reducing stigma, guiding users to appropriate services, and providing personalized interactions that support motivation and engagement []. Notably, chatbots have been shown to reduce negative attitudes and stigma toward mental health, potentially improving help-seeking behaviors []. Taken together, these findings highlight the need for a multifaceted approach that ensures timely and equitable access to mental health care while also addressing patient engagement and systemwide awareness.
Future Directions
Future research could focus on conducting cost-effectiveness analyses of different referral pathways to provide policymakers and health care providers with a systematic comparison for prioritizing resource allocation. Additionally, exploring qualitative aspects of patient experiences with referral pathways may offer a more comprehensive understanding of the factors influencing treatment outcomes. Together, these approaches could guide the development of more effective and accessible ICBT services. Further, investigating the influence of potential mediators and moderators could clarify the relationship between referral pathways and treatment outcomes. Adjusting for factors like baseline characteristics may provide valuable insights into the mechanisms underlying guided ICBT effectiveness.
Conclusion
This study demonstrates that both GP referral and self-referral pathways to guided ICBT are effective for moderate depression, PD, and SAD when delivered in a specialized routine care clinic. Notably, self-referred patients experienced significantly greater treatment outcomes both from pretreatment to posttreatment and from pretreatment to 6-month follow-up compared to those referred by a GP. We suggest that this additional improvement may stem from differences in internal motivation, with self-referred patients being more motivated to engage with the treatment, leading to greater long-term benefits. Our results highlight the vital role of self-referral and patient autonomy in driving sustained progress, particularly for depression and PD. At the same time, the comparable outcomes for SAD suggest that referral pathways may be less influential for this condition. These findings underscore the need for a health care system that supports both referral pathways, ensuring timely and equitable access to evidence-based psychological treatment while also fostering patient engagement and long-term recovery. Further research is needed to examine the effect of self-referral to guided ICBT when the pathway is implemented on a larger scale.
Acknowledgments
The authors would like to express their gratitude to all patients in the study. A special thank you goes to eCoping’s clinical coordinator, Hanne Halseth Lund Gulbrandsen, and clinical psychologist, Reidar Nævdal, for their valuable insights and contributions to this paper. This study was part of the Center for research-based innovation on Mobile Mental Health (ForHelse), which is funded by the Norwegian Research Council (NFR 309264).
Conflicts of Interest
None declared.
Multimedia Appendix 1
Estimated outcome measures over time for general practitioner–referred and self-referred groups.
DOCX File, 34 KBMultimedia Appendix 2
Observed outcome measures preintervention, postintervention, and at 6-month follow-up.
DOCX File, 30 KBReferences
- GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Psychiatry. Feb 2022;9(2):137-150. [CrossRef] [Medline]
- WHO. Transforming Mental Health for All. World Health Organization; 2022. ISBN: 9240049339
- Chisholm D, Sweeny K, Sheehan P, et al. Scaling-up treatment of depression and anxiety: a global return on investment analysis. Lancet Psychiatry. May 2016;3(5):415-424. [CrossRef] [Medline]
- Wilhelm S, Weingarden H, Ladis I, Braddick V, Shin J, Jacobson NC. Cognitive-behavioral therapy in the digital age: presidential address. Behav Ther. Jan 2020;51(1):1-14. [CrossRef] [Medline]
- Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ. Nov 2004;82(11):858-866. [Medline]
- Kazdin AE. Addressing the treatment gap: a key challenge for extending evidence-based psychosocial interventions. Behav Res Ther. Jan 2017;88:7-18. [CrossRef] [Medline]
- Cuijpers P, Miguel C, Harrer M, et al. Cognitive behavior therapy vs. control conditions, other psychotherapies, pharmacotherapies and combined treatment for depression: a comprehensive meta-analysis including 409 trials with 52,702 patients. World Psychiatry. Feb 2023;22(1):105-115. [CrossRef] [Medline]
- Cuijpers P, van Straten A, van Oppen P, Andersson G. Are psychological and pharmacologic interventions equally effective in the treatment of adult depressive disorders? A meta-analysis of comparative studies. J Clin Psychiatry. Nov 2008;69(11):1675-1685. [CrossRef] [Medline]
- Churchill R, Moore THM, Furukawa TA, et al. 'Third wave' cognitive and behavioural therapies versus treatment as usual for depression. Cochrane Database Syst Rev. Oct 18, 2013;(10):CD008705. [CrossRef] [Medline]
- McHugh RK, Whitton SW, Peckham AD, Welge JA, Otto MW. Patient preference for psychological vs pharmacologic treatment of psychiatric disorders: a meta-analytic review. J Clin Psychiatry. Jun 2013;74(6):595-602. [CrossRef] [Medline]
- Hoffmann D, Rask CU, Hedman-Lagerlöf E, Eilenberg T, Frostholm L. Accuracy of self-referral in health anxiety: comparison of patients self-referring to internet-delivered treatment versus patients clinician-referred to face-to-face treatment. BJPsych Open. Sep 9, 2019;5(5):e80. [CrossRef] [Medline]
- Depression in adults: recognition and management. National Institute for Health and Care Excellence. 2009. URL: https://www.nice.org.uk/guidance/cg90 [Accessed 2024-07-23]
- Generalised anxiety disorder and panic disorder in adults: management. National Institute for Health and Care Excellence. 2011. URL: https://www.nice.org.uk/guidance/cg113 [Accessed 2025-03-16]
- Andersson G. The latest developments with internet-based psychological treatments for depression. Expert Rev Neurother. Feb 2024;24(2):171-176. [CrossRef] [Medline]
- Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. Feb 2014;13(1):4-11. [CrossRef] [Medline]
- Newby J, Mason E, Kladnistki N, et al. Integrating internet CBT into clinical practice: a practical guide for clinicians. Clin Psychol (Aust Psychol Soc). May 4, 2021;25(2):164-178. [CrossRef]
- Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. Oct 2014;13(3):288-295. [CrossRef] [Medline]
- Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. Jan 2018;47(1):1-18. [CrossRef] [Medline]
- Hedman-Lagerlöf E, Carlbring P, Svärdman F, Riper H, Cuijpers P, Andersson G. Therapist-supported Internet-based cognitive behaviour therapy yields similar effects as face-to-face therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. World Psychiatry. Jun 2023;22(2):305-314. [CrossRef] [Medline]
- Hedman E, Ljótsson B, Kaldo V, et al. Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord. Feb 2014;155:49-58. [CrossRef] [Medline]
- Andersson G, Hedman E. Effectiveness of guided internet-based cognitive behavior therapy in regular clinical settings. Verhaltenstherapie. 2013;23(3):140-148. [CrossRef]
- Nordgreen T, Gjestad R, Andersson G, Carlbring P, Havik OE. The effectiveness of guided internet-based cognitive behavioral therapy for social anxiety disorder in a routine care setting. Internet Interv. Sep 2018;13:24-29. [CrossRef] [Medline]
- Titov N, Dear B, Nielssen O, et al. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv. Sep 2018;13:108-115. [CrossRef] [Medline]
- Andersson G. Innovating CBT and answering new questions: the role of internet-delivered CBT. J Cogn Ther. 2024;17(2):179-190. [CrossRef]
- Andersson G, Rozental A, Shafran R, Carlbring P. Long-term effects of internet-supported cognitive behaviour therapy. Expert Rev Neurother. Jan 2018;18(1):21-28. [CrossRef] [Medline]
- Folker AP, Mathiasen K, Lauridsen SM, Stenderup E, Dozeman E, Folker MP. Implementing internet-delivered cognitive behavior therapy for common mental health disorders: a comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv. Mar 2018;11:60-70. [CrossRef] [Medline]
- Brown JSL, Boardman J, Whittinger N, Ashworth M. Can a self-referral system help improve access to psychological treatments? Br J Gen Pract. May 2010;60(574):365-371. [CrossRef] [Medline]
- Haug T, Nordgreen T, Öst LG, Havik OE. Self-help treatment of anxiety disorders: a meta-analysis and meta-regression of effects and potential moderators. Clin Psychol Rev. Jul 2012;32(5):425-445. [CrossRef] [Medline]
- Ross CE, Wu CL. The links between education and health. Am Sociol Rev. Oct 1995;60(5):719. [CrossRef]
- Zajacova A, Lawrence EM. The relationship between education and health: reducing disparities through a contextual approach. Annu Rev Public Health. Apr 1, 2018;39(1):273-289. [CrossRef] [Medline]
- Priorities and operational planning guidance. NHS England. 2022. URL: https://www.england.nhs.uk/publication/2023-24-priorities-and-operational-planning-guidance [Accessed 2025-03-16]
- Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. Jan 2000;55(1):68-78. [CrossRef] [Medline]
- Deci EL, Ryan RM. The “what” and “why” of goal pursuits: human needs and the self-determination of behavior. Psychol Inq. Oct 2000;11(4):227-268. [CrossRef]
- Mathers N, Mitchell C. Are the gates to be thrown open? Br J Gen Pract. May 2010;60(574):317-318. [CrossRef] [Medline]
- Miller BM, Fritz Z. In this uncertain world, patient-centred care must not mean patient-led care. Br J Gen Pract. May 2019;69(682):259-260. [CrossRef] [Medline]
- Harvey-Sullivan A, Lynch H, Tolley A, Gitlin-Leigh G, Kuhn I, Ford JA. What impact do self-referral and direct access pathways for patients have on health inequalities? Health Policy. Jan 2024;139:104951. [CrossRef] [Medline]
- Greenfield G, Foley K, Majeed A. Rethinking primary care’s gatekeeper role. BMJ. Sep 23, 2016;354:i4803. [CrossRef] [Medline]
- Bury TJ, Stokes EK. A global view of direct access and patient self-referral to physical therapy: implications for the profession. Phys Ther. Apr 2013;93(4):449-459. [CrossRef] [Medline]
- Babatunde OO, Bishop A, Cottrell E, et al. A systematic review and evidence synthesis of non-medical triage, self-referral and direct access services for patients with musculoskeletal pain. PLoS One. 2020;15(7):e0235364. [CrossRef] [Medline]
- Haller K, Becker P, Niemeyer H, Boettcher J. Who benefits from guided internet-based interventions? A systematic review of predictors and moderators of treatment outcome. Internet Interv. Sep 2023;33:100635. [CrossRef] [Medline]
- Lindberg MS, Brattmyr M, Lundqvist J, et al. Is the Norwegian stepped care model for allocation of patients with mental health problems working as intended? A cross-sectional study. Psychother Res. Jul 22, 2024;2024:1-13. [CrossRef] [Medline]
- Jonker L, Thwaites R, Fisher SJ. Patient referral from primary care to psychological therapy services: a cohort study. Fam Pract. 2019;37(3):395-400. [CrossRef]
- Staples LG, Webb N, Asrianti L, et al. A comparison of self-referral and referral via primary care providers, through two similar digital mental health services in Western Australia. Int J Environ Res Public Health. Jan 14, 2022;19(2):905. [CrossRef] [Medline]
- 64th WMA General Assembly Fortaleza Brazil, October 2013. World Medical Association. 2018. URL: https://www.wma.net/policies-post/wma-declaration-of-helsinki [Accessed 2024-08-17]
- eBehandling. Helse Bergen HU. 2024. URL: https://www.helse-bergen.no/emeistring [Accessed 2024-08-14]
- Prioriteringsveileder - psykisk helsevern for voksne, Oslo. Helsedirektoratet. URL: https://www.helsedirektoratet.no/veiledere/prioriteringsveiledere/psykisk-helsevern-for-voksne [Accessed 2024-08-15]
- Sheehan DV, Lecrubier Y, Sheehan KH, et al. The MINI-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 Suppl 20(20):22-33. URL: https://psycnet.apa.org/record/1998-03251-004 [Medline]
- Nordgreen T, Blom K, Andersson G, Carlbring P, Havik OE. Effectiveness of guided Internet-delivered treatment for major depression in routine mental healthcare - an open study. Internet Interv. Dec 2019;18:100274. [CrossRef] [Medline]
- Nordgreen T, Gjestad R, Andersson G, Carlbring P, Havik OE. The implementation of guided Internet-based cognitive behaviour therapy for panic disorder in a routine-care setting: effectiveness and implementation efforts. Cogn Behav Ther. Jan 2018;47(1):62-75. [CrossRef] [Medline]
- Svanborg P, Asberg M. A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatr Scand. Jan 1994;89(1):21-28. [CrossRef] [Medline]
- Fantino B, Moore N. The self-reported Montgomery-Asberg Depression Rating Scale is a useful evaluative tool in Major Depressive Disorder. BMC Psychiatry. May 27, 2009;9:26. [CrossRef] [Medline]
- Chambless DL, Caputo GC, Bright P, Gallagher R. Assessment of fear of fear in agoraphobics: the body sensations questionnaire and the agoraphobic cognitions questionnaire. J Consult Clin Psychol. Dec 1984;52(6):1090-1097. [CrossRef] [Medline]
- Mattick RP, Clarke JC. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther. Apr 1998;36(4):455-470. [CrossRef] [Medline]
- Brown EJ, Turovsky J, Heimberg RG, Juster HR, Brown TA, Barlow DH. Validation of the Social Interaction Anxiety Scale and the Social Phobia Scale across the anxiety disorders. Psychol Assess. 1997;9(1):21-27. [CrossRef]
- Zantvoort K, Hentati Isacsson N, Funk B, Kaldo V. Dataset size versus homogeneity: a machine learning study on pooling intervention data in e-mental health dropout predictions. Digit Health. 2024;10:20552076241248920. [CrossRef] [Medline]
- IBM SPSS statistics for windows, version 29.0. IBM Corp. 2023. URL: https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-29020 [Accessed 2025-03-16]
- Cohen J. Statistical Power Analysis for the Behavioral Sciences: Routledge. 2013. ISBN: 0203771583
- Heck RH, Thomas SL, Tabata LN. Multilevel and Longitudinal Modeling with IBM SPSS. 3rd ed. New York: Routledge: Taylor Francis Ltd; 2022. ISBN: 97803674244619
- Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. Feb 1991;59(1):12-19. [CrossRef] [Medline]
- Enders CK. Applied Missing Data Analysis. th ed. Guilford Publications; 2022. ISBN: 9781462549863
- Wang J, Wang X. Structural Equation Modeling: Applications Using Mplus. th ed. John Wiley & Sons; 2019. [CrossRef] ISBN: 9781119422109
- Muthen LK, Muthen BO. Mplus. 2023. URL: https://www.statmodel.com/index2.shtml [Accessed 2025-03-16]
- Etzelmueller A, Vis C, Karyotaki E, et al. Effects of internet-based cognitive behavioral therapy in routine care for adults in treatment for depression and anxiety: systematic review and meta-analysis. J Med Internet Res. Aug 31, 2020;22(8):e18100. [CrossRef] [Medline]
- Wilhelmsen M, Lillevoll K, Risør MB, et al. Motivation to persist with internet-based cognitive behavioural treatment using blended care: a qualitative study. BMC Psychiatry. Nov 7, 2013;13(1):296. [CrossRef] [Medline]
- Bobele M, Slive A, Hair HJ. Reimagining the 'gold standard'. In: Cornish P, Berry G, editors. Stepped Care 20: The Power of Conundrums. Cham: Springer International Publishing; 2023:209-227. [CrossRef]
- Ridd MJ, Thompson MJ. Would primary care paediatricians improve UK child health outcomes? No. Br J Gen Pract. Apr 2020;70(693):196-197. [CrossRef] [Medline]
- Duffy D, Richards D, Hisler G, Timulak L. Implementing internet-delivered cognitive behavioral therapy for depression and anxiety in adults: systematic review. J Med Internet Res. Jan 28, 2025;27:e47927. [CrossRef] [Medline]
- Borghouts J, Eikey E, Mark G, et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res. Mar 24, 2021;23(3):e24387. [CrossRef] [Medline]
- Ng JYY, Ntoumanis N, Thøgersen-Ntoumani C, et al. Self-determination theory applied to health contexts: a meta-analysis. Perspect Psychol Sci. Jul 2012;7(4):325-340. [CrossRef] [Medline]
- Neben T, Seeger AM, Kramer T, White A. Regaining joy of life: theory-driven development of mobile psychotherapy support systems. ICIS 2015 Proceedings. 2015. URL: http://aisel.aisnet.org/icis2015/proceedings/IShealth/17/ [Accessed 2025-03-16]
- Schønning A, Nordgreen T. Predicting treatment outcomes in guided internet-delivered therapy for anxiety disorders-the role of treatment self-efficacy. Front Psychol. 2021;12:712421. [CrossRef] [Medline]
- Moskalenko MY, Hadjistavropoulos HD, Katapally TR. Barriers to patient interest in internet-based cognitive behavioral therapy: informing e-health policies through quantitative analysis. Health Policy Technol. Jun 2020;9(2):139-145. [CrossRef]
- Hedman E, Ljótsson B, Rück C, et al. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr Scand. Dec 2013;128(6):457-467. [CrossRef] [Medline]
- El Alaoui S, Hedman E, Kaldo V, et al. Effectiveness of Internet-based cognitive-behavior therapy for social anxiety disorder in clinical psychiatry. J Consult Clin Psychol. Oct 2015;83(5):902-914. [CrossRef] [Medline]
- Mishra S. Social networks, social capital, social support and academic success in higher education: a systematic review with a special focus on ‘underrepresented’ students. Educational Research Review. Feb 2020;29:100307. [CrossRef]
- Hedman E, Axelsson E, Andersson E, Lekander M, Ljótsson B. Exposure-based cognitive-behavioural therapy via the internet and as bibliotherapy for somatic symptom disorder and illness anxiety disorder: randomised controlled trial. Br J Psychiatry. Nov 2016;209(5):407-413. [CrossRef] [Medline]
- Mamukashvili-Delau M, Koburger N, Dietrich S, Rummel-Kluge C. Long-term efficacy of internet-based cognitive behavioral therapy self-help programs for adults with depression: systematic review and meta-analysis of randomized controlled trials. JMIR Ment Health. Aug 22, 2023;10:e46925. [CrossRef] [Medline]
- Bisby MA, Karin E, Hathway T, et al. A meta-analytic review of randomized clinical trials of online treatments for anxiety: inclusion/exclusion criteria, uptake, adherence, dropout, and clinical outcomes. J Anxiety Disord. Dec 2022;92:102638. [CrossRef] [Medline]
- Stech EP, Lim J, Upton EL, Newby JM. Internet-delivered cognitive behavioral therapy for panic disorder with or without agoraphobia: a systematic review and meta-analysis. Cogn Behav Ther. Jul 2020;49(4):270-293. [CrossRef] [Medline]
- Guo S, Deng W, Wang H, et al. The efficacy of internet‐based cognitive behavioural therapy for social anxiety disorder: a systematic review and meta‐analysis. Clin Psychology and Psychoth. May 2021;28(3):656-668. [CrossRef]
- Beck A, Naz S, Brooks M, Black JM. Asian and minority ethnic service user positive practice guide. 2019. URL: https://babcp.com/Therapists/BAME-Positive-Practice-Guide-PDF [Accessed 2025-03-16]
- Habicht J, Viswanathan S, Carrington B, Hauser TU, Harper R, Rollwage M. Closing the accessibility gap to mental health treatment with a personalized self-referral chatbot. Nat Med. Feb 2024;30(2):595-602. [CrossRef] [Medline]
- Knapstad M, Nordgreen T, Smith ORF. Prompt mental health care, the Norwegian version of IAPT: clinical outcomes and predictors of change in a multicenter cohort study. BMC Psychiatry. Aug 16, 2018;18(1):260. [CrossRef] [Medline]
- Knapstad M, Lervik LV, Sæther SMM, Aarø LE, Smith ORF. Effectiveness of prompt mental health care, the Norwegian version of improving access to psychological therapies: a randomized controlled trial. Psychother Psychosom. 2020;89(2):90-105. [CrossRef] [Medline]
- Ramirez A, Ekselius L, Ramklint M. Mental disorders among young adults self-referred and referred by professionals to specialty mental health care. Psychiatr Serv. Dec 2009;60(12):1649-1655. [CrossRef] [Medline]
- Soria-Martínez M, Navarro-Pérez CF, Pérez-Ardanaz B, Martí-García C. Conceptual framework of mental health literacy: results from a scoping review and a Delphi survey. Int J Ment Health Nurs. Apr 2024;33(2):281-296. [CrossRef] [Medline]
- Lee YC, Cui Y, Jamieson J, Fu W, Yamashita N. Exploring effects of chatbot-based social contact on reducing mental illness stigma. In: Yamashita N, editor. Presented at: CHI ’23; Apr 23-28, 2023; Hamburg Germany. URL: https://dl.acm.org/doi/proceedings/10.1145/3544548 [Accessed 2026-03-16] [CrossRef]
Abbreviations
| BSQ: Body Sensation Questionnaire |
| CBT: cognitive behavioral therapy |
| FIML: full-information maximum likelihood |
| GP: general practitioner |
| IAPT: Improving Access to Psychological Therapies |
| ICBT: internet-based cognitive behavioral therapy |
| LMM: linear mixed modeling |
| MADRS-S: Montgomery Åsberg Depression Rating Scale, Self-rating version |
| MI: multiple imputation |
| MINI: Mini-International Neuropsychiatric Interview |
| PD: panic disorder |
| RCI: reliable change index |
| SAD: social anxiety disorder |
| SDT: Self-Determination Theory |
| SPS: Social Phobia Scale |
Edited by John Torous; submitted 01.11.24; peer-reviewed by Karen Rowa, Nur Hani Zainal; final revised version received 25.02.25; accepted 26.02.25; published 25.03.25.
Copyright© Jill Bjarke, Rolf Gjestad, Tine Nordgreen. Originally published in JMIR Mental Health (https://mental.jmir.org), 25.3.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.