This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Despite the increasing amount of research on Web-based mental health interventions with proven efficacy, high attrition rates decrease their effectiveness. Continued process evaluations should be performed to maximize the target population’s engagement. Google Analytics has been used to evaluate various health-related Web-based programs and may also be useful for Web-based mental health programs.
The objective of our study was to evaluate WalkAlong.ca, a youth-oriented mental health web-portal, using Google Analytics to inform the improvement strategy for the platform and to demonstrate the use of Google Analytics as a tool for process evaluation of Web-based mental health interventions.
Google Analytics was used to monitor user activity during WalkAlong’s first year of operation (Nov 13, 2013-Nov 13, 2014). Selected Google Analytic variables were overall website engagement including pages visited per session, utilization rate of specific features, and user access mode and location.
The results included data from 3076 users viewing 29,299 pages. Users spent less average time on Mindsteps (0 minute 35 seconds) and self-exercises (1 minute 08 seconds), which are important self-help tools, compared with that on the Screener tool (3 minutes 4 seconds). Of all visitors, 82.3% (4378/5318) were desktop users, followed by 12.7 % (677/5318) mobile phone and 5.0% (263/5318) tablet users. Both direct traffic (access via URL) and referrals by email had more than 7 pages viewed per session and longer than average time of 6 minutes per session. The majority of users (67%) accessed the platform from Canada.
Engagement and feature utilization rates are higher among people who receive personal invitations to visit the site. Low utilization rates with specific features offer a starting place for further exploration of users in order to identify the root cause. The data provided by Google Analytics, although informative, can be supplemented by other evaluation methods (ie, qualitative methods) in order to better determine the modifications required to improve user engagement. Google Analytics can play a vital role in highlighting the preferences of those using Web-based mental health tools.
As technologies such as internet, mobile phones, and computers have become ubiquitous, Web-based interventions have become one of the major treatment and preventative tools for mental disorders. More than 100 randomized controlled trials have been published to demonstrate the efficacy of internet interventions for psychiatric disorders [
Despite their promising benefits, online mental health interventions face problems in engagement [
Engagement is especially challenging in the context of mental health. Dropout from traditional treatment among those with mental illness is already a cause for concern [
A key step to improve engagement is identifying ways to improve intervention uptake in real-world settings through process evaluation [
In this context, one tool that can be used is Google Analytics, which is an open tool that provides free quantitative data on website usage that can be leveraged for continual website improvements (
Google Analytics can be used as a tool for process evaluation by receiving information on user traffic and subsequently informing website improvement. This process can also continue as a cycle for continual improvement of the website.
WalkAlong home page.
The goal of this study was to evaluate WalkAlong, a Web-based mental health platform, using Google Analytics. This platform is a youth-oriented mental health Web-portal designed to provide young people with tools and resources required to manage their own mental health (
Our objective was to use Google Analytics as a tool for conducting a process evaluation of the WalkAlong platform. As part of the process evaluation, the following evaluation questions using Google Analytics were asked: (1) How engaged (ie, time spent on the website) are users with the WalkAlong platform? (2) How can the WalkAlong platform be improved to better engage the users? (3) How can the marketing strategy be shaped to engage and reach out to more users? Another objective of this project is to extend the work to a mental health platform from other health interventions and inform website design and marketing strategies to effectively impact user behavior [
Google Analytics was used to access user data over the first year of WalkAlong (Nov 13, 2013-Nov 13, 2014). Focusing on the first year of operation was considered the most appropriate approach in order to capture a snapshot of web traffic following the initial launch. The Google Analytics data do not contain any personally identifiable information and are presented in the form of aggregate data, making it an accessible tool used in research settings without ethical concerns [
The research team installed Google Analytics by adding a tracking tag for WalkAlong [
Several indicators from Google Analytics that would allow inference of a level of engagement were calculated. Such indicators include the number of returning users (n), bounce rate (%), number of pages accessed per session (n), mean session duration (minutes, seconds), and goal conversion rate (%).
The number of returning users refers to the number of sessions visited through the same client id. A high number of returning users has been used as an indicator for a strong level of engagement with the platform [
The bounce rate is the percentage of only a single page visit during a session. A high bounce rate could indicate minimal exposure to the intervention due to minimal interaction, but it could also indicate users exiting as they have found what they were looking for right away. However, generally, a low bounce rate can be considered indicative of a high overall engagement, especially for a multicomponent platform like WalkAlong [
The number of pages per session refers to the number of webpages within the platform that the user viewed in a single session, and the mean session duration (minutes, seconds) refers to the mean duration of time the users spent on the platform. There are limitations to ascertaining engagement through these indicators since they allow for multiple interpretations: a high number of pages per session could result from an increased engagement, but it could also result from a superficial exploration of several pages; similarly, a long session duration can result from increased engagement, but it could also result from a user keeping the webpage open while engaging in other irrelevant activities. Nevertheless, despite these caveats, traffic information provides an approximation of the level of exposure the users had with the platform [
The goal conversion rate measures the proportion of sessions that achieved a goal out of the total sessions. The goal was predefined as creating an account but can be defined as any activity the web developer or owner chooses (eg, buying a product). As discussed above, users who create an account are able to access more resources than those who do not (anonymous users). Thus, creating an account was assumed to indicate a stronger level of engagement. Overall, a high number of returning users, low bounce rate, high number of pages viewed per session, high mean session duration, and high goal conversion rates collectively translate to an estimate of a strong level of engagement [
Several indicators from Google Analytics that can inform the improvement of the platform were also selected. Indicators of user behavior such as page views, mean duration of visit, and bounce rate when accessing self-help tools (eg, Mindsteps page, Self-Help Exercises page, and Screener) were analyzed. In addition, the most visited pages were observed in terms of their overall entrance rate, exit rate, and bounce rate to understand which tools or pages were most used or viewed. The entrance rate represents a proportion of sessions starting from a given page, while the exit rate represents a proportion of sessions ending from a given page. The information regarding the entrance rate may provide an understanding about which webpage is serving as the first impression for the users, and the exit rate may indicate the point when users felt disengaged or, on the contrary, had adequate information needed for the session.
Google Analytics also provided data on the type of devices used for access. Such information can allow us to consider whether developing a mobile app for WalkAlong would be helpful or not. The three main devices of interest to the current investigation were desktops, tablets, and mobile phones (counted here as mobile devices).
Google Analytics was also used to inform our marketing strategy, with the goal of reaching as many users as possible. At the outset, the research team had reached out to different youth and university organizations, especially around Vancouver. Twitter and a Facebook accounts were also created to spread awareness about the platform. To improve the marketing strategy, the channels used to access the platform were observed. The channels are direct link (ie, typing the web URL directly into a browser); organic search (ie, entry through a search engine); and referrals via another website, via social media, and via email. Understanding which channels are underutilized and which channel results in the highest level of engagement can help improve the marketing strategy. Locations of users from different countries around the world were also observed.
The first year of operation for the WalkAlong platform saw a total of 3076 users, amounting to 5318 sessions and 29,299 page views (
In terms of the frequency of visits, 80% (4259/5318) of sessions came from users visiting less than nine times, indicating a level of disengagement after a certain number of visits (
The number of sessions during the study period decreased with increasing number of visits. However, there was a slight increase at the upper end of sessions from high-frequency visits: 5.8% (311/5318) accounted for 26-50 visits and 4.7% (250/5318) accounted for 51-100 visits over the time period. The number of sessions also decreased with longer session durations (
WalkAlong overview presented in Google Analytics.
Proportion of total sessions and number of visits.
Visits | Sessions (N=5318), n (%) |
1 | 3066 (57.65) |
2 | 550 (10.3) |
3 | 241 (4.5) |
4 | 139 (2.6) |
5 | 100 (1.9) |
6 | 67 (1.3) |
7 | 50 (0.9) |
8 | 46 (0.9) |
9-14 | 180 (3.4) |
15-25 | 205 (3.9) |
26-50 | 311 (5.8) |
51-100 | 250 (4.7) |
101-200 | 112 (2.1) |
201+ | 1 (0.0) |
Duration of session.
Session duration (in minutes) | Sessions (N=5318), n (%) |
≤1 | 3477 (65.4%) |
1-3 | 527 (9.9%) |
3-10 | 581 (10.9%) |
>10 | 733 (13.8%) |
Visits to the Mindsteps tool comprised 11.9% (3493/29,299) of total page views, with mean duration spent of 35 seconds. Visits to the Self-Help Exercises page comprised 6.13% (1797/29,299), with mean duration spent of 1 minute 8 seconds. Visits to the Screener comprised only 3.36% (983/29,299) of the total page views, but the mean duration spent on the Screener was 3 minutes 4 seconds.
A list of devices used by WalkAlong’s users to access the site is presented in
Direct traffic accounted for the highest proportion (2420/5318, 45.5%) of all visits to the site (
Approximately two-thirds or 67.6% (2079/3076) of the users belonged to Canada. Users from Canada also had a relatively low bounce rate (34.35%), high number of pages viewed per session (6.57 pages per session), and long session duration (6:10). However, the users accessed the platform from around the world (
Entrance and exit rates for the most viewed pages.
Page | Entrances n (%)a | Exits (%)b | Bounce rate (%) |
Home page | 3487 (65.6) | 29.2 | 37.6 |
Depression in Canada | 115 (2.2) | 73.4 | 86.1 |
Self-Help Exercises | 70 (1.3) | 14 | 47.9 |
Mindsteps | 62 (1.2) | 5.4 | 24.2 |
Screener | 55 (1.0) | 32.0 | 50.9 |
aThe numbers do not add up to 100% because only several of the most viewed pages are included in the table.
bThe exit rate is calculated by the number of exits/number of times that page was viewed. Thus, the added percentages are higher than 100% where each row has different number of exits and the number of pages viewed.
Devices used to access WalkAlong.
Device | Sessions (N=5318), n (%) | Bounce rate (%) | Pages per session, n | Mean session duration | Conversion rate (%) |
Desktop | 4378 (82.32) | 39.6 | 6.17 | 5 min 43 s | 22.7 |
Mobile phone | 677 (12.7) | 61.6 | 2.15 | 1 min 53 s | 7.2 |
Tablet | 263 (5.0) | 48.7 | 3.08 | 3 min 15 s | 11.8 |
Proportion of total sessions for each type of channel.
Channels | Sessions (N=5318), n (%) | Bounce rate (%) | Pages per session, n | Mean session duration | Conversion rate (%) |
Direct Traffic | 2,420 (45.51) | 36.6 | 7.4 | 6 min 38 s | 24.4 |
Organic Search | 1,256 (23.62) | 50.4 | 3.5 | 3 min 42 s | 11.3 |
Referrals | 849 (16.0) | 46.9 | 3.5 | 3 min 46 s | 22.6 |
Social Media | 717 (13.5) | 48.8 | 4.6 | 3 min 44 s | 18.0 |
76 (1) | 17.1 | 10.5 | 7 min 39 s | 25.0 |
Map overlay about locations of users from Google Analytics.
The first year of operation for the WalkAlong platform saw a total of 3076 users, amounting to 5318 sessions of 5 minutes 6 seconds on average, 29,299 page views, and 31.7% goal completion rate. However, the high proportion of sessions comprising first visits and short session durations suggest a degree of disengagement for the users (
The platform can also be improved based on user behavior. For instance, there could be further efforts to improve engagement with tools such as the “Mindsteps” and “Self-Help Exercises.” The mean duration of 35 seconds spent on Mindsteps or 1 minute 8 seconds spent on Self-Help Exercises may be deemed too short considering that they are important components of WalkAlong intended to improve mental health outcomes. The relatively short time spent on “Mindsteps” and “Self-Help Exercises” needs to be addressed using better engagement strategies such as improved web design or involving youth to be part of the design process [
In terms of the devices used to access the WalkAlong platform, the site was viewed mostly via desktops. However, as the usage of mobile phones is ubiquitous among youth, future improvements in WalkAlong may benefit from making the platform more accessible and engaging for mobile phone users [
The data indicate that some form of personal referrals indicated by either email or prior knowledge of the URL (direct traffic) results in a relatively stronger engagement (ie, longer average duration, more pages viewed per session, etc) than less personal channels such as referrals through social media or organic searches. In other words, direct referrals such as word-of-mouth strategies among peers could help increase the number of engaged users [
When observing the location of the users, 67% (2079/3076) users belonged to Canada. This finding aligns with the limited marketing strategy used in Vancouver. WalkAlong can be used by all English-speaking countries, but it can also be used as a template for other platform developments internationally. The WalkAlong team could consider spreading awareness beyond Canada to ensure that such a resource is available to as much youth population as possible.
Although Google Analytics has provided promising data on the usage patterns of the WalkAlong platform, the tool should be used with careful consideration. For example, comparing the results across various interventions is currently difficult as they serve different purposes with different standards in the number of users, sessions, and page views [
Google Analytics also conforms to a marketing perspective of Web-based behavior rather than to a full evaluation of user behavior [
Google Analytics was helpful in informing the process evaluation of an open-access Web-based mental health platform. The process evaluation provided information about marketing strategies as well as the aspects of the platform that required improvement. Ideas for future improvements may include marketing the WalkAlong platform outside Canada to get more users from other countries and making the platform more accessible and engaging for mobile users. The rich aggregate data, when combined with other evaluation methods, may provide more accurate interpretations to reinforce or challenge these ideas. Therefore, future studies should focus on developing a mixed methodology that includes Google Analytics to conduct process evaluation of open-access Web-based mental health platforms. With high-quality process evaluation, Web-based mental health interventions may be not only effective but also engaging.
Screenshots of pages from WalkAlong.
posttraumatic stress disorder
None declared.