The recruitment area for the survey covered 3 major metropolitan cities in mainland China: Beijing, Shanghai, and Shenzhen (a major city in Guangdong Province). According to the 2010 census data, the population aged between 10 and 39 years in the 3 cities is 28,417,551, accounting for 53.6% of their overall population. A cross-sectional open Web survey was conducted from October 2015 to May 2016 to identify and reach our target population, ie, youth in social withdrawal.

The survey was created and stored using a Chinese survey platform called Sojump, which is operated by Changsha Xingran Information Technology Company and is one of the biggest survey platforms in China. The survey host provided a unique link to the survey, and its system helped to block attempts by respondents with the same internet protocol (IP) address or the same electronic device to fill out the survey more than once. After completing the survey, respondents were able to review their answers and were reminded to finish unanswered mandatory items before submission.

Three recruitment methods were experimented sequentially. They included (1) Weibo, (2) WeChat group and microblog, and (3) an online game website.

The respondents’ consent to participate in the study was obtained online. They were required to read and approve an informed consent form before proceeding to the questionnaire. Assent from adolescent participants was collected with the same consent process described above. The information sheet of the study was designed at the reading level of a 13 year old.

To limit response bias, the nature of the study was introduced in a general sense as concerning youth and internet use. The questions in the survey pertain to this general introduction, and no deception is involved throughout the research. At the end of the questionnaire, a message containing contact information and numbers for crisis intervention hotlines of local nongovernmental organizations that offer mental health services in the 3 cities was provided to the respondents to encourage them to seek help if they felt distressed upon completing the questionnaire.

Because sensitive information such as risk behavior was collected in the survey, measures were taken to ensure data security throughout the research. During the data collection process, only the principal investigator had access to the account on the website that hosted the survey. The survey host also followed the cyber security protocol to prevent data leakage. Once data collection was finished, all of the research data were coded and stored on password-protected drives. Ownership of and access to the data were restricted to the research team. The study obtained ethical approval from the Human Research Ethics Committee for Non-Clinical Faculties at the University of Hong Kong (Ref# EA1508009).

Descriptive statistics for the continuous variables were illustrated by means and standard deviations, whereas categorical variables were shown by numbers and percentages. Fisher exact test and the Mann-Whitney-Wilcoxon test were used to compare the categorical and continuous variables, respectively, between the comparison group and the withdrawal group, the asocial group, and the hikikomori group.

The Web-based questionnaire was administered at 3 sites (ie, Weibo, WeChat, and Wandianba). According to the statistics generated by Weibo, the survey was exposed 596,772 times to targeted users and reached 206,139 users. Of these, 517 (click-through rate=0.087%) either clicked on the survey link or forwarded the post to others. However, only 85 completed the survey via Weibo (Figure 1). The posts on WeChat groups yielded 43 responses, and Wandianba attracted 9 responses (Table 1). The survey host recorded a total of 192 visits with independent IPs to the survey, making the completion rate 71.4%. The survey ultimately collected a total of 137 responses within a period of 7 months.

Advertising on Sina Weibo is managed on the internet, and pricing is based on a bidding system. There are 2 ways to bid for the target group among advertisers with the same aim: to pay per 1000 impressions and to pay per click. During the 7-month advertising on Sina Weibo, we tried both methods of advertisement with different bidding prices based on factors such as the response rate and the number of advertisers opting for the same target group in different time periods. On the first day of advertisement, we chose to pay by impressions. A price of CNY ¥10.8 (around US $1.67) per 1000 impressions was set; however, there were no responses. Thus, starting on October 20, 2015, the bidding method was changed to pay by click starting with prices ranging from CNY ¥10 per click to CNY ¥20 per click (the average price was CNY ¥12.5). It was found that the bidding price did not significantly influence the response rate. However, the number of visits to the recruitment page and the responses to the survey decreased remarkably during the seasonal marketing campaign of the e-commerce website (taobao, the largest e-commerce website in China).

Ultimately, the cost of the 7-month campaign on Sina Weibo was CNY ¥3300 (US $511.07), and the overall service charges (including monthly fee, administration fee for the lucky draw, and distribution of cash rewards) from the survey host (Sojump) and cash rewards (CNY ¥1000 in total) were CNY ¥3128 (US $484.44). Thus, the average cost to reach a participant who completed the survey through Sina Weibo was CNY ¥46.92 (US $7.27).

Except for the incentives in the form of cash coupons of random amounts, there was no additional cost for advertisement via WeChat group and Wandianba. As compared with the average cost per participant of a telephone survey targeting the same population, the cost is relatively lower and could be a viable method in future studies.

In total, 137 participants completed the survey. Of the 137 responding participants, 85 were recruited through Weibo (over 7 months at a cost of around US $500), 43 from WeChat (over 20 days at no cost), and 9 from Wandianba website (over 30 days at no cost). Among the 137 participants, 108 (78.8%) did not show evidence of social withdrawal symptoms and therefore were classified as the control, or comparison, group. The remaining 29 (21.2%) participants had symptoms of social withdrawal and were classified as follows: the 13 (9.5%) who indicated that they stayed at home nearly every day for a period of more than 3 months were categorized as the withdrawal group; the 7 (5.1%) participants who reported that they avoided most social contact for a period of more than 3 months were categorized as the asocial group; and the 9 (6.6%) participants who met both the withdrawal and asocial criteria for more than 3 months were categorized as the hikikomori group. Both the withdrawal group and the hikikomori group were more likely than the comparison group to access the Web-based survey through Wandianba but were less likely to assess the survey through WeChat (Table 2). The hikikomori group was more likely than the comparison group to access the Web-based survey through a computer.

The withdrawal group had fewer online friends on Weibo and WeChat than the comparison group, whereas the hikikomori group had fewer online friends on WeChat and QQ (a widely used instant messaging software in China) than the comparison group. The withdrawal group was younger and included more students (62%, 8/13) and unemployed individuals (15%, 2/13) than the comparison group (Table 3). The hikikomori group was more likely to have visited psychiatric hospital or clinics than other participants. The withdrawal group had more online bridging social capital than the comparison group, whereas the hikikomori group had less offline bridging social capital than the comparison group (Table 4). There was no significant difference between the asocial group and the comparison group in terms of the studied variables.

As far as we know, this is the first quantitative study to evaluate hikikomori and social withdrawal behavior among young people in China. Given the high penetration of social media among young people in China, we explored the feasibility of using various social media platforms to recruit young people to participate in our Web survey.

The response from the 3 recruitment methods shows very interesting patterns. Although Weibo helped in recruiting the largest portion of participants, it took 7 months to reach that number, whereas it took only 20 days to recruit 43 participants through personal contacts using WeChat. Although it took a month to recruit 9 participants at Wandianba, 3 of those participants were hikikomori. In other words, online gaming sites merit further exploration as a venue to identify socially withdrawn youth in China.

Despite the small dataset, we identified 13 physically withdrawn youths, 7 asocial youths, and 9 hikikomori, making the percentages among the overall participants 9.5% (13/137), 5.1% (7/137), and 6.6% (9/137), respectively. This finding indicates that we reached a high percentage of individuals at risk of social withdrawal behavior and that we appeared to have targeted the correct group of people through our various social media recruitment methods. Although the data collected from convenient/snowball sampling could be biased, it seems that the use of WeChat and social networking sites that cater to particular user groups are especially effective in reaching over 6% of potential socially withdrawn youth in China, with reference to the prevalence rate in modern society of around 2% [4,13]. This exploratory study also consolidates our experiences with online questionnaire administration in China.

First, because of the tight internet censorship of the Chinese Government, to avoid being shut down, internet content providers will self-censor their content [25]. For example, there were several words in our questionnaire (eg, names of illicit drugs) that were considered sensitive, and the research team was asked by the survey host (Sojump) to change the wording or the display of the words. On the basis of this incident, it is estimated that similar surveys distributed through official avenues such as schools and residential committees are very likely to be sanctioned.

Second, following the auction principle set up by Weibo, the research team acted as a bidder offering a price that represents the maximum willingness to pay for an advertisement. The team had to compete with other commercial advertisers to gain audiences. However, because social media platforms are pervasively used for commercial purposes, the cost of administering the survey has increased. In addition, there are several waves of commercial campaigns in China, such as the big sales on November 11 (known as “Single’s Day”), December 12 (known as “Double Twelve”), and Chinese New Year’s Eve, and responses stagnated during these marketing periods. Therefore, to make the advertisement cost-effective, it is important to find a strategy to buffer the impact of these commercial campaigns on response rate.

Our findings suggest 4 approaches to secure a higher response rate for Web-based surveys in China. First, the use of a prominent Sina microblog account to promote the survey was especially conducive to data collection. In our study, we created a new account to introduce the information of the Web survey, which limited exposure to the targeted population and raised administration costs. In contrast, Guan et al [21] used a celebrity account to promote their survey and easily gathered more than 1000 responses within a shorter period. Therefore, we highly recommend that future research should use a microblog account with a greater fan base.

Second, the length of the survey should be within the participants’ attention spans. Although several measures were used to shorten the survey, such as using a short version of the scale and adopting skip logic to ensure that certain scales were only shown to respondents who met the criteria of the target population, the average time to complete the survey was 15 min. Although the duration is not unbearable to general participants, it is still a challenge to sustain participants’ attention; 192 participants with independent IP addresses entered the survey but only 137 of them completed it, resulting in a completion rate of 72.4%.

Third, the use of scales and the presentation of the survey should be intelligible to the targeted population. Comments from participants left on Wandianba regarding the survey recorded several complaints about the format of the questions. To individuals without much exposure to psychological measurement, many questions in the survey seemed redundant, such as the same set of questions asked regarding different variables such as online social capital and offline social capital. Researchers can also consider using application programming interfaces provided by social media to collect users’ publicly available digital records. For instance, users’ profiles and messages can be extracted to show their psychological [21,26] and behavioral features [27].

Fourth, the snowball sampling method we used on the Wandianba website has helped us to reach 9 potential severely withdrawn young people. This seems to be an effective recruitment method. In fact, this method is similar to a more advanced snowball sampling technique named as respondent-driven sampling. In respondent-driven sampling, a sample is collected using a chain-referral procedure, meaning that respondents are selected not from a sampling frame but from the social network of existing members of the sample. In addition, researchers keep track of who recruited whom and their numbers of social contacts, and then a mathematical model of the recruitment process weighs the sample to compensate for nonrandom recruitment patterns. This model is based on a synthesis and extension of 2 areas of mathematics, Markov chain theory and biased network theory, which were not part of the standard tool kit of mathematical sampling theory. The resulting statistical theory enables researchers to provide both unbiased population estimates and measures of the precision of those estimates [28]. This extends the realm within which statistically valid samples can be drawn, to include many groups of importance to public health, public policy, and arts and culture [29]. In the future, adopting the respondent-driven sampling method may be useful to recruit a bigger and more targeted sampling with a statistical weighting technique.

The findings of the withdrawal group, the asocial group, and the hikikomori group suggested that they tend to converge on internet platforms that specialize in a certain culture or type of entertainment. For example, in our case, the gaming website Wandianba seems to appeal to socially withdrawn youth regardless of their withdrawal status. This feature indicates the necessity for social workers and other service providers to develop sufficient knowledge about popular youth culture and websites in their attempts to work with socially withdrawn youths [30]. This not only opens up possible channels to reach out to socially withdrawn youths but also serves as an approach to engage these young people.

Although it is found that online or cyber social capital can positively contribute to individuals’ well-being [31,32], no study has yet measured their social relationships and social networks online and how they affect young people’s social withdrawal behavior. Our study reveals an intriguing phenomenon in that young people in the asocial group did not avoid all kinds of social interaction. Moreover, by connecting with people from different backgrounds online, the withdrawal group could establish significant online bridging social capital. However, the hikikomori group might be prone to cut themselves off from weak ties, such as those with former colleagues and classmates, and as a result, offline bridging social capital cannot be built. The findings indicate the need to adopt a strength-based approach in working with different types of socially withdrawn youths. By examining and capitalizing on how those who work with socially withdrawn youths develop and maintain online social relationships with these young people, we may finally help them to figure out ways into the real social world.

Although internet use has reached nearly 50% in China [33], it clearly does not include everyone in some target audiences. Socioeconomic factors such as internet and smartphone accessibility limit the types of individuals who are exposed to a survey [34]. This clearly limits the audience reach. In addition, as a general limitation of the Web survey, the nonresponse rate increases the bias of estimators [35]. It is likely that the socially withdrawn youths who responded to the survey were those who were still motivated to connect with the outside world. Young people who totally cut themselves out from the social world might not be reached through this study. Moreover, studies that examine sensitive topics may not be as feasible in countries such as China because of internet censorship and a government-controlled media environment. This situation calls for a joint effort from mental health professionals, social workers, and psychiatrists to codevelop research and practices to assist the young people to reconnect with our society [36].

As the first attempt to understand the population and situation of hikikomori in mainland China, this study has explored 3 different Web-based platforms in China (ie, 2 prevalently used social media Weibo and WeChat and 1 online gaming website) to reach youth in social withdrawal. We found that these platforms are viable and serve as inexpensive tools to reach socially withdrawn youth. It was also found that given the behavioral pattern of socially withdrawn youth, they tend to disengage from common social interaction, internet platforms that specialize in a certain culture or type of entertainment were more likely to reach such population.