Two samples were used in this study: a French-speaking sample of World of Warcraft (WoW) players and a French-speaking sample of Internet poker players. These two samples completed the same modified version of the IAT. The ethical committee of the Department of Psychology of the University of Geneva—for the WoW sample—and the ethical committee of the Geneva University Hospitals—for the poker players sample—approved the study.

The WoW sample was taken from a larger study on the relationships between players’ self-reported motives to play and their in-game behaviors [46]. To participate in this study, an individual had to be a French-speaking WoW player and at least 18 years old. Participants were recruited through advertisements posted in dedicated French-language forums: a guilds forum, an official Blizzard WoW forum, and more general online and video games forums. Some participants also joined the study after hearing about it through the local press or television interviews. All participants gave their online consent prior to starting the online survey. The sample included 920 subjects who completed the French-language translation of the IAT [32] modified for online gaming. The mean age of IAT completers was 26.0 years (SD 7.8) and 807 of the 920 subjects (87.7%) were men.

Because no clear factor structure has emerged in the literature and because different studies that found the same number of factors were inconsistent regarding factor loadings, we decided to assess the factor structure underlying this questionnaire from scratch (ie, without imposing a specific model or number of factors). In order to achieve this goal, both samples were randomly split into two subsamples of half of the size of the original ones (ie, 107 subjects for the poker sample and 460 for the WoW sample). Two principal component analyses (PCAs) were first performed on the first subsamples separately. With the discrete nature of the IAT items, PCA is preferred over factor analysis since PCA does not assume any particular multivariate model, which is not the case for factor analysis [54]. Moreover, it is known that when the same number of factors or components are extracted, both techniques yield highly similar results [55]. The number of components to extract was determined by the scree test [56] and by Velicer’s minimum average partial (MAP) test done on the correlation matrix [57]. The reliability of the questionnaire was assessed by using the Cronbach alpha coefficient [58], which is a measure of internal consistency.

In a second step, two confirmatory factor analyses (CFAs) were conducted to validate the structure that emerged from the PCA. The CFAs were run on the second subsamples. For the same reasons that PCA was preferred, the unweighted least-square method was chosen as the procedure for estimation. Four preestablished criteria were selected as indicators of the goodness of fit to the data: (1) goodness-of-fit index >.90 [59], (2) adjusted goodness-of-fit index >.80 [59], (3) normed-fit index >.90 [60], and (4) root-mean-square error <.08. The use and cutoff of the goodness-of-fit index and the adjusted goodness-of-fit index were recommended by Cole [61], the normed-fit index by Bentler and Bonnet [60], and the root-mean-square error by Hu and Bentler [62].

The PCA was done with R 3.1.0, using psych and bootstrap packages, and the CFA was done with AMOS 21.0.0 [63].

The MAP test and the scree test clearly suggested in both subsamples that one component be extracted. In order to evaluate the stability of the PCA, a bootstrap technique [64] was performed first with the MAP test, which confirmed the one-factor solution—among the 1000 bootstrap samples, 60.10% (601) and 84.60% (846) suggested retaining one factor in the poker and in the WoW subsamples, respectively. The bootstrap was also applied to factor loadings of the PCA. Items 4 and 6 had a very low loading on the factor, confirmed by the confidence intervals based on the bootstrap in both subsamples, suggesting that these questions may not be well-suited for the questionnaire (Table 2).

The percentage explained variance (95% CI) was 41.6 (31.6-51.1) for poker players and 36.1 (32.6-39.8) for WoW players. The reliability, as reported by Cronbach alpha (95% CI), was .92 (.88-.95) for poker players and .90 (.88-.92) for WoW players.

Cronbach alpha was above .90 in both subsamples, which was found to be excellent. It is worth noting that when Item 4 or Item 6 were removed, Cronbach alpha increased from .92 to .93 for poker players and from .90 to .91 for WoW players.

According to the cutoff defined above, all four goodness-of-fit indices were considered excellent in both subsamples (Table 3).

Because some items had low loadings and some questions had more missing values than occurred in the rest of the questionnaire, we performed additional investigations. In particular, Question 4—“How often do you form new relationships with fellow game users?”—seemed somewhat outdated and thus no longer relevant. Moreover, it had a low loading and decreased Cronbach alpha. Therefore, we decided to remove it.

Since Question 6—“How often do your grades or school work suffer because of the amount of time you spend in-game?”—is more suitable for school-aged persons, whereas Question 8—“How often does your job performance or productivity suffer because of the game?”—is more adapted to adults, we decided to merge the two questions into one. This new question addresses the consequences for the participant’s principal occupation, either school or work, preventing the participant from omitting the answer because it is not applicable. For the same reasons, we also merged Question 3—“How often do you prefer the excitement of the game to intimacy with your partner?”—and Question 19—“How often do you choose to spend more time in-game over going out with others?”

These modifications led to a 17-item questionnaire. Despite the fact that this version had not been tested on new subjects, we performed the same analyses as we did for the original questionnaire—randomly split both samples into two subsamples, running MAP, PCA, and CFA—by using the data at hand. For the merged questions, we decided to create two new items as follows: use the maximum mark of the IAT Item 3 and Item 19, as well as of Item 6 and Item 8, for each participant when both questions have been answered, or use the mark of only a single answered question.

In accordance with these modifications, the WoW sample size of IAT completers increased from 920 to 942 subjects, and the French-speaking poker sample size increased from 214 to 232. As expected, the results with this new method of coding the questionnaire led to the same conclusions regarding the number of components to extract—the bootstrapped MAP test suggested retaining only one factor in 74.2% and 95.6% of the poker and the WoW subsamples, respectively—regarding the factorial solution with the benefit of avoiding low loadings (Table 4), and regarding Cronbach alpha. Moreover, every fit index from the CFA was the same or slightly better (Table 3).

The percentage explained variance (95% CI) was 48.4 (37.5-58.1) for poker players and 37.3 (33.4-41.0) for WoW players. The reliability, as reported by Cronbach alpha (95% CI), was .93 (.90-.96) for poker players and .89 (.87-.91) for WoW players.

This study is the first to assess, to our knowledge, the psychometric characteristics, and specifically the factorial structure, of the IAT in online samples of WoW gamers and poker players. The main finding is that the one-factor model of the IAT has good psychometric properties and fits the data well in these samples.

In consideration to both the important discrepancies in the factorial solutions found in the previous studies on the IAT and the inconsistencies in the items included in a given factor in studies with a similar number of factors (Multimedia Appendix 1), the study at hand used an exploratory approach (ie, PCA) rather than an assessment of multiple competitive CFA models. 

Although heterogeneous results were found regarding the one-factor solution in previously reported studies, it was considered the best factor solution, or as possibly an acceptable factor solution (Multimedia Appendix 1) [32-34,37-39,44,45].

As suggested elsewhere [39], the instability of the IAT factorial structure is possibly linked to its multiple strongly correlated facets. The variability of the samples studied, as well as some specific item-related aspects, may have contributed to the phenomenon. This consideration led some authors to propose modifications of the IAT [27,28,30,31,34,39].

The modified 17-item IAT scale proposed here (Table 4) also has a one-factor structure. The modification and combination of items related to a specific consequence on lifestyle (eg, students, workers) may reduce the sample variability of the proposed tool. The changes related to Item 6—impact on school—and Item 8—impact on occupation—are concordant with other findings [29,30,32,33], which previously led some authors to remove one of the two items [28,34,44,45]. Removing Item 6 or 8, however, may cause variable results, depending on the population studied (ie, students or workers). The modification proposed here may allow the answer to the combined item to depend less on the population studied. Similarly, observations about Item 3 and Item 19 of the IAT led some authors to discard or modify one of these items [28]. Again, the 17-item IAT may reduce sample-related changes in the scale characteristics.

The problematic loading of Item 4 was also reported in some studies [27,28,31,36]. In consideration of the emergence and wide dissemination of online social networks, this item appears to be less pertinent for the assessment of Internet addiction. In contrast with the results of previous studies, however, those of this study did not indicate specific problems related to Item 7—“How often do you check your email before something else that you need to do?” [27,28,35,41,44]—or Item 14—“How often do you lose sleep due to being online late at night?” [28]. Because of the concerns related to Item 7, some authors chose to delete it [28,30,41], whereas others proposed replacing “emails” with “Internet” [31]. In relation to this, the modification proposed by Lee and colleagues [31] seems relevant for further studies. The word “Internet” could be replaced with the specific Internet usage studied (ie, game). An interesting rewording of Item 14 was also proposed elsewhere [28] as follows: “How often do you lose sleep due to being online late at night?” The development of the Internet has led to possible permanent access without a requirement to log in. The lack of a specific problem with Items 7 and 14 in this study was possibly due to the age of the sample participants, for whom checking email or logging in were more familiar concepts than they were to younger people. The 17-item IAT could be adapted to include the modification proposed by Lee et al [31] for Item 7 and to that proposed by Pawlikowski et al [28] for Item 14 (Table 4).

The 17-item IAT scale (Textbox 2) and the 20-item IAT (Textbox 1, Table 1) offer interesting coverage of the main items of the DSM-5 criteria for IGD, with the exception of tolerance and time frame, which were not covered by these tools. As shown in Textbox 2, it is easy to add a time frame using a question such as “During the last year, how often…?”

As there is an absence of questions related to tolerance in the original IAT, it is then also the case for the proposed data-driven 17-item IAT. A further study may add questions specifically related to tolerance in order to assess the full range of symptoms proposed by the DSM-5 criteria for IGD.

The relatively good coverage of the 17-item IAT of the DSM-5 criteria for IGD is a possible advantage in comparison to other shorter forms of the IAT that lead, for example, to withdrawal of the escape-related item. Unsurprisingly, the 17-item IAT scale and the original IAT do not cover items related to the financial conflict-related items of the DSM-5 criteria for gambling disorder.

One of the strengths of this paper is related to the design of a 17-item IAT scale through a data-driven approach, rather than a priori choices.

Despite the possible lack of items linked to specific Internet use, it appears from this study that the original IAT and the 17-item IAT are interesting assessment tools for disorders related to excessive Internet use. In particular, the results assessed in this study add to the validity of IAT use with specific rewording, such as replacing “Internet” with “game.”

The main limitations of this study include the representativeness of self-selected samples [65], the lack of direct assessment with the modified 17-item IAT version, and the lack of concomitant diagnostic interviews.

Further studies may test the 17-item IAT among various samples in parallel with other psychopathological and Internet-related behavior assessments, including the use of the DSM-5 proposed criteria and other assessment tools, with possibly different or complementary coverage of the concept of Internet addiction [20,66-70]. The use of the original IAT, and possibly the 17-item IAT, may stimulate further research on IGD and Internet addiction. Due to the wide variety of Internet activities, such as Internet gambling and Internet gaming, it should be necessary to specify the assessed behavior (eg, WoW, other game, gambling) for each use of the 17-item IAT. Research in these fields seems to be of particular interest, as suggested by the recent inclusion of IGD in the DSM-5, Section III. The relevance of the proposed criteria in the DSM-5 needs to be confirmed, however, as some criteria, such as deception and tolerance, continue to be debated [17].