This paper is in the following e-collection/theme issue:

Workaholism (Work Addiction) (1) Research Instruments, Questionnaires, and Tools (576) Diagnostic Tools in Mental Health (242)

Published on in Vol 5, No 1 (2018): Jan-Mar

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/8215, first published .
Work Addiction Test Questionnaire to Assess Workaholism: Validation of French Version

Work Addiction Test Questionnaire to Assess Workaholism: Validation of French Version

Work Addiction Test Questionnaire to Assess Workaholism: Validation of French Version

Authors of this article:

Hortense Ravoux1, 2 Author Orcid Image ;   Bruno Pereira3 Author Orcid Image ;   Georges Brousse4, 5 Author Orcid Image ;   Samuel Dewavrin6 Author Orcid Image ;   Thomas Cornet6 Author Orcid Image ;   Martial Mermillod7, 8 Author Orcid Image ;   Laurie Mondillon2 Author Orcid Image ;   Guillaume Vallet2 Author Orcid Image ;   Farès Moustafa9 Author Orcid Image ;   Frédéric Dutheil1, 2, 10 Author Orcid Image
Article Authors Cited by (17) Tweetations (3) Metrics

    Original Paper

    1Service Santé Travail Environnement, Centre Hospitalier Universitaire de Clermont-Ferrand, Clermont-Ferrand, France

    2Équipe Stress physiologique et psychosocial, Laboratoire de Psychologie Sociale et Cognitive, Université Clermont Auvergne, Centre National de la Recherche Scientifique, Clermont-Ferrand, France

    3Service de Biostatistique, Direction de la Recherche Clinique et de l'Innovation, Centre Hospitalier Universitaire de Clermont-Ferrand, Clermont-Ferrand, France

    4Service Addictologie, Centre Hospitalier Universitaire de Clermont-Ferrand, Clermont-Ferrand, France

    5Neuro-Psycho-pharmacologie des systèmes dopaminergiques sous-corticaux, Université Clermont Auvergne, Clermont-Ferrand, France

    6WittyFit, Paris, France

    7Laboratoire de Psychologie et NeuroCognition, Centre National de la Recherche Scientifique, Université Grenoble Alpes, Grenoble, France

    8Institut Universitaire de France, Paris, France

    9Service des Urgences, Centre Hospitalier Universitaire de Clermont-Ferrand, Clermont-Ferrand, France

    10Faculty of Health, Australian Catholic University, Melbourne, Victoria, Australia

    Corresponding Author:

    Frédéric Dutheil, MD, PhD

    Service Santé Travail Environnement

    Centre Hospitalier Universitaire de Clermont-Ferrand

    58 rue Montalembert

    Clermont-Ferrand, F-63000

    France

    Phone: 33 6 88 22 48 48

    Fax:33 4 73 27 46 49

    Email: fred_dutheil@yahoo.fr


    Background: Work addiction is a significant public health problem with a growing prevalence. The Work Addiction Risk Test (WART) is the gold standard questionnaire to detect workaholism.

    Objective: The main objective of this study was to validate the French version of the WART.

    Methods: Questionnaires were proposed to voluntary French workers using the WittyFit software. There were no exclusion criteria. The questionnaire was administered anonymously for initial validity testing and readministered one week later for test-retest reliability. We also assessed the workers’ sociodemographic characteristics, as well as other measurements for external validity, such as stress, well-being, and coaddictions to tobacco, alcohol, and cannabis. Several psychometric properties of the French-WART were explored: acceptability, reliability (internal consistency [Cronbach alpha coefficient] and reproducibility [Lin concordance coefficient]), construct validity (correlation coefficients and principal component analysis), and external validity (correlation coefficients).

    Results: Among the 1580 workers using WittyFit, 187 (11.83%) agreed to complete the WART questionnaire. Of those, 128 completed the test-retest survey (68.4%). Acceptability found that all respondents had fully completed the questionnaire, with few floor or ceiling effects. Reliability was very good with a Cronbach alpha coefficient at .90 (internal consistency) and Lin concordance coefficient at .90 (95% CI .87-.94] with a difference on the retest of .04 (SD 4.9) (95% CI −9.6 to 9.7) (reproducibility). We identified three main dimensions (construct validity). Relationships between WART and stress and well-being confirmed its external validity.

    Conclusions: The French version of the WART is a valid and reliable instrument to assess work addiction with satisfactory psychometric properties. Used in occupational medicine, this tool would allow the diagnosis of work addiction and can be easily implemented in current practice.

    JMIR Ment Health 2018;5(1):e12

    doi:10.2196/mental.8215

    Keywords

    behavior, addictive; work; validation studies as topic; questionnaires; social welfare; health; public health 


    Addiction to work, or workaholism, is defined as “a compulsion or an uncontrollable need to work incessantly” [1-3]. This pathology is in line with the general criteria of addiction, that is, preoccupation with an addictive object or behavior, mood modification, interpersonal conflict, withdrawal syndrome, tolerance, relapses, or continuation of this behavior despite the knowledge of its negative effects [2-5]. There is a real typology based on the major characteristics of individuals (the compulsive-dependent, the perfectionists, the achievement-oriented, the bulimic, the relentless, etc) [2,6,7]. It is important to differentiate workers suffering from workaholism from those who are engaged at work [4,8,9]. Workaholics are propelled by an obsessive inner drive they cannot resist, whereas engaged workers are intrinsically motivated [8,10]. Addiction to work results from the individual’s predisposition, sociocultural experiences, and behavioral reinforcements [1,2,9,11]. Addiction to work is a growing public health concern [1,11,12] with a prevalence ranging from 7.6% [13] to 22.2% [14] in European countries. Workaholics dedicate more time and energy to work than seems necessary [2,7,8]. This behavioral addiction would negatively affect the individual’s health and could lead to relationship problems (family conflicts, marital problems, impact on their children, and poor social relationship) [10,11,15], neuropsychic troubles (depression, burnout, sleep disorders, and general dissatisfaction) [2,4,8-10,15-17], consequences to professional life in the long term (lower productivity levels, absences, and strain at work) [1-3,11,17], and poorer physical health [11].

    As we are being confronted with this growing health problem, it appears absolutely essential to possess validated tools. The Work Addiction Risk Test (WART) is a reference questionnaire for work addiction [2]. This test was developed by Robinson et al in 1999 [2,18-20] based on the experiences of clinicians treating workaholics [2]. We chose this tool because of its wide use (approximately 150 studies) and usability [2]. The English version of the WART has satisfactory psychometric properties [6,20]. Reliability is represented by internal consistency with a Cronbach alpha coefficient ranging from .85 [21] to .88 [18], and reproducibility with a test-retest correlation coefficient of .83 [20,22], and a Spearman-Brown split-half reliability coefficient of 0.85 [20]. Construct validity is built around five dimensions: compulsive tendencies, control, impaired communication and self-absorption, inability to delegate, and self-worth [20]. For external validity, work addiction was linked to a high level of stress [3,14,15,18,23] and a poor level of well-being [1-4]. In addition to work addiction, the same worker may also suffer from several addictions, such as consuming tobacco, cannabis, or alcohol [1,24]. To our knowledge, no studies have reported acceptability of the English version of WART.

    The main objective of this investigation was the validation of the French version of the WART to allow its use in current practice. We aimed to evaluate its acceptability, reliability, construct, validity, and external validity. Stress, well-being, and coaddictions to tobacco, alcohol, and cannabis were used for evaluating external validity.


    Recruitment

    Questionnaires were proposed to voluntary French workers using the WittyFit software [25]. WittyFit is a Web platform that aims to improve the well-being at work, with a public-private partnership with the University Hospital of Clermont-Ferrand. Workers using WittyFit answer-validated questionnaires on behavioral data for baseline health profiling. The concept of WittyFit is to provide individualized feedback based on evidence-based medicine, with an aim to support behavioral change using a formal evaluation of changes in knowledge, practices, and health outcomes over time. The database is implemented from a human resource–generated number, which is then automatically converted into another number in the WittyFit database. Data provided by employers (such as from the professional roles or the occupational sector) are automatically associated with the human resource–generated number. All data are anonymous, and the name of the employee is never entered into the database. The study was approved by the National Commission for Data Protection and Liberties and by the South-East VI Ethics Committee (ClinicalTrials.gov NCT number NCT02596737). There were no exclusion criteria. The WittyFit users were informed of a forthcoming questionnaire validation study through this platform explaining the purpose of the study and the need to complete the questionnaire twice (test and retest, 1 week later).

    The Work Addiction Risk Test Questionnaire

    The WART is a self-administered test with 25 statements for which the answers are scored 1-never true to 4-always true [20,26]. Respondents read the statements and mark their answers to describe their work habits [19,27]. The total score is the sum of the responses to the items—25 to 100—and the higher the score, the more one is considered addicted to work [18,21]. Scores from 25 to 56 were defined as “at low-risk of work addiction”; 57 to 66 as “at medium-risk of work addiction”, and from 67 to 100 as “at high-risk of work addiction” [12,18]. The WART consists of five dimensions, including compulsive tendencies (9 items: 3, 5, 6, 7, 8, 15, 18, 19, 20); control (7 items: 2, 4, 11, 12, 16, 17, 22); impaired communication and self-absorption (5 items: 13, 21, 23, 24, 25); inability to delegate (1 item: 1); and self-worth (2 items: 9 and 10) [1,15,20]. The first two dimensions are the key elements to differentiate workaholics [1].

    Translation of the Work Addiction Risk Test

    In accordance with the literature [28], the following steps were performed for the validation of the French version of the WART: (1) translations of the WART into French performed by 2 independent native French translators; (2) back translation of the French version of the WART into English by 2 native English speakers, who had no knowledge of the original English version; (3) synthesis and comparison of all translations by a committee of experts, multidisciplinary and bilingual, to develop the final version of the WART; (4) validation study of the French version. The questionnaire was administered for the initial validity testing and readministered 1 week later for test-retest reliability. Workers received an individual alert through the WittyFit software to complete the surveys. The French version of the WART is presented in Multimedia Appendix 1.

    External Validity: Other Measurements

    Well-being and perceived stress at work and at home were evaluated using visual analog scales (VAS) by moving a cursor on a horizontal, noncalibrated line, ranging from very low (0) on the left to very high (100) on the right [29-31]. Furthermore, tobacco, alcohol, and cannabis consumption were evaluated using 3 specific VAS—the number of cigarettes smoked per day from 0 to 30, the number of glasses of alcohol consumed per day from 0 to 8, and the number of cannabis consumption per month from 0 to 30.

    Statistical Analysis

    The number of subjects required were determined in advance by following recommendations [32] and in accordance with our recruitment abilities. In this context, a complement of at least 120 participants appeared to be relevant for the test and 60 for the retest.

    Statistical analysis was carried out using the Stata software version 13 (StataCorp). Qualitative data were described in size and associated frequencies, and the data were compared between groups—those who only completed the questionnaire once (test) and those who completed the survey twice (test and retest) —with the chi-square test or with the Fisher exact test. Quantitative data, expressed by the mean (SD) or the median (interquartile ranges) regarding the statistical distribution (the Shapiro-Wilk test), were compared between groups with the Student t test (or an analysis of variance [ANOVA]) or Mann-Whitney U test or the Kruskal-Wallis test, if the t test’s conditions were not respected (normality and homoscedasticity considered by the Bartlett test) for the quantitative variables. When appropriate (P<.05), a post hoc test for multiple comparison was deemed, namely, the Tukey-Kramer postANOVA test and Dunn test after the Kruskal-Wallis test. The comparisons between the groups for category parameters were achieved with the chi-square test or with the Fisher exact test. The difference was defined as statistically significant when the level of significance (P) was less than .05 (alpha risk=5%).

    The psychometric properties of the WART were explored. We assessed the acceptability based on the calculation of missing data for each item and the dimension of the WART—data quality was deemed acceptable if less than 5% of data were missing. The accepted maximum for floor and ceiling effects was 15% [32]. The reliability of the French version was evaluated according to two criteria: (1) internal consistency based on the calculation of the Cronbach alpha coefficient (desirable values higher than .70-0.80) [32-39] and (2) reproducibility. The correlation coefficient (Pearson or Spearman, as per the statistic distribution) and Lin concordance coefficient were computed to assess test-retest reliability [40]. Values above .7 were considered satisfactory. An analysis using a mixed model (random effect subject time) and Bland and Altman’s graphic illustrations completed the analysis. The construct validity of the French version of the WART was explored by reviewing interitem and interdimensional correlations and using multidimensional factorial analysis (principal component analysis). The analysis of the interdimensional correlations assessed the redundancy between dimensions with expected positive correlations but which were not too high (.60-.80) [32-39]. The multidimensional analysis allowed the assessment of the gathered items with regard to different dimensions. The external validity was assessed by studying correlations between the WART and other psychological measures, such as perceived stress, well-being, or other putative addiction.


    Participants

    Among the 1580 workers using WittyFit, 11.83% (187/1580) agreed to answer the WART questionnaire. Among them, 86.1% (161/187) completed the sociodemographic characteristics and the VAS. The test-retest survey was completed by 68.4% (128/187) workers. Workers’ characteristics did not differ between those who only completed the questionnaire once (test) and those who completed the survey twice (test and retest), except in tobacco consumption, with more smoker participants responding only in the test than the participants who responded both at the test and retest (32 vs 16% of smokers, P=.01) (Table 1).

    Results From the Work Addiction Risk Test Questionnaire

    Of the 187 individuals who completed the WART questionnaire, 45.5% (85/187) were at low risk of work addiction, 32.6% (61/187) at medium risk, and 20.8 (41/197) at high risk. Women had a higher risk of work addiction than men did (27% vs 15% of workers at high risk of work addiction, P=.02). According to the WART, individuals exhibiting a high risk of work addiction worked for an average of 7 more hours per week than those at a low risk—46.9 (13.6) hours versus 39.4 (10.9) hours, P=.005.

    Table 1. Difference between people at the test and retest in terms of sociodemographic characteristics.
    VariableSourceP valuea

    		Test (n=187)Test only (n=59)Test and retest (n=189)
    Sex, Women, n (%)95 (50.8)30 (52.6)58 (55.8).70
    Age (years), mean(SD)41.6 (11.7)42.0 (12.2)41.8 (11.7).92
    Family situation, n (%)




    		Single36 (20.5)17 (29.8)17 (16.4).15

    		De facto48 (27.3)12 (21.1)32 (30.8)

    		Married91 (51.7)28 (49.1)54 (51.9)

    		Widow(ed)1 (0.6)0 (0.0)1 (1.0)
    Education level, n (%)




    		General Certificate of Secondary Education2 (1.1)1 (1.8)1 (1.0).88

    		General Certificate of Education–Advanced Level8 (4.6)4 (7.0)4 (3.9)

    		Higher national diploma14 (8.0)4 (7.0)9 (8.7)

    		Bachelor’s degree22 (12.5)7 (12.3)13 (12.5)

    		Master’s degree130 (73.9)41 (71.9)77 (74.0)
    Occupational group, n (%)




    		Merchants-business6 (3.4)1 (1.8)5 (4.8).62

    		Employees31 (17.6)13 (22.8)15 (14.4)

    		Intermediate profession12 (6.8)3 (5.3)7 (6.7)

    		Inactive employment10 (5.7)4 (7.0)6 (5.8)

    		Manager-intellectual profession117 (66.5)36 (63.2)71 (68.3)
    Hours worked per week, mean (SD)41.6 (12.1)42.5 (12.3)40.8 (12.4).41
    Seniority in the company (years), mean (SD)10.8 (10.5)10.9 (10.2)11.1 (11.2).83
    body mass index, kg.m−2, mean (SD)24.2 (4.4)25.1 (5.0)23.7 (4.0).06
    metabolic equivalent of task, mean (SD)50.9 (55.5)53.3 (53.1)47.6 (51.4).44
    Tobacco smoker, n (%)39 (20.9)19 (32.2)20 (15.6).01
    Alcohol users,n (%)30 (16.0)9 (15.3)21 (16.4).84
    Cannabis consumer,n (%)14 (7.5)5 (8.5)9 (7.0).77
    WARTb




    		Score, mean (SD)57.8 (11.2)56.2 (11.6)58.6 (11.0).14

    		% of participants with a score <5645.550.843.0.51

    aP value between test only and test and retest.

    bWART: Work Addiction Risk Test.

    Acceptability

    The results for data quality and acceptability of the French version of the WART are displayed in Figure 1 (see Multimedia Appendix 2). Data quality was commonly considered satisfactory if 95% of the scale was fully completed [41,42] (at least 24 of the 25 items). All of the 187 individuals who completed the WART questionnaire did so fully. In fact, no one partially replied to the questionnaire. Therefore, there were no missing data.

    Internal Validity

    Internal Consistency

    The entire WART had a Cronbach alpha of .90. The Cronbach alpha values for the various dimensions of the WART were .85 for compulsive tendencies, .82 for control, and .57 for impaired communication and self-absorption.

    ‎
    Figure 1. Data quality and acceptability of the French version of the Work Addiction Risk Test (WART) (n=187).
    Correlation

    Item-total correlation coefficients for the scale as a whole ranged from .02 to .59. Interitem correlations ranged from .23 (Questions 3 and 20) to .54 (Questions 7 and 8) for compulsive tendencies; .19 (Questions 16 and 22) to .50 (Questions 2 and 17) for control; and .08 (Questions 13 and 24) to .37 (Questions 21 and 23) for impaired communication and self-absorption. The assessment of the correlations between the questionnaire in its entirety and each dimension was statistically significant (P<.05) and showed that the correlation coefficient between WART and compulsive tendencies was .89; control was .84; impaired communication and self-absorption was .74; inability to delegate was .52; and self-worth was .31.

    Principal Component Analysis

    By applying the Kaiser’s criteria, in other words, the associated eigenvalues above 1 associated with a plot of the eigenvalues, we have determined four main components. Components 1 and 2 together explained the maximal variance. As presented in Figure 2, the first dimension of the French WART was associated with Component 1 and was composed of items 2, 9, 10, 11, 12, 13, 14, 16, 17, 22, and 25; the second dimension was associated with Component 2 and was composed of items 3, 4, 5, 6, 7, 8, 18, 19, and 21; and the third dimension was associated with Component 1 and was composed of items 15, 20, and 23.

    Reproducibility

    The Lin concordance coefficient was .90 (95% CI 0.87-0.94) for the entire WART with a difference between the test and the retest of 0.04 (SD 4.92) (95% CI −9.61 to 9.69). The Bland and Altman plot is shown in Figure 3. For each dimension, Lin concordance coefficients were as follows: .86 (95% CI 0.82-0.91) for compulsive tendencies, .86 (95% CI 0.82-0.91) for control, .76 (95% CI 0.68-0.83) for impaired communication and self-absorption, .73 (95% CI 0.65-0.81) for self-worth, and .66 (95% CI 0.56-0.75) for the inability to delegate (Table 2). Exhaustive results for the Lin concordance coefficient and Cohen kappa for each item are shown in Figure 4 (see Multimedia Appendix 3).

    External Validity

    External validity was evaluated by calculating a correlation coefficient between the WART and the others questionnaires (Table 3). The WART was well correlated to the VAS Stress at Work (coefficient correlation .43) and Stress at Home (.41) and inversely to VAS Well-being (−.40) (P<.05). The VAS Well-being had a reverse correlation with all dimensions of the WART (P<.05). The WART was poorly correlated with tobacco, alcohol, or cannabis consumption.

    ‎
    Figure 2. Principal component analysis: circle of correlation or the three dimensions of the French Work Addiction Risk Test (WART).
    ‎
    Figure 3. Bland and Altman plot or representation of agreement between both series of measures for the French Work Addiction Risk Test (WART).
    Table 2. Test-retest reproducibility for each dimension with measurement of the Lin concordance coefficient.
    DimensionsLin concordance coefficient (95% CI)Difference (SD), 95% CI
    Compulsive tendencies.86 (0.82-0.91)0.09 (2.71), −5.24 to 5.41
    Control.86 (0.82-0.91)0.02 (2.05), −4.00 to 4.03
    Impaired communication and self-absorption.76 (0.68-0.83)0.05 (1.69), −3.27 to −3.36
    Self-worth.73 (0.65-0.81)−0.12 (0.91), −1.91 to 1.68
    Inability to delegate.66 (0.56-0.75)0.11 (0.49), −0.85 to 1.06
    ‎
    Figure 4. Lin concordance coefficient and Cohen kappa for each item of the Work Addiction Risk Test (WART).
    Table 3. External validity of the French Work Addiction Risk Test (WART), measure of correlation between WART and its dimension, and others questionnaires.
    VariableDimensions of WART

    		WART totalCompulsive tendenciesControlImpaired communication and self-absorptionSelf-worthInability to delegate
    Tobacco, number of cigarettes smoked per day from 0 to 30−.10−.10.02−.11−.13−.06
    Alcohol, number of glass of alcohol per day from 0 to 8−.12.07.11.16a.08.02
    Cannabis, consumption per month from 0 to 30−.07−.10.02−.05.02.07
    Stress at work, VASb from 0 to 100.43a.41a.34a.32a.10.00
    Stress at home, VAS from 0 to 100.41a.36a.35a.32a.07.08
    Well-being, VAS from 0 to 100−.40a−.33a−.38a−.26a−.17a−.11a

    aP<.05.

    bVAS: visual analog scale.


    This study allowed the validation of the WART questionnaire in French and focused on its acceptability, internal validity and reproducibility, construct validity, and external validity.

    Prevalence of Work Addiction and Relationships With Workers’ Characteristics

    We found that 22% of the workers were at a high risk of suffering from work addiction, with predominance in women. Previous literature using WART demonstrated similar prevalence in similar populations, such as 13% of hospital doctors [12] and 22% for academic employees [14]. However, the prevalence of work addiction can be lower in other populations, such as in Italian teenagers (8%) [13], or with the use of other questionnaires (8%) [5,43]. Although some studies did not find gender differences in the prevalence of work addiction using the WART [12-14] or other questionnaires [5,43], the most recent studies report that women are at a higher risk of workaholism [23,44], which is in line with our results (27% vs 15%). This may suggest an evolution of women’s emancipation in our society, with more involvement at work [45,46]. We demonstrated that individuals with work addiction worked 7 hours more per week than those at low risk (46.9 vs 39.4 hours). The engagement in work, in terms of hours spent, is a characteristic of workaholism [6,16], with individuals devoting a majority of their time to work and working beyond what is required [6,16]. Our sensitivity analysis found no differences between the workers’ characteristics at the test and the retest, except for smoking. Among the workers responding only at the test, there were more smokers as compared with the workers responding to both the test and the retest (32% vs 16%).

    The smokers could have been less motivated to respond twice, with literature suggesting a link between smoking and low levels of conscientiousness [47], impulsivity [48], lack of attention [49], impaired working memory [50], or less access to Internet [51].

    Acceptability of the French Version of the Work Addiction Risk Test

    A floor or ceiling effect occurs when more than 15% of participants have the lowest or highest possible score [32,52]. A floor or ceiling effect may signify that extreme items are missing in the lower or upper end of the scale and, thus, may limit content validity. Therefore, participants with the extreme scores cannot be distinguished from each other, and reliability, as well as responsiveness is reduced because change cannot be measured in these participants [32,52]. In the French validation of the WART questionnaire, the majority of the items presented a threshold lower than 15%, as recommended in the literature [32,52]. We observed a ceiling effect for only 5 items: 1 (25% of respondents), 2 (16%), 5 (16%), 9 (35%), and 20 (16%). For example, with a possible score ranging from 1 to 4 on a 4-point Likert scale, the mean score for item 1 was 3.1 (SD 0.6) with a median of 3; 2% of responders had the lowest possible value (1) and 25% had the highest value (4). The Unfortunately, there are no studies examining the acceptability of the WART that can be used to compare our results with. So, we cannot conclude whether these results are a characteristic of our responders, a consequence of our translation, or a characteristic already present in the original English version. However, the acceptability of the French version of the WART is correct with few floor or ceiling effects. Moreover, some other well-recognized and validated questionnaires did not report acceptability in their original studies [53-57], or others reported poorer acceptability [42,58-60].

    Internal Consistency and Reproducibility

    The WART’s internal consistency appeared satisfactory with a Cronbach alpha value, which is higher than its value in the validation of the English version (.85 [21], 0.88 [18]). The English version of the WART consists of five dimensions. We highlighted the items for compulsive tendencies (Cronbach alpha=.85), and the items for control (Cronbach alpha=.82) were closely interlinked, in line with the literature [20].

    We found a high level of correlation between the overall WART score and the total score for 3 dimensions—(1) compulsive tendencies (coefficient correlation .89), (2) control (.84), and (3) impaired communication and self-absorption (.74). These three dimensions would have the greatest impact in differentiating individuals with work addiction among the population, which is in line with the literature [20]. The reproducibility study appeared satisfactory for the whole questionnaire, its dimensions, and the stand-alone items. Actually, the Lin concordance coefficient was .90 for the entire WART with a difference between the test and retest of 0.04 (SD 4.92), reflecting a very good reliability over time. This result was in line with the literature on the English version, which reported a Lin concordance coefficient for the test-retest of .83 at a 2-week interval [20-22,26]. Moreover, despite the fact that the literature on the English version of the WART did not report the Lin concordance coefficient for each dimension, we retrieved a Lin concordance coefficient higher than .80 for two dimensions (compulsive tendencies and control) in our French version.

    Construct Validity

    Despite the fact that the French version of the WART has strong psychometric properties, results of the factorial analysis did not confirm the five dimensions of the latest study [20] but instead confirmed three dimensions. Moreover, correlations between items of each dimension remained weak, as well as a poor Cronbach alpha for items of impaired communication and self-absorption. This could be explained when examining this latest construct of the five dimensions, which was based on a low level of the correlation coefficient (.30) [20] when we used a cutoff set at .6 to .8 as described in the statistical section. Moreover, the original English version of the WART had five other different dimensions that were not drawn from a statistical approach but from the five symptoms used by clinicians for the diagnosis of workaholism: “overdoing” (Items 3, 5, 6, 7, and 15), “self-worth” (Items 8, 9, 10, 19, and 20), “control-perfectionism” (Items 1, 2, 4, 11, 12, 16, 17, 18, 21, 22, and 25), “intimacy” (Items 23 and 24), and “mental preoccupation-future reference” (Items 13 and 14) [18,21]. Those five dimensions were also not demonstrated in our analysis. However, construct differences between two language versions of the same questionnaire are common, as seen for Karasek [61,62], Hospital Anxiety and Depression Scale [60], or other questionnaires [59,63,64]. Although unlikely, translation may have changed the weight of some items. As the psychometric properties of the WART were mainly assessed on university students, our sample of French workers may emphasize some cultural specificities and work habits [65]. Cultural specificities would be investigated in subsequent work.

    External Validity

    We demonstrated the relationships between the WART and VAS for “stress at work” or “stress at home” in accordance with the literature [14,15,23], and we confirmed the external validity of the French version of the WART questionnaire. The greater the risk of work addiction, the higher the stress [18]. Conversely, the well-being level was negatively correlated with the WART scores, as previously reported [2]. We did not find any significant relationships between the French-WART and tobacco, alcohol, or cannabis addictions, but to our knowledge, no studies have previously demonstrated such links.

    Limitations

    The response rate may seem low compared with other studies also using a questionnaire in the French population [66-71]. However, we included a substantial sample size of workers, allowing to carry out the statistical analyses with the number of subjects required, determined a priori. Moreover, a number of respondents followed recommendations for the validation of the questionnaires [32-39]. Our sample size retrieved a sufficient prevalence of workers with a high risk of work addiction to allow a robust analysis. Despite the literature reports that a high dropout rate is inherent to this type of study with several questionnaires on the Internet [72], the number of participants who responded to both the test and retest was higher than commonly reported in the literature [60,61]. Our study may have included too many questionnaires in addition to the French version of the WART, which could have negatively affected participation [72]. Despite the observed difference of the construct, our study emphasized the excellent psychometric properties of the French version of the WART in terms of internal consistency, reproducibility, and external validity. Furthermore, there are other validated questionnaires with an internal construct differing from their original version [60-64]. We used some nonvalidated VAS. We did not control the size and type of screens used by the workers to complete the questionnaires, which may have affected our results, especially for VAS. To our knowledge, no studies have previously evaluated the influence of perception side on scores at VAS; and a study comparing answers to VAS of stress and well-being throughout different supports (paper, large computer screen, tablet, and a smartphone) is needed.

    Conclusions

    The French version of the WART is a valid and reliable instrument to assess work addiction, with satisfactory psychometric properties. Used in occupational medicine, this tool would allow the diagnosis of work addiction and can be easily implemented in current practice.

    Acknowledgments

    The study is funded by WittyFit. We express our sincere gratitude to all the voluntary workers using WittyFit, who participated in this study.

    Authors' Contributions

    FD participated as the main investigator. HR and FD contributed to the conception of the protocol. SD, TC, and FD created the specific software for this study. HR, BP, and FD contributed to data analysis. HR and FD drafted the manuscript. All authors revised the manuscript and added substantial input. All authors read and approved the final manuscript. HR, BP, and FD take responsibility for the integrity of the data analysis.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    The French version of the WART.

    PDF File (Adobe PDF File), 35KB
    ‎

    Multimedia Appendix 2

    Data quality and acceptability of the French version of the WART.

    PDF File (Adobe PDF File), 27KB
    ‎

    Multimedia Appendix 3

    Data of Lin concordance coefficient and Cohen kappa for each item of the WART.

    PDF File (Adobe PDF File), 29KB
    1. Quinones C, Griffiths MD. Addiction to work: a critical review of the workaholism construct and recommendations for assessment. J Psychosoc Nurs Ment Health Serv. Oct 2015;53(10):48-59. [CrossRef] [Medline]
    2. Andreassen CS. Workaholism: An overview and current status of the research. J Behav Addict. Mar 2014;3(1):1-11. [FREE Full text] [CrossRef] [Medline]
    3. Sussman S. Workaholism: a review. J Addict Res Ther. Jan 10, 2012;Suppl 6(1):pii: 4120. [FREE Full text] [CrossRef] [Medline]
    4. Shimazu A, Schaufeli WB, Kamiyama K, Kawakami N. Workaholism vs. work engagement: the two different predictors of future well-being and performance. Int J Behav Med. Feb 2015;22(1):18-23. [CrossRef] [Medline]
    5. Andreassen CS, Griffiths MD, Hetland J, Kravina L, Jensen F, Pallesen S. The prevalence of workaholism: a survey study in a nationally representative sample of Norwegian employees. PLoS One. 2014;9(8):e102446. [FREE Full text] [CrossRef] [Medline]
    6. Buelens M, Poelmans SA. Enriching the Spence and Robbins' typology of workaholism. J Organ Change Manag. Oct 2004;17(5):440-458. [CrossRef]
    7. Scott KS, Moor KS, Miceli MP. An exploration of the meaning and consequences of workaholism. Hum Relat. 1997;50(3):287-314. [CrossRef]
    8. Schaufeli WB, Shimazu A, Taris TW. Being driven to work excessively hard: the evaluation of a two-factor measure of workaholism in the Netherlands and Japan. Cross Cult Res. May 19, 2009;43(4):320-348. [CrossRef]
    9. Van Beek I, Taris TW, Schaufeli WB. Workaholic and work engaged employees: dead ringers or worlds apart? J Occup Health Psychol. Oct 2011;16(4):468-482. [CrossRef] [Medline]
    10. Schaufeli WB, Bakker AB, van der Heijden FM, Prins JT. Workaholism, burnout and well-being among junior doctors: the mediating role of role conflict. Work Stress. Apr 2009;23(2):155-172. [CrossRef]
    11. Ng TW, Sorensen KL, Feldman DC. Dimensions, antecedents, and consequences of workaholism: a conceptual integration and extension. J. Organ Behav. Jan 2006;28(1):111-136. [CrossRef]
    12. Rezvani A, Bouju G, Keriven-Dessomme B, Moret L, Grall-Bronnec M. Workaholism: are physicians at risk? Occup Med (Lond). Sep 2014;64(6):410-416. [CrossRef] [Medline]
    13. Villella C, Martinotti G, Di Nicola M, Cassano M, La Torre G, Gliubizzi MD. Behavioural addictions in adolescents and young adults: results from a prevalence study. J Gambl Stud. Jun 2011;27(2):203-214. [CrossRef] [Medline]
    14. Bartczak M, Oginska-Bulik N. Workaholism and mental health among Polish academic workers. Int J Occup Saf Ergon. 2012;18(1):3-13. [CrossRef] [Medline]
    15. Matuska KM. Workaholism, life balance, and well‐being: a comparative analysis. J Occup Sci. Jun 2010;17(2):104-111. [CrossRef]
    16. Spence JT, Robbins AS. Workaholism: definition, measurement, and preliminary results. J Pers Assess. Feb 1992;58(1):160-178. [CrossRef] [Medline]
    17. Kubota K, Shimazu A, Kawakami N, Takahashi M, Nakata A, Schaufeli WB. Association between workaholism and sleep problems among hospital nurses. Ind Health. 2010;48(6):864-871. [Medline]
    18. Robinson BE. The Work Addiction Risk Test: development of a tentative measure of workaholism. Percept Mot Skills. Feb 1999;88(1):199-210. [CrossRef] [Medline]
    19. Robinson BE, Post P. Validity of the work addiction risk test. Percept Mot Skills. Feb 1994;78(1):337-338. [CrossRef] [Medline]
    20. Flowers CP, Robinson B. A structural and discriminant analysis of the work addiction risk test. Educ Psychol Meas. Jun 2002;62(3):517-526. [CrossRef]
    21. Robinson BE, Post P. Risk of addiction to work and family functioning. Psychol Rep. Aug 1997;81(1):91-95. [CrossRef] [Medline]
    22. Robinson BE, Post P, Khakee JF. Test-retest reliability of the work addiction risk test. Percept Mot Skills. Jun 1992;74(3 Pt 1):926. [CrossRef] [Medline]
    23. Andreassen CS, Griffiths MD, Sinha R, Hetland J, Pallesen S. The relationships between workaholism and symptoms of psychiatric disorders: a large-scale cross-sectional study. PLoS One. 2016;11(5):e0152978. [FREE Full text] [CrossRef] [Medline]
    24. Haylett SA, Stephenson GM, Lefever RM. Covariation in addictive behaviours: a study of addictive orientations using the Shorter PROMIS Questionnaire. Addict Behav. Jan 2004;29(1):61-71. [Medline]
    25. Dutheil F, Duclos M, Naughton G, Dewavrin S, Cornet T, Huguet P, et al. WittyFit-Live Your Work Differently: study protocol for a workplace-delivered health promotion. JMIR Res Protoc. Apr 13, 2017;6(4):e58. [FREE Full text] [CrossRef] [Medline]
    26. Robinson BE. Concurrent validity of the work addiction risk test as a measure of workaholism. Psychol Rep. Dec 1996;79(3 Pt 2):1313-1314. [CrossRef] [Medline]
    27. Robinson BE, Phillips B. Measuring workaholism: content validity of the work addiction risk test. Psychol Rep. Oct 1995;77(2):657-658. [CrossRef] [Medline]
    28. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. Dec 1993;46(12):1417-1432. [Medline]
    29. Dutheil F, Trousselard M, Perrier C, Lac G, Chamoux A, Duclos M, et al. Urinary interleukin-8 is a biomarker of stress in emergency physicians, especially with advancing age--the JOBSTRESS* randomized trial. PLoS One. 2013;8(8):e71658. [FREE Full text] [CrossRef] [Medline]
    30. Dutheil F, Boudet G, Perrier C, Lac G, Ouchchane L, Chamoux A, et al. JOBSTRESS study: comparison of heart rate variability in emergency physicians working a 24-hour shift or a 14-hour night shift--a randomized trial. Int J Cardiol. Jul 12, 2012;158(2):322-325. [CrossRef] [Medline]
    31. Dutheil F, Pereira B, Moustafa F, Naughton G, Lesage FX, Lambert C. At-risk and intervention thresholds of occupational stress using a visual analogue scale. PLoS One. 2017;12(6):e0178948. [FREE Full text] [CrossRef] [Medline]
    32. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. Jan 2007;60(1):34-42. [CrossRef] [Medline]
    33. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. May 2012;21(4):651-657. [FREE Full text] [CrossRef] [Medline]
    34. Terwee CB, Prinsen CA, Ricci Garotti MG, Suman A, de Vet HC, Mokkink LB. The quality of systematic reviews of health-related outcome measurement instruments. Qual Life Res. Apr 2016;25(4):767-779. [FREE Full text] [CrossRef] [Medline]
    35. Mokkink LB, Terwee CB, Gibbons E, Stratford PW, Alonso J, Patrick DL, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) checklist. BMC Med Res Methodol. Sep 22, 2010;10:82. [FREE Full text] [CrossRef] [Medline]
    36. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. Mar 18, 2010;10:22. [FREE Full text] [CrossRef] [Medline]
    37. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC Med Res Methodol. Jan 24, 2006;6:2. [FREE Full text] [CrossRef] [Medline]
    38. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. May 2010;19(4):539-549. [FREE Full text] [CrossRef] [Medline]
    39. Mokkink LB, Terwee CB, Stratford PW, Alonso J, Patrick DL, Riphagen I, et al. Evaluation of the methodological quality of systematic reviews of health status measurement instruments. Qual Life Res. Apr 2009;18(3):313-333. [CrossRef] [Medline]
    40. Knell G, Gabriel KP, Businelle MS, Shuval K, Wetter DW, Kendzor DE. Ecological momentary assessment of physical activity: validation study. J Med Internet Res. Jul 18, 2017;19(7):e253. [FREE Full text] [CrossRef] [Medline]
    41. Pourtier-Piotte C, Pereira B, Soubrier M, Thomas E, Gerbaud L, Coudeyre E. French validation of the Foot Function Index (FFI). Ann Phys Rehabil Med. Oct 2015;58(5):276-282. [FREE Full text] [CrossRef] [Medline]
    42. Rieu I, Martinez-Martin P, Pereira B, De Chazeron I, Verhagen Metman L, Jahanshahi M, et al. International validation of a behavioral scale in Parkinson's disease without dementia. Mov Disord. Apr 15, 2015;30(5):705-713. [CrossRef] [Medline]
    43. Taris TW, Van Beek I, Schaufeli WB. Demographic and occupational correlates of workaholism. Psychol Rep. Apr 2012;110(2):547-554. [CrossRef] [Medline]
    44. Beiler-May A, Williamson RL, Clark MA, Carter NT. Gender bias in the measurement of workaholism. J Pers Assess. 2017;99(1):104-110. [CrossRef] [Medline]
    45. Palumbo AJ, De Roos AJ, Cannuscio C, Robinson L, Mossey J, Weitlauf J, et al. Work characteristics associated with physical functioning in women. Int J Environ Res Public Health. Apr 15, 2017;14(4):pii: E424. [FREE Full text] [CrossRef] [Medline]
    46. Peckham TK, Baker MG, Camp JE, Kaufman JD, Seixas NS. Creating a future for Occupational health. Ann Work Expo Health. Jan 01, 2017;61(1):3-15. [CrossRef] [Medline]
    47. Yáñez AM, Leiva A, Estela A, Čukić I. The associations of personality traits and parental education with smoking behaviour among adolescents. PLoS One. Mar 2017;12(3):e0174211. [FREE Full text] [CrossRef] [Medline]
    48. Dakwar E, Popii M, Coccaro EF. Lifetime history of cigarette smoking associated with aggression and impulsivity in both healthy and personality disordered volunteers. J Pers Disord. Oct 2011;25(5):645-655. [FREE Full text] [CrossRef] [Medline]
    49. Treur JL, Willemsen G, Bartels M, Geels LM, van Beek JH, Huppertz C, et al. Smoking during adolescence as a risk factor for attention problems. Biol Psychiatry. Nov 01, 2015;78(9):656-663. [CrossRef] [Medline]
    50. Jacobsen LK, Krystal JH, Mencl WE, Westerveld M, Frost SJ, Pugh KR. Effects of smoking and smoking abstinence on cognition in adolescent tobacco smokers. Biol Psychiatry. Jan 01, 2005;57(1):56-66. [CrossRef] [Medline]
    51. Cunningham JA. Access and interest: two important issues in considering the feasibility of web-assisted tobacco interventions. J Med Internet Res. Nov 03, 2008;10(5):e37. [FREE Full text] [CrossRef] [Medline]
    52. Wamper KE, Sierevelt IN, Poolman RW, Bhandari M, Haverkamp D. The Harris hip score: do ceiling effects limit its usefulness in orthopedics? Acta Orthop. Dec 2010;81(6):703-707. [FREE Full text] [CrossRef] [Medline]
    53. Fagerström KO. Measuring degree of physical dependence to tobacco smoking with reference to individualization of treatment. Addict Behav. 1978;3(3-4):235-241. [Medline]
    54. Karasek R, Brisson C, Kawakami N, Houtman I, Bongers P, Amick B. The job content questionnaire (JCQ): an instrument for internationally comparative assessments of psychosocial job characteristics. J Occup Health Psychol. Oct 1998;3(4):322-355. [Medline]
    55. Saunders JB, Aasland OG, Babor TF, De la Fuente JR, Grant M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption--II. Addiction. Jun 1993;88(6):791-804. [Medline]
    56. Snaith RP. The hospital anxiety and depression scale. Health Qual Life Outcomes. Aug 01, 2003;1:29. [FREE Full text] [CrossRef] [Medline]
    57. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. Jun 1983;67(6):361-370. [CrossRef]
    58. Supino PG, Borer JS, Franciosa JA, Preibisz JJ, Hochreiter C, Isom OW, et al. Acceptability and psychometric properties of the Minnesota living with heart failure questionnaire among patients undergoing heart valve surgery: validation and comparison with SF-36. J Card Fail. Apr 2009;15(3):267-277. [CrossRef] [Medline]
    59. Micoulaud-Franchi JA, Sauteraud A, Olive J, Sagaspe P, Bioulac S, Philip P. Validation of the French version of the acceptability e-scale (AES) for mental e-health systems. Psychiatry Res. Mar 30, 2016;237:196-200. [CrossRef] [Medline]
    60. Reda AA. Reliability and validity of the Ethiopian version of the hospital anxiety and depression scale (HADS) in HIV infected patients. PLoS One. Jan 25, 2011;6(1):e16049. [FREE Full text] [CrossRef] [Medline]
    61. Hoang TG, Corbière M, Negrini A, Pham MK, Reinharz D, Hoang TG, et al. Validation of the Karasek-job content questionnaire to measure job strain in Vietnam. Psychol Rep. Oct 2013;113(2):363-379. [CrossRef] [Medline]
    62. Pelfrene E, Vlerick P, Mak RP, De Smet P, Kornitzer M, De Backer G. Scale reliability and validity of the Karasek 'job demand-control-support' model in the Belstress study. Work Stress. Oct 2001;15(4):297-313. [CrossRef]
    63. Chamayou JL, Tsenova V, Gonthier C, Blatier C, Yahyaoui A. [French validation of the frustration discomfort scale]. Encephale. Aug 2016;42(4):325-332. [CrossRef] [Medline]
    64. Dachez J, Ndobo A, Ameline A. French validation of the multidimensional attitude scale toward persons with disabilities (MAS): the case of attitudes toward autism and their moderating factors. J Autism Dev Disord. Aug 2015;45(8):2508-2518. [CrossRef] [Medline]
    65. Maio G. The cultural specifity of research ethics--or why ethical debate in France is different. J Med Ethics. Jun 2002;28(3):147-150. [FREE Full text] [Medline]
    66. Dutheil F, Delaire P, Boudet G, Rouffiac K, Djeriri K, Souweine B, et al. [Cost/effectiveness comparison of the vaccine campaign and reduction of sick leave, after vaccination against influenza among the Clermont-Ferrand University Hospital staff]. Med Mal Infect. 2008;38(11):567-573. [CrossRef] [Medline]
    67. Dutheil F, Kelly C, Biat I, Provost D, Baud O, Laurichesse H, et al. [Relation between the level of knowledge and the rate of vaccination against the flu virus among the staff of the Clermont-Ferrand University hospital]. Med Mal Infect. Nov 2008;38(11):586-594. [CrossRef] [Medline]
    68. Kelly C, Dutheil F, Haniez P, Boudet G, Rouffiac K, Traore O, et al. [Analysis of motivations for antiflu vaccination of the Clermont-Ferrand University Hospital staff]. Med Mal Infect. Nov 2008;38(11):574-585. [CrossRef] [Medline]
    69. Lanhers C, Pereira B, Garde G, Maublant C, Dutheil F, Coudeyre E. Evaluation of 'I-Preventive': a digital preventive tool for musculoskeletal disorders in computer workers-a pilot cluster randomised trial. Br Med J Open. Sep 22, 2016;6(9):e011304. [FREE Full text] [CrossRef] [Medline]
    70. Lopez V, Chamoux A, Tempier M, Thiel H, Ughetto S, Trousselard M, et al. The long-term effects of occupational exposure to vinyl chloride monomer on microcirculation: a cross-sectional study 15 years after retirement. Br Med J Open. Jun 20, 2013;3(6):pii: e002785. [FREE Full text] [CrossRef] [Medline]
    71. Tekath M, Dutheil F, Bellini R, Roche A, Pereira B, Naughton G, et al. Comparison of the ultra-low-dose Veo algorithm with the gold standard filtered back projection for detecting pulmonary asbestos-related conditions: a clinical observational study. Br Med J Open. May 30, 2014;4(5):e004980. [FREE Full text] [CrossRef] [Medline]
    72. Edwards P, Roberts I, Sandercock P, Frost C. Follow-up by mail in clinical trials: does questionnaire length matter? Control Clin Trials. Feb 2004;25(1):31-52. [CrossRef] [Medline]

    ‎
    ANOVA: analysis of variance
    VAS: visual analog scale
    WART: Work Addiction Risk Test

    Edited by G Eysenbach; submitted 30.06.17; peer-reviewed by L Fontana, P Jimenez; comments to author 07.09.17; revised version received 25.10.17; accepted 30.11.17; published 13.02.18.

    Copyright

    ©Hortense Ravoux, Bruno Pereira, Georges Brousse, Samuel Dewavrin, Thomas Cornet, Martial Mermillod, Laurie Mondillon, Guillaume Vallet, Farès Moustafa, Frédéric Dutheil. Originally published in JMIR Mental Health (http://mental.jmir.org), 13.02.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.