Participants for this study were 101 veterans of the Australian Defence Force who were recruited via the Trialfacts health research platform. Five participants were excluded due to missing questionnaire responses, leaving 96 participants. The inclusion criteria for this study were being a former member of the Australian Defence Force, having experienced a traumatic event, and scoring above 33 on the Posttraumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (PCL-5). Most of the sample were men (78/96, 81%), and the mean age of participants was 52.38 (SD 11.80) years. The size of the sample was determined based on the requirements of the extreme gradient boosting (XGBoost) classification models to identify those scoring as alexithymic on the TAS-20. Our sample size was larger than those in previous studies examining language features distinctive of individuals high on alexithymia [22] and using an ML classification approach to identify individuals with PTSD [29].

Before the experimental session, participants were asked to confirm if they felt comfortable discussing the traumatic event that had been affecting them the most. Participants were then asked to “think for a moment about a traumatic event you have been through” and to “tell me about this memory in detail... let yourself really try to get into this memory and how it made you feel.” Their responses were recorded for 1 minute.

The study was conducted via Zoom for all participants because we recruited veterans from across Australia. Participants initially completed informed consent, then completed the PCL-5, the TAS-20, and a range of demographic measures via Qualtrics. Participants were informed that they would be asked to describe a traumatic event they had experienced in detail. These descriptions were audio-visually recorded using the Apple QuickTime app and the record function in Zoom (Zoom Communications, Inc). Each video recording had a frame rate of 60 frames per second. After providing the description, participants were asked to rate how they felt while talking about the traumatic event on the Analogue Affect Scale. After completing this process, they were debriefed on the nature and purpose of the study.

Participants were classified in terms of alexithymic status according to scores above or below the threshold of 61 on the TAS-20. This categorization resulted in 64 participants (n=53, 83% men; n=11, 17% women) being classified as alexithymic, and 32 participants (n=25, 78% men; n=7, 22% women) as nonalexithymic. XGBoost classification models were then built using the digital phenotype variables extracted from participant descriptions of traumatic events they have experienced. XGBoost classification models have been found to be both efficient and accurate in making classifications of features extracted from recordings of individuals with psychiatric conditions in past research [54]. In this study, these models were built using the scikit-learn package [55] in Python to attempt to classify individuals who scored in the “alexithymia” range on the TAS-20. The hyperparameters were set at the default values for the XGBoost classification algorithm in scikit-learn. Feature selection using the Recursive Feature Elimination method was completed within the inner folds in a 5-fold nested cross-validation pipeline to reduce the bias involved with doing feature selection on the whole sample and subsequently, the possibility of overfitting [56]. The number of features that contributed most to maximizing the precision score were selected and retained in a “best model.” The estimated classification performance of this model was evaluated across the outer 5 folds based on average scores for the precision, recall, F₁-score, and area under the curve (AUC) metrics. Precision measures the rate at which the model is correctly identifying individuals as being in the “alexithymia” group by the number of times it is making the classification of “alexithymia.” Recall refers to the rate at which the model correctly identifies every individual with “alexithymia.” F₁-scores are based on the harmonic mean of the precision and recall scores. The F₁-score was used instead of the standard accuracy score in scikit-learn, as it has been found to be a more robust and subsequently appropriate measure when the two groups to be classified are imbalanced [57]. The AUC provides an indication of the probability that the model will rank each individual scoring above the cut-off for “alexithymia” as having a higher probability of being alexithymic than nonalexithymic. The average AUC scores derived from the receiver operating characteristic curve were used as they have been shown to be suitable for assessing classification with imbalanced datasets [58]. Variable feature importance was scored based on the average decrease in Gini impurity across all decision trees in the best-performing XGBoost model within the inner 5 folds of the nested cross-validation. The experimental process is represented in Figure 1.

All procedures for the study were approved by the University of New South Wales Human Research Ethics Committee (HC230175). All methods were performed in accordance with the relevant guidelines and regulations. Informed consent was obtained from all participants via an online form explaining what was involved in the study before signing. The privacy and confidentiality of participants’ data have been stringently maintained, with no identifying details or features being presented or included in the write up of the study. Participants were given a gift voucher of Aus $100 for participating in the study.

A chi-square test for participants’ sex at birth in the alexithymia and nonalexithymia groups found no significant group differences. One-way ANOVAs indicated no significant difference between those in the alexithymia and nonalexithymia groups for age, PCL-5, and BDI-II scores (Multimedia Appendix 1). This suggests that differences in digital phenotypes between those who were alexithymic and nonalexithymic were not driven by any of these examined covariates. Summary statistics for these variables are shown in Table 1.

The best-performing XGBoost classification model for determining whether participants scored above the cut-off for alexithymia had an average precision of 0.71, an average recall of 0.87, an average F₁-score of 0.78, and an average AUC of 0.87 (Figure 2) across the 5 outer testing folds, with a model that used 148 features (performance metrics across each fold are displayed in Table 2). The high recall score indicates that the XGBoost classification model was performing well at correctly identifying those individuals who scored above the threshold for alexithymia (“true positives”). However, in terms of the accuracy of all the alexithymia classifications it made, it was not performing as well, with 29% of those classifications being made incorrectly (“false positives”). These rates are illustrated in Figure 3, which is a confusion matrix depicting the predictions of the best performing classification model across the 5 outer folds relative to the true alexithymia labels. These predictions produced an average overall accuracy (F₁-score) of 0.78 (SD 0.07). The average AUC of 0.87 (SD 0.12) suggests that the model was performing well at assigning a higher probability that an individual scoring above the cut-off for “alexithymia” was alexithymic across each of the outer folds. The receiver operating characteristic curve depicting AUC across the outer folds of the nested cross-validation pipeline for this model is displayed in Figure 3.

Figure 4 displays the feature importance plot for the digital phenotype variables that were most important for classifying individuals as alexithymic based on their Gini importance scores. Language, facial, and vocal variables were important to the capacity of the XGBoost model to classify alexithymia, with “word use related to feeling” emerging as the predictor with the largest Gini importance score. Other language (such as “first person language sentiment” and “language sentiment”), facial (such as “mean mouth openness”), and vocal (eg, “mel frequency cepstral 10 variance”) variables were also important to the classification capacity of the XGBoost models. The importance of language, facial, and vocal variables highlights the value of taking this multimodal approach to identifying a construct such as alexithymia, which has a distinctive presentation across multiple domains.

This study examines the estimated capacity of an XGBoost classification model, built with digital phenotype variables extracted from recordings of war veterans with PTSD describing traumatic events they had experienced, to accurately classify those veterans with alexithymia. These models were built and evaluated in a nested cross-validation pipeline to minimize the impact of bias [56,59].

In line with our hypothesis, the XGBoost classification model tuned and built within the nested cross-validation pipeline demonstrated a level of accuracy and performance that indicated it could be used for classifying alexithymia in PTSD. Regarding the high recall score, with individuals who scored above the threshold for alexithymia, the model was estimated to be able to accurately classify these individuals as having alexithymia 87% of the time. The precision of the model, measuring how many of the classifications of alexithymia were accurate, was much lower, with only 71% of those classifications being accurate, suggesting the model may have been making too many “alexithymia” classifications. The XGBoost classification model had an average overall F₁-score of 0.78, which is lower than the average F₁-score achieved by the classification model for PTSD built using a similar approach [29]. This could be explained by this prior study attempting to classify trauma survivors with and without PTSD [29], whereas the present study focused only on identifying a subgroup of those with PTSD (those with alexithymia). The average AUC was 0.87, which is close to that identified for PTSD classification in a previous study and is considered model performance that suggests it has “considerable” clinical use [60]. However, this result needs to be interpreted with caution given its lack of stability, with the model only achieving an AUC of 0.55 in one of the folds. Overall, the performance of this XGBoost classification model suggests that such a model built with multiple digital phenotypes could be useful for identifying alexithymia in PTSD. This model must also be tested and validated on an independent sample of veterans with PTSD that was used in the model training process.

This study was the first to examine multiple digital phenotypes in the context of alexithymia in PTSD, and in doing so found that not only language variables but also facial and vocal features were important for the estimated classification of alexithymia. Mean mouth openness was the most relevant facial feature that contributed to classification performance. This may show that differences in how much individuals spoke, as demonstrated by the openness of their mouths, were a factor in the accurate classification of alexithymia. The most relevant vocal feature was the mel-frequency cepstral coefficient variables (mel-frequency cepstral 10 variance) and variance in CPP. The importance of variance in CPP, which is a measure of voice pathology [61,62], to the estimated capacity of the XGBoost model to classify individuals as alexithymic is consistent with past findings linking alexithymia with experiences of voice pathology using other measures [63,64]. The contribution of these facial and vocal features to the estimated classification of alexithymia expands and enhances the understanding of the expressions of emotional experience that could be relevant to this construct.

As hypothesized, language variables were important to the estimated capacity of the XGBoost model to classify individuals as alexithymic. The language variables that had the highest Gini importance scores were associated with the use of feeling words, sentiment of language, and first-person pronoun use. This aligns with foundational theoretical understandings of alexithymia as a deficit in the description of experiences that are associated with feelings and emotional sentiment [3]. It also supports previous findings that those who score higher on the TAS-20 display differences in their expression of language sentiment [22,23]. In terms of first-person pronoun use, this aligns with theoretical understandings of alexithymia involving differences in the focus placed on oneself [2] and past findings that it is associated with differences in personal pronoun use [65]. This consistency between important predictors in an XGBoost classification model and expectations based on the research domain knowledge about that construct from theoretical models and past findings is an important indication of validity for ML models [66]. However, given that it was not only language variables alone that were important for the estimated classification of alexithymia but also vocal and facial variables, this aligns more closely with the attention-appraisal model’s understanding of alexithymia as a multifaceted construct [3] than that of the language hypothesis of alexithymia [67].

This study had several limitations. The size of the sample (N=96) is modest for an ML model such as XGBoost. The higher number of features (148 features) used in the best performing XGBoost classification model relative to the number of participants in the sample can increase the possibility of overfitting and deleteriously impact the stability of the model [68,69]. However, the use of nested cross-validation has been shown to minimize the impact of overfitting even in small samples, such that the models developed are replicated well in independent test sets [59,70]. Another limitation is that there was a large difference in the size of the groups to be classified (those scoring above or below the cut-off for “alexithymia” on the TAS-20), with more veterans in the sample scoring above the cut-off for “alexithymia.” This imbalance in groups impacts the capacity of the XGBoost classification model to be accurately evaluated. In the case of imbalanced classes, the classification of majority classes tends to be more accurate than that of the minority classes [71,72]. This phenomenon likely contributed to the much higher recall score found for this model relative to the precision score. However, this imbalance also reflects the generally higher incidence of alexithymia in veteran populations with PTSD, and trying to adjust these imbalances through cost-sensitive learning that incorporates oversampling or undersampling has been shown to have substantial limitations such as increasing overfitting [72-74]. The mostly male sample limits the generalizability of the digital phenotype findings to other PTSD populations (eg, civilians and women) and may have also contributed to the imbalance of groups on either side of the “alexithymia” cut-off, given that there is a small effect of sex on the TAS-20, with men generally scoring higher [75]. However, this higher proportion of men is representative of the defense force veteran sample used in this study [76]. Furthermore, this population is one in which alexithymia has a substantial impact on [77], which emphasizes the importance of improving our capacity to identify those who are alexithymic in this population.

Overall, this study suggests that facial, vocal, and language indicators could be used in the identification of veterans with PTSD who are experiencing alexithymia. We emphasize that the model requires further validation in independent samples, but the findings represent an important first step and attest to the merits of continued research in this area. Particularly considering the limitations of self-report measures of alexithymia, this paradigm has the potential to advance research paradigms and the assessment of alexithymia in clinical settings. These advances could ultimately contribute to alexithymia being more easily identified in psychiatric contexts, leading to the allocation of more tailored and effective treatment resources for addressing the specific challenges associated with alexithymia. Future research involving the implementation of this approach in clinical settings is required to examine its feasibility, efficiency, and ease of integration into clinical assessment, as well as its accuracy in identifying alexithymia. The improved identification of alexithymia in PTSD would be an important step in ameliorating the specific impacts that alexithymia has on the course and treatment of psychiatric conditions such as PTSD [12].