Participants were recruited via social media advertising, through an undergraduate psychology course, Prolific, flyers, and word-of-mouth. At baseline, 190 participants (aged 18‐24 years) residing in Australia, the United Kingdom, or the United States were included. About 67% (n=130) of respondents were female, 26% (n=50) were male, and 5% (n=10) had other gender identifications, with moderate ethnic diversity: 52% (n=98) Asian; 28% (n=53) White; 19% (n=37) other ethnicities; and 1% (n=2) preferred not to say. The sample had limited socioeconomic diversity, with 70% (n=133) being highly educated, and 73% (n=140) of respondents perceiving themselves as fairly, rather, or very well-off (Table S1 in Multimedia Appendix 1). Of the 190 participants who completed baseline, 89% (n=169) then completed the week-long EMA and 48% (n=91) completed the 1-month follow-up (Figure S1 in Multimedia Appendix 1). Attrition analyses revealed no significant differences between completers and noncompleters on baseline anxiety, depression, social sensitivity, or screen time variables (Table S2 in Multimedia Appendix 1).

Prolific participants were compensated £9 (GBP £1=US $1.33 as of September 26, 2024) per hour. Undergraduate students were awarded course credits for completion of the baseline assessment and EMA. All other participants were compensated with a Prezee or Amazon voucher in their local currency at US $13.30 per hour for all parts, as were the undergraduate students for the 1-month follow-up. For nonstudent participants, a bonus payment of US $6.50 was awarded if participants completed 80% of the EMA surveys within 1 hour of the notification.

Ethics approval for this study was granted by the UNSW Human Research Ethics Advisory Panel (HREAP6389).

Participants were provided with comprehensive participant information, including the opportunity to opt out for any reason at any time, and provided consent online via Qualtrics before responding to part 1 questionnaires. Participant privacy was managed according to the UNSW Privacy Management Plan, a copy of which was made accessible to participants. Before analysis, all identifying data were removed and replaced with a randomly generated code. Identifying information was kept separately for the sole purpose of reidentifying participants wishing to withdraw from the study.

After providing informed consent, participants completed the baseline survey measuring demographic characteristics, along with mental health symptoms and individual differences, followed by 7 days of EMA surveys. EMA survey links were sent to Prolific participants via email, and all other participants received prompts via SMS. The baseline survey (with the exception of demographics) was completed again after 1 month. At the end of the baseline assessment and the EMA survey period, participants also uploaded screenshots of the total average daily time spent on their device, as well as weekly time spent on each app. All surveys and uploads were completed in Qualtrics.

All analyses were conducted in R (version 4.4.2).

All linear mixed models included random intercepts for participant ID; these are not noted again in this section. Standardized predictors were used in all regression-based models. Outlier analysis using the IQR method (1.5 × IQR below Q1 or above Q3) indicated a small number of outliers in responses by connection type (range: 0‐3 per group), consistent with natural variation in a community sample. Outliers were therefore not removed. Repeating primary analyses excluding these outliers did not change the pattern of results.

Total participant device (H1a) and online social interaction time (H1b) were included as fixed factors in separate linear regression models, with baseline anxiety and depression scores as outcomes. These relationships were also investigated longitudinally across the 1-month follow-up period using linear mixed models with timepoint as within-subjects fixed effects and time spent on devices and online social interaction as between-subjects fixed effects.

Mixed models including EMA timepoint (1-21) were all run twice: once with EMA timepoint as an independent within-subjects fixed effect and once with EMA timepoint allowed to interact with the other fixed effects in the model. Model fit indices were then compared, and if formal comparison yielded a significant difference between models, the better-fitting (lower Akaike information criterion by at least two points) model was chosen. If the model fit was not significantly different between models, the more parsimonious (without interaction term) model was selected.

To test H2, EMA timepoint and connection type (in-person vs online) were included as within-subject effects. To examine the association between anxiety and depression symptoms and responses to social interaction (H3a), linear mixed models including EMA timepoint as within-subjects effects and symptoms of anxiety and depression as between-subjects effects were built. Connection type was then added to the H3a model as a within-subjects fixed effect to test whether the effect of mental health symptoms differed between online and in-person social interactions (H3b). Intraclass correlation coefficients (ICCs) for these models ranged from 0.26 to 0.41 across anxiety and depression outcomes, indicating a high level of within-person variability and within accepted thresholds for mixed modeling [66,67]. To examine these associations (H3) over time, mean affective and affiliative response scores collected across the 7-days of EMA surveys were calculated for each participant, scaled, and then included as a fixed between-subjects effect in a model including timepoint (baseline vs 1-month) as a fixed within-subjects effect to predict symptoms of anxiety. ICCs for these longitudinal models ranged from 0.62 to 0.67 across anxiety and depression [66,67]. Finally, to assess whether the associations between mean responses to social interactions and mental health outcomes were moderated by baseline social sensitivity (H3), both cross-sectionally (between participants at each timepoint) and longitudinally (within participants between timepoint 1 and timepoint 2), linear regression and linear mixed models were used, respectively.

Effect sizes for all models were reported as partial eta-squared (η²_p), which represents the proportion of variance explained by the relevant variable, and is therefore equal to eta-squared for linear regression models. Eta-squared of .01, .06, and .14 are considered small, medium, and large effect sizes, respectively [68]. All significant interaction effects were examined with simple slope analyses.

For all EMA data analyses (H2, H3a/b), restricted maximum likelihood (REML) estimation was used [69], thereby using all available social interaction responses contributed by each participant, regardless of how many social interactions they reported. This approach provides unbiased parameter estimates under the assumption that data are missing at random [70]. Responses from prompts where no social interaction was reported were not included in the present analyses. The analyses including baseline and 1-month as timepoints applied listwise deletion, thereby only including participants for whom there was data at both timepoints to ensure meaningful analysis.

Usage time predictors (device time, social media time, texting time, and combined social interaction time) were specified prior to analysis to enable direct comparison with previous literature that has differentially operationalized screen and social media time exposure. Moderation analyses were similarly theory-driven, based on a priori hypotheses derived from the “differential susceptibility to media effects” model and prior research. No additional correction for these predictors was therefore applied. For the two outcomes of interest (anxiety and depression symptoms), the α threshold was set to .025, applying a Bonferroni correction such that an overall family-wise error rate of .05 was maintained across both outcomes, with tests set as 2-tailed. Post hoc analyses were completed via pairwise comparisons with Tukey adjustment to maintain an overall α threshold of .05.

Sensitivity analyses were conducted for all primary analyses, restricting the sample to participants who completed at least 50% of EMA surveys (≥7 of 21 surveys; 112 participants). Effects were consistent in direction and significance across the full and 50% minimum completion samples, indicating the moderate compliance rate did not systematically bias the results (Tables S4-S6 in Multimedia Appendix 1).

Overall, 169 participants completed at least 1 EMA, resulting in 2306 EMA survey responses. Of the 21 EMA surveys, participants completed 13.6 (SD 6.9) on average. Average compliance, therefore, was 65% (SD 33%; range 5%‐100%), with a good average response time (within 37.5 minutes of the prompt). Neither symptoms of anxiety (β=0, t₁₆₇=0.03, P=.98) nor depression (β=0, t₁₆₇=0.36, P=.72) were significantly associated with EMA compliance rate. Attrition analyses comparing completers and noncompleters of the 1-month follow-up found that completers demonstrated significantly higher EMA compliance than noncompleters (76% vs 54%; t₁₅₁=−4.69, P<.001; Table S2 in Multimedia Appendix 1). Reactivity was minimal with no significant change in participant time spent on either their smartphone (t₉₄=−0.39, 95% CI −27.51 to 18.48; P=.70) or online social interaction (t₇₃=−1.04, 95% CI −26.79 to 8.40; P=.30) from baseline to after the EMA.

Participants spent an average of 6 hours and 23 minutes per day on their phones (SD=3 hours and 11 minutes), of which 2 hours and 32 minutes (SD=1 hour and 49 minutes) was spent using online social interaction apps. There was no significant association between time spent on smartphones and symptoms of anxiety and depression at baseline (Table 1) or at the 1-month follow-up (Table 2). However, at baseline, average daily time spent on online social interaction apps was associated with both higher anxiety and depressive symptoms (Table 1).

Across the 1-month follow-up period, there were no significant changes in anxiety (F_1,104=2.06, P=.15, η²_p=.02) or depression (F_1,109=0.29, P=.59, η²_p=.00). The small association between online social interaction time and anxiety remained significant; however, there were no significant associations between online social interaction time and symptoms of depression across the 1-month follow-up period (Table 2). There were no notable differences in the relationship between social media time and texting time, and mental health symptoms (Tables S7 and S8 in Multimedia Appendix 1).

Of the 1598 EMAs where participants had interacted socially in the prior 2 hours, 67% (n=1074) of their most important social interactions were in-person, while 20% (n=323) of their most important interactions were online via social media or texting apps (Table S9 in Multimedia Appendix 1).

For all analyses including the EMA timepoint, the model adding the EMA timepoint as a fixed effect that did not interact with other within- and between-subjects effects showed the best fit (Table 3).

The model H2 showed no main effect of EMA timepoint (affective: F_20,1265=1.48, P=.08, η²_p=.02; affiliative: F_20,1251=1.54, P=.06, η²_p=.02) but affective (F_1,1353=29.01, P<.001, η²_p=.02) and affiliative (F_1,1321=23.35, P<.001, η²_p=.02) responses varied by connection type (Figure 1). Specifically, affective responses to online social interactions were consistently less positive than responses to in-person social interactions (β=−6.51, 95% CI −4.14 to −8.89). The same pattern emerged for affiliative responses (β=–5.51, 95% CI −3.27 to −7.75), with individuals feeling less included following online relative to in-person interactions across EMA timepoints. The direction of the relationship between responses and both social media and texting was consistent (Table S10 in Multimedia Appendix 1).

Models H3a predicting affective responses to social interactions showed a significant main effect of symptoms of anxiety (F_1,151=5.42, P=.02, η²_p=.03) but not depression (F_1,151=4.71, P=.03, η²_p=.03). Greater symptoms of anxiety (β=−2.33, 95% CI −4.29 to −0.37) were associated with more negative affective responses to social interactions. Adding connection type to the model showed that the main effect of connection type remained significant when accounting for mental health, but mental health symptoms did not interact with connection type (Table 4). That is, affective responses to online social interactions were less positive compared to responses to in-person interactions irrespective of individuals’ symptoms of anxiety or depression (Table S11 in Multimedia Appendix 1). Baseline mental health symptoms were not associated with how included individuals felt in response to social interactions (anxiety: F_1,157=3.05, P=.08, η²_p=.02; depression: F_1,157=2.25, P=.14, η²_p=.01). Adding connection type to the model showed no significant interaction between mental health symptoms and connection type (Table 4).

Across time, from baseline to 1-month, average affective and affiliative responses to in-person social interactions were inversely associated with depressive symptoms (Table 5), with the effect remaining stable across time (ie, no response × time interaction). Online social interactions were not significantly associated with mental health symptoms across 1-month (Table 5). This was consistent for both social media and texting interactions (Table S12 in Multimedia Appendix 1)

While higher social sensitivity, as measured at baseline, was associated with higher levels of anxiety and depression at both baseline and 1-month follow-up, social sensitivity did not moderate the relationship between individuals’ average affective or affiliative responses to online or in-person social interactions and mental health, at either timepoint (Tables S13 and S14 in Multimedia Appendix 1). However, social media interactions (without texting) showed a different pattern of association, and as they deviate from the “online interaction findings,” they are reported in the main manuscript. Social sensitivity moderated the association between baseline depressive symptoms and both affective (F_1,39=6.84, P=.01, η²_p=.15) and affiliative (F_{1, 39}=6.77, P=.01, η²_p=.15) responses to social media interactions (Table 6). Post hoc simple slopes analyses indicated that this relationship was only significant for individuals with relatively high levels of social sensitivity (+1SD), such that more positive affective (β=2.65, t₃₉=2.66, P=.01, 95% CI 0.63-4.66) and affiliative (β=2.59, t₃₉=2.52, P=.02, 95% CI 0.51-4.60) responses to social media interactions were associated with greater depressive symptoms (Figure 2). However, this interaction varied by timepoint, such that the significant moderating effect of social sensitivity on the association between affective and affiliative responses and depression was not maintained at the 1-month follow-up (Table S14 in Multimedia Appendix 1).

Rising rates of youth anxiety and depression have coincided with increased smartphone use and opportunities for online social interaction, fueling a popular belief that online social interaction is inherently harmful. However, meta-analyses have yielded mixed findings [12], highlighting the need for identifying individual differences in susceptibility to the effects of online social interaction. As hypothesized, and consistent with the existing literature [16,17,71,72], there was no significant relationship between symptoms of anxiety and depression and objectively measured smartphone time, and only small associations with online social interaction time. However, participants felt less positive and included after online social interactions than in-person interactions. Interestingly, baseline anxiety symptoms were negatively associated with how happy someone felt after social interactions, but there was no significant relationship with depression, nor between feelings of inclusion and anxiety or depression. Further, over the month of the study, while feeling more positive and included after in-person social interactions was associated with lower depression symptoms, responses to online social interactions were not associated with mental health, and these associations were not moderated by social sensitivity. However, social media differed from the overall online interactions here. Notably, individuals higher in social sensitivity showed a positive association between depressive symptoms at baseline and more positive and stronger affiliative responses to online social interactions. This moderating effect of social sensitivity was not maintained at 1-month. Together, these findings highlight the need for elucidating dispositional and developmental sensitivity to online interactions to develop a more nuanced understanding of their associations with mental health.

Interestingly, given reports of many 13‐19-year-old adolescents using social media “almost constantly” [71,72], across the week of EMA, these 18‐24-year-old participants disproportionately chose in-person interactions as their “most important” recent interaction. Consistent with previous EMA studies [30,32,73], participants reported feeling more positive and included during in-person, compared to online, social interactions. Feeling less positive and less included after online social interactions compared to in-person interactions may have been due to temporal asynchrony (eg, delayed replies) and impoverished social cues that lead to a perception of reduced thoughtfulness and empathy [25,42,74,75]. Alternatively, online social interactions may elicit less positive feelings than in-person interactions because they can be used for less emotionally charged purposes, such as exchanging information or arranging meetings [76] via text. Richer contextual cues, as well as differing affordances and motivations for in-person compared to online interactions, may therefore make offline connections more meaningful. Future research would benefit from directly comparing the mechanisms through which each modality affects mood and feelings of inclusion. For example, examining whether synchronicity, audience closeness and size, or social evaluation features differentially predict affective and affiliative responses to online interactions across modalities.

The finding that anxiety symptoms were associated with participants feeling less happy after social interactions is consistent with the high correlation between adolescent anxiety and negative interpretation bias in social situations [49], which has been found to increase with adolescent age [77]. It is also consistent with recent meta-analyses of prospective studies, which found that anxiety in adolescents across the age spectrum cultivates lower levels of companionship, intimacy, and support in relationships [78,79], arguably the factors that make social interactions a positive experience. The lack of association with symptoms of depression may be a power issue, with the effect size and direction of the association being the same as for anxiety.

In contrast, the positive association between mood and feelings of inclusion following in-person social interactions and mental health is consistent with evidence that emerging adults’ ability to meet their needs for autonomy, relatedness, and competence is optimized by engaging in mostly offline social interaction [80]. While previous studies have found there can be enhancement effects when online social interaction complements offline interaction [25], these findings suggest that if online social interactions displace in-person social interactions, young people may feel relatively less happy and included more often. Future research is needed to examine whether online interactions are indeed displacing in-person interactions. The introduction of the social media age restrictions in Australia offers an interesting opportunity to test the displacement hypothesis by comparing online versus offline social interaction frequency in adolescents in a country with government-restricted access, to online versus offline interaction ratios in adolescents from other countries.

In line with the drive toward identifying individual differences that account for the mental health impacts of online interactions, this study examined the effects of social sensitivity, which was associated with anxiety and depressive symptoms. Counterintuitively, individuals high in social sensitivity and depressive symptoms responded more positively to social media interactions. Importantly, given the cross-sectional nature of this association, the risk of reverse causality cannot be eliminated, and several interpretations of this finding warrant consideration. First, more positive responses to online social interactions for those high in social sensitivity may reflect social avoidance, where participants who are especially high in social sensitivity are more likely to avoid in-person interactions during which they fear getting rejected [81]. This is consistent with compensatory online use, where socially anxious individuals perceive online interactions more positively, as a format offering greater control [82]. Alternatively, this finding is also consistent with the social risk hypothesis [81], where people with mild to moderate depression are motivated by a fear of social rejection to seek safe forms of social contact in order to restore their social value. Social media may offer a means to carefully curate an interaction to minimize the risk of social rejection, thus resulting in greater momentary positive affect and feeling included. However, in the longer term, these interactions may be unfulfilling [83], drive lower feelings of self-worth via pressure to maintain social relevance [84], and/or promote negative self-comparisons [85]. Equally, it may be that those who are socially sensitive and experiencing anhedonia due to depression find the energy requirements of in-person social interactions excessive, but are able to selectively engage with, and respond more positively to, online social interactions.

In contrast with this argument is the finding that the moderating effect of social sensitivity was not maintained at the 1-month follow-up. The positive affect experienced during online interactions may have offered support for their mental health challenges over the month-long period [86,87], though this hypothesis will need to be explicitly tested through future research.

Additionally, effect sizes for all observed relationships in this study were small, emphasizing the multifactorial nature of influences on adolescent mental health and highlighting that understanding individual differences in susceptibility to the pros and cons of online social interactions is a promising avenue for studying its mental health impacts. Further, small effects may still have meaningful implications when considered at the population level, especially among younger adolescents who are arguably more vulnerable to the effects of social media [88], and given the ubiquity of social media use by young people. Therefore, even modest effects may indicate opportunities for intervention.

These results should be considered within the context of the study’s strengths and limitations. The study’s main strengths were (1) high-frequency assessment in the real-world, minimizing recall bias [30] (ie, remembering the prior 2 hours vs 2 weeks); (2) testing of theory-driven individual differences in susceptibility to social media effects; and (3) using an objective assessment of time spent on smartphones and social media apps (screenshot). However, the observational study design precludes causal inferences about the role of online versus in-person social interaction on mental health.

The study findings should be interpreted in the context of our specific sample of older adolescents and emerging adults aged 18‐24 years, who volunteered to participate, and 73% (n=140) of whom described themselves as fairly to very well-off. This means these results may not generalize to younger, less wealthy adolescents, or those unlikely to volunteer, for whom responses, mental health, and developmental differences in emotional regulation and social sensitivity may differ. Equally, it is possible that those who completed all parts of the study were more motivated, conscientious, or digitally engaged than noncompleters, creating systematic bias toward those with these personality traits and interests [89]. Taken together, this limits the generalizability of findings. Further, while 98% of social media users connect via smartphones [90], online social interaction time may still have been underestimated for some participants by only capturing time on smartphones, as opposed to other digital devices. This is especially the case for Android users, for whom detailed app usage data was less available. Further, using participant-uploaded screenshots may have limited data collection as inaccurate screenshots could not be used, and in the era of AI, screenshots could have been subject to manipulation. However, this is unlikely as there was no incentive for individuals to alter screenshots provided, and there was a disincentive given it would be time-consuming to do so. Smartphone sensing and digital phenotyping are more reliable means to capture time spent online in future studies.

While the novel EMA used in this study enabled a unique view of adolescents’ social experiences while minimizing recall bias [30], EMA compliance of 65% was less than the 79% average found in a recent meta-analysis of EMA studies [59]. However, overall attrition at follow-up was also substantial (48%), and once noncompleters were removed, EMA compliance was 76%, with sensitivity analyses indicating no change in the direction or significance of results. Additionally, EMA compliance rates have been found to be substantially higher in older, compared to younger, adults [91], and therefore, there may not have been as much of a difference in compliance in this study compared with other studies also exclusively involving emerging adults. Regardless, identifying ways to increase young people’s compliance would optimize the analytical power and efficiency of future studies.

Finally, while the 1-month follow-up period was sufficient to examine short-term longitudinal associations, future research should use extended follow-up periods to enable the detection of longer-term effects of online social interaction on mental health trajectories.

Overall, this study supports the literature to date indicating that time spent on smartphones and online social interaction is only weakly correlated with older adolescent mental health. It also extends evidence of online social interactions as mostly positive experiences to older adolescents and emerging adults; albeit, these experiences may not make adolescents feel as happy or as included relative to in-person interactions. To better understand what makes online social interaction harmful for some young people and not others, this study offers an EMA blueprint for evaluating adolescents’ momentary experiences of online and offline social interactions, as well as individual and developmental susceptibilities to online social interaction effects. It also provides preliminary evidence of the potential moderating role of social sensitivity on the relationship between adolescents’ responses to social interactions and mental health outcomes. Further research across the adolescent age spectrum is needed to understand the individual characteristics that influence adolescents’ vulnerability to both positive and negative online versus offline social interactions.