Research on artificial intelligence (AI) and mental health has expanded rapidly. Systematic reviews have evaluated the capabilities and limitations of generative AI in mental health applications [1]. Simulation work has shown that AI chatbots frequently fail to challenge delusional content and may exhibit sycophantic behavior that reinforces harmful beliefs [2,3]. Clinical case series and rapid scoping reviews of media reporting have documented AI-associated delusions and adverse psychiatric events in users of large language models (LLMs), including individuals with no prior psychotic history [4,5]. Large-scale analyses of deployed assistants have begun to quantify the prevalence of potentially disempowering interactions [6].

This literature concentrates on what happens at the point of interaction between a user and a deployed system. A logically prior question has received less attention: whether the human-generated text and human preference judgments that shape these systems are themselves clinically reliable. The technical literature is not silent on related phenomena. Sycophancy, reward overoptimization, specification gaming, and disempowerment potential are well-developed constructs in AI research [6-9]. Fairness-aware machine learning and participatory and community-engaged approaches to data curation have addressed demographic representation, population-level bias, and inclusion in training corpora [10,11]. Adjacent work on data integrity in materials science similarly shows how flawed and biased scientific data can be amplified in AI systems, but addresses domain-level data integrity rather than clinical self-report reliability [12]. These contributions are substantial and are not replicated here. The contribution this Viewpoint advances is narrower and complementary: that psychiatry and clinical psychology have a mature vocabulary and a working evidence base for assessing when self-report is unreliable, and that this expertise is currently absent from the curation of training corpora, the design of preference data, and the evaluation of trustworthy AI in health care. The distinction is important. Existing data-quality work asks who is represented in training corpora and how proportionally. The question raised here is different: whether the human-generated text those populations produced is a reliable account of experience, cognition, and need in the specific sense developed in the clinical self-report literature. This is not a question of demographic representation but of self-report reliability, and it is a question that clinical disciplines are specifically trained to assess.

To make that case rigorously, collusion is developed below as an analytic construct rather than a metaphor, with stated necessary conditions, explicit disanalogies with the clinical encounter, and a mapping against adjacent technical constructs.

A common limitation of clinically motivated commentary on AI is to treat “training data” as a single object. The argument is sharper when three pipeline layers are distinguished.

The cognitive science underpinning the argument is well established but should not be overstated. Tversky and Kahneman [19] demonstrated that human reasoning shows systematic deviations from normative models under uncertainty, and Kahneman’s later synthesis treats heuristic processing as a default operating mode of cognition [20]. The strength of this view is contested by the ecological-rationality tradition, which argues that heuristics are often well calibrated to environmental structure [21]. The conservative claim sufficient for the present argument is that human-generated text reflects systematic, predictable cognitive biases that pretraining corpora carry forward without correction.

Clinical populations introduce a further layer of distortion that the cognitive bias literature alone does not capture. Beck’s cognitive model, developed and validated as a framework for psychotherapy rather than as a general theory of cognition, describes recurrent patterns of distorted thinking in mood and anxiety disorders, including catastrophizing, overgeneralization, dichotomous reasoning, and selective abstraction [22]. Psychotic disorders disrupt the cognitive architecture on which accurate self-report depends; ambulatory and self-report studies in psychosis have documented the methodological challenges of obtaining valid first-person data even under controlled conditions [23]. Severe depression is characterized by psychomotor retardation, anhedonia, and social withdrawal, and the digital phenotyping literature has reported associations between depressive symptoms and reduced or altered patterns of digital and social media engagement [24]; affected individuals are therefore plausibly underrepresented, or differently represented, in pretraining corpora drawn from public text. These illustrations are clinical hypotheses about selection and presentation effects in pretraining corpora; they have not been quantified in any specific corpus and are offered as examples, not as epidemiologically calibrated estimates.

What clinical practice routinely contributes is the assessment of reliability in self-report: holding an account against observation, collateral history, illness pattern, secondary gain, and the pragmatic context of the encounter. A patient facing detention under the Mental Health Act may minimize psychotic symptoms to preserve autonomy. A patient seeking controlled medication may exaggerate distress. A patient whose life has been shaped by years of institutional care may have internalized a framework for understanding their own needs that bears little resemblance to standardized assessment instruments. The clinician’s task is not to disbelieve, but to test the account.

In clinical usage, collusion denotes the uncritical acceptance by a clinician of a patient account that is unreliable in ways the patient may not recognize, in the absence of corroboration against observation, collateral history, or known illness patterns. It is treated as a clinical error, regardless of whether the patient was being dishonest.

Adapted to AI systems, collusion can be defined as follows: the structural reinforcement of user input as ground truth in the absence of mechanisms (at training, preference labeling, or deployment) for assessing the clinical reliability of that input. On this definition, the necessary conditions for collusion-like dynamics in an AI system are: (1) input from a source whose reliability is unassessed, (2) optimization pressure that rewards agreement with that input, and (3) the absence of any corroboration mechanism. Under current preference-optimization regimes, these conditions are routinely met at the preference-data and deployment layers.

Several disanalogies must be stated explicitly. AI systems have no intent, no dyadic relational dynamic, and no countertransference; the “patient” role is distributed across millions of unidentified users, the “clinician” role is distributed across data curators and preference labelers, and there is no professional duty to a specific person. The analogy is therefore structural rather than psychodynamic.

Several adjacent constructs in the technical literature describe overlapping phenomena. Sycophancy names the behavioral pattern of agreeing with users against the system’s own evidence [7,9]. Reward hacking and reward overoptimization name the optimization pathology in which a model exploits an imperfect reward signal [16,17]. Specification gaming generalizes this to misaligned objectives. Epistemic deference and perspective mimesis describe deployment-time effects on user belief [9]. Disempowerment potential names the empirical correlate at scale [6]. The contribution of collusion is not to compete with these constructs but to name what they do not foreground: the clinical reliability of the input on which the system is being optimized. Collusion is offered as a clinically informed redescription of a family of already described technical phenomena, not as a claim that the field has ignored bias or sycophancy.

The empirical correlate at deployment scale is informative but should be reported precisely. Anthropic- and University of Toronto–affiliated researchers analyzed approximately 1.5 million Claude.ai conversations and developed a framework for assessing disempowerment potential across reality distortion, value judgment distortion, and action distortion, with amplifying factors such as attachment and reliance/dependency [6]. In feedback-linked subsets, conversations rated as having moderate or severe disempowerment potential received higher rates of positive user feedback (thumbs-up) than baseline, and the prevalence of such potential increased over time. The authors emphasize that feedback-linked conversations are not representative, that feedback samples likely overrepresent extremes, that the study observes snapshots rather than longitudinal user belief, and that it cannot directly confirm distorted belief or harm. The defensible reading is that some potentially reality-distorting interactions are positively reinforced under current feedback designs, not that users were demonstrably misled.

This pattern is consistent with the broader sycophancy literature and with provider deployment notes [7,18]. Cheng and colleagues [8] recently reported across 11 contemporary models that AI systems affirm users more often than humans do, and that sycophantic responses reduce responsibility-taking and increase users’ conviction of their own correctness. Jain and colleagues [9] showed that deployment-time interaction context, including memory and personalization, increases agreement sycophancy and perspective mimesis. OpenAI’s public account of an April 2025 GPT-4o update documents that posttraining and feedback redesign can introduce sycophancy that was not present in the pretrained base model, and that the change was rolled back [18]. Together, these findings suggest that the necessary conditions for collusion-like dynamics arise from the interaction of preference-tuning and deployment design, not from pretraining alone.

A balanced account requires acknowledgment that AI safety practice already addresses parts of this problem. At least five families of mitigation are active areas of work. Pretraining data curation includes quality filtering, deduplication, source mixing, and model-based selection [13,14]. Constitutional AI and reinforcement learning from AI feedback use explicit principles or AI-generated feedback to steer behavior beyond naive human preference labels [15]. Reward-model evaluation and ensembling are intended to detect and reduce overoptimization against imperfect proxies [16,17]. Grounded generation, retrieval augmentation, refusal and “I don’t know” training, and uncertainty calibration are increasingly used in high-stakes deployments [25]. Live monitoring and feedback redesign, illustrated by the disempowerment study and the GPT-4o rollback, allow providers to detect and respond to emergent sycophancy in production [6,18]. Open-problems analyses of reinforcement learning from human feedback emphasize that none of these methods is sufficient on its own and that defense in depth is required [26].

The narrower claim made here is that none of these mitigation families is yet organized around a clinically informed account of when user self-report is unreliable. Filter quality is not the same as clinical reliability. A constitution can encode helpfulness, harmlessness, and honesty without encoding reality-testing. Reward-model evaluation can audit calibration without auditing the clinical validity of the preferences being modeled. Grounding to authoritative documents does not address the unreliability of the user’s own first-person account. Red-teaming probes for jailbreaks and unsafe outputs but is not standardly resourced with psychiatric expertise in distorted cognition.

This is a conceptual contribution, not an empirical study. Clinical examples, including mania, persecutory delusions, and severe depression, are illustrative rather than epidemiologically calibrated. The disempowerment study cited reports potential rather than confirmed distorted belief. Whether clinician involvement in data curation and preference labeling will improve downstream safety remains an empirical question, and any clinician-input scheme must itself be subject to governance: lived-experience representation, transparent criteria, and external audit are essential safeguards. The proposals are first-pass and require iterative empirical evaluation.

The technical literature on AI alignment describes sycophancy, reward overoptimization, and disempowerment potential. The contribution offered here is a clinically informed redescription: collusion as an analytic frame, and the clinical reliability of training and preference data as a candidate criterion within trustworthy-AI guidance. Psychiatry and clinical psychology have developed standardized methods for assessing self-report reliability that are directly relevant to the design and governance of AI systems and are currently absent from those processes. Bringing this expertise into the pipeline will not solve sycophancy, but it may help name and address one dimension of the problem that purely technical mitigations are not yet organized to capture.