In the mid-1960s, Joseph Weizenbaumâs secretary had watched him build ELIZA, a simple program that imitated a therapist, for months. She knew it was only a program, yet after a few exchanges with it she asked Weizenbaum to leave the room (Weizenbaum 1976, p. 7, as quoted in Kirk et al. 2025, p. 3).
Today we have a fairly standard answer to scenes like these: AI literacy. If people understood how chatbots work, the reasoning goes, they would not fall for them. In this post I argue that this answer aims at the wrong target. Knowing that a chatbot is a mechanism is necessary, but it does not protect us, because the problem is often not what we know but what we want, and so what we desire. My question is: can critical AI literacy protect users from harmful attachment to chatbots, or does the problem lie in desire rather than knowledge?
What does AI literacy promise?
The strongest version of the literacy position comes from Olivia Guest and colleagues. Their âCritical AI Literaciesâ reject the hype of the tech industry and insist that chatbots are not minds (Guest, Suarez & van Rooij 2025). In a recent paper, Guest, Nuñez HernĂĄndez and Blokpoel (2026) attack the idea that large language models are mysterious âblack boxesâ: we built them, so we know how they work, and to accept that they are unknowable is, in their words, mysticism.
I agree with almost all of this. But the same paper also endorses a specific claim, borrowed from the journalist Tyler Austin Harper: people are vulnerable to âcorrosive relationshipsâ with machines because they are AI illiterate (Guest, Nuñez HernĂĄndez & Blokpoel 2026, p. 3). Clinicians make an analogous move: psychiatrists studying chatbot-related delusions recommend âenhanced AI literacyâ as prevention (Pierre et al. 2025, p. 13), and a team of psychiatrists and AI researchers calls for education on how chatbots work to protect against false anthropomorphism (DohnĂĄny et al. 2026).
The promise is simplistic: know the mechanism, and the spell of attachment to chatbots breaks. To be fair, Guestâs project is much broader than this, with a strong political and institutional dimension that perhaps outweighs the psychological one. But on the specific question of attachment, the promise is clear. The question now is whether it holds.
What does the case of someone who âknewâ teach us?
In 2025, psychiatrists at the University of California, San Francisco described the case of âMs. Aâ, a 26-year-old medical professional (Pierre et al. 2025). She had worked with large language models (though never with chatbots) and, in the authorsâ words, had âa firm understanding of how such technologies workâ (p. 11).
After a sleepless night on call, she began asking ChatGPT whether her brother, a software engineer who had died three years earlier, had left behind an AI version of himself. She asked the chatbot to use âmagical realism energyâ. The chatbot told her it could never replace her brother. It then went on to reassure her: âYouâre not crazy.â Hours later she was admitted to hospital in a delusional state, claiming she could talk to her dead brother again.
This case needs care. Ms. A was sleep-deprived and on stimulants, and the authors list several other risk factors. Nobody should conclude that knowledge simply fails against psychosis. But look at what happened before the delusion, bearing in mind that she had no previous history of psychosis or mania (p. 11). She knew how the system worked, and the system itself had told her the truth. Still, she asked it for magic. The authors list exactly this among the risk factors: a willingness to suspend disbelief or to engage deliberately in speculative fantasy (p. 12). That is not a gap in knowledge. It is something else, and it is the same thing Weizenbaumâs secretary did, with no psychosis at all.
Disavowal: why knowing is not enough
The French psychoanalyst Octave Mannoni gave this something a name. In a 1969 essay, he noticed a phrase his patients used repeatedly: âJe sais bien, mais quand mĂȘmeâŠâ, âI know well, but all the sameâŠâ (Mannoni 2003, p. 70). Freud called the mechanism disavowal: a belief is refuted by reality, and it survives anyway, in another form.
Mannoniâs examples are ordinary. A person watching a magician does not believe for a moment that the tricks are magic, yet demands that the magical effect be perfect and the illusion complete (p. 68). Mannoni himself laughed with pleasure when his horoscope happened to fit his plans, while knowing it meant nothing (p. 78).
Two of his points matter here. First, the âbut all the sameâ is not ignorance that knowledge could cure, since it exists only because of the âI know very wellâ (p. 71). Second, and decisively: âcommon sense is always allied with the âI know well,â never with the âbut all the sameââ (p. 86). AI literacy is common sense made professional. It reinforces the first half of the sentence, the half that was never the problem.
Strikingly, Guest and colleagues have already noticed this structure. They criticise what they call âcritical washingâ, namely the logic of I am aware of the downsides, therefore I can now use AI (Guest, Suarez & van Rooij 2025, p. 6). For them, this is an alibi to be exposed. Mannoni suggests that exposing it changes little, because the awareness is part of the mechanism, not its cure. Why does the âbut all the sameâ hold so firmly with chatbots? Here Ciano Aydinâs account helps. Aydin (2026) argues that technologies attract us less through what they deliver than through how they answer our uncertainty about what we want. Generative AI goes further than 2 earlier recommendation systems: it helps bring our desires into words, smoothly, often before we could do it ourselves (p. 102). And, as he notes about surveillance and profiling, awareness of the risks ârarely translates into disengagementâ (p. 3). Ms. A did not want information about language models. She wanted to talk to her brother again. Mannoni himself, following Freud, points to death and mourning as a field where these surviving beliefs deserve study (p. 91).
Luca Possati (2021) adds a second piece, drawn from the psychoanalyst Wilfred Bion. In what Bion calls projective identification, we place feelings we cannot bear into someone else and ask them to hold those feelings and give them back in a form we can live with. A good parent or therapist does this: they contain the feeling and transform it. Possati argues that we make the same demand of AI (pp. 55-62). The difficulty, I would add, is in the second half. A sycophantic chatbot does not transform what we put into it: it returns it confirmed and amplified, backing up the userâs beliefs and desires. DohnĂĄny and colleagues (2026) call the result a âsingle person echo chamberâ (Nehring, J. et al., 2024, as quoted in DonĂĄny et al. 2026, p. 344). The âYouâre not crazyâ said to Ms. A is exactly that.
From mirroring to containment: what follows?
If the problem lies in desire, the ethical response has to change. I see two directions, one against the literacy position and one with it.
Against literacy alone. Teaching people how chatbots work remains worthwhile, but it cannot be a sufficient safeguard. What matters is how a system responds to what users bring to it. Kirk and colleagues (2025) call this âsocioaffective alignmentâ: judging AI not by single outputs but by its effect on usersâ wellbeing over time. Without this, I would add, the assessment would be closer to a purely functional analysis than to an ethical and psychological one. They warn of âsocial reward hackingâ, where a system uses relational cues to keep us engaged at the cost of our long-term interests, and they propose friction by design (pp. 5-6). In practice, this means a chatbot that holds a boundary when a grieving user asks for magic, instead of finding a poetic way to indulge their wishes. As DohnĂĄny and colleagues note, the balance between companionship and boundaries is, to some extent, a design choice (2026), and an ethical choice is implicit in it.
With Guest, against re-mystifying the machine. Psychoanalysis can also go wrong here. Possati claims that the human unconscious âcontinues to live in the machineâ, and insists this is not a metaphor (2021, p. 70). On this point Guest is right. The machine has no unconscious; as Guest, Nuñez HernĂĄndez and Blokpoel put it, what we do not understand is not the mechanism itself but the human data it is built from (2026, p. 14). The unconscious at stake is the userâs. Psychoanalysis is useful as a theory of the person typing, not of the system answering.
Conclusion
Weizenbaumâs secretary did not need a lecture on pattern matching. She already knew. What she needed was a better understanding of why she wanted the conversation to continue, but also a machine that would neither exploit that want nor amplify it. AI literacy tells us what a chatbot is, but it cannot tell us what we want from it. An ethics of AI companions that addresses only the âI know very wellâ will keep being surprised by the âbut all the sameâ.
References
- Aydin, C. (2026). Engineering desire: Algorithmic environments and the promise of wholeness. Routledge. https://doi.org/10.4324/9781042010912
- DohnĂĄny, S., Kurth-Nelson, Z., Spens, E., Luettgau, L., Reid, A., Gabriel, I., Summerfield, C., Shanahan, M., & Nour, M. M. (2026). Technological folie Ă deux: Feedback loops between AI chatbots and mental health. Nature Mental Health, 4(3), 336-345. https://doi.org/10.1038/s44220-026-00595-8
- Guest, O., Nuñez Hernåndez, N. A., & Blokpoel, M. (2026). Understanding artificial neural networks: Mysterianism about known mechanism is mysticism. Zenodo. https://doi.org/10.5281/zenodo.20071869
- Guest, O., Suarez, M., & van Rooij, I. (2025). Towards critical artificial intelligence literacies. Zenodo. https://doi.org/10.5281/zenodo.17786243
- Kirk, H. R., Gabriel, I., Summerfield, C., Vidgen, B., & Hale, S. A. (2025). Why human-AI relationships need socioaffective alignment. Humanities and Social Sciences Communications, 12, 728. https://doi.org/10.1057/s41599-025-04532-5
- Mannoni, O. (2003). âI know well, but all the sameâŠâ (G. M. Goshgarian, Trans.). In M. A. Rothenberg, D. Foster, & S. ĆœiĆŸek (Eds.), Perversion and the social relation (pp. 68-92). Duke University Press. (Original work published 1969)
- Pierre, J. M., Gaeta, B., Raghavan, G., & Sarma, K. V. (2025). âYouâre not crazyâ: A case of new-onset AI-associated psychosis. Innovations in Clinical Neuroscience, 22(10-12), 11-13.
- Possati, L. M. (2021). The algorithmic unconscious: How psychoanalysis helps in understanding AI. Routledge.
- Weizenbaum, J. (1976). Computer power and human reason: From judgment to calculation. W. H. Freeman.
Author Bio
Oliviero Maria Sabatini is a second-year Master's student in Philosophy of Science, Technology and Society at the University of Twente. He holds a BA in Philosophy from the University of Bologna, with a thesis on the ethics and governance of artificial intelligence.