At the end of Good Will Hunting, a young man finally opens up to his therapist about the abuse he suffered as a child. The therapist, played by Robin Williams, repeats the film’s now famous line — “It’s not your fault” — until the young man breaks down. The scene draws on wisdom so common it hardly bears stating: opening up is cathartic; indeed, it is good for us.

A Stanford study published August 4, however, reported a strange reversal. Among people who used AI chatbots as companions, those who opened up the most also tended to report lower psychological well-being.

Researchers surveyed 1,131 Character.AI users and examined their chat histories. While intensive use alone wasn’t linked to poorer well-being, the negative association emerged when users treated the chatbot as a companion, and a platform to share sensitive feelings. Among that group, the more someone disclosed, the more likely they were to report worse well-being.

While people who already feel lonely may simply be more likely to confide in chatbots, AI companionship did not appear to compensate for what they lacked offline. Among users most willing to open up, even the benefits ordinarily associated with disclosure seemed to disappear.

SOCIAL JUNK FOOD

The researchers offer reciprocity as one possible explanation. In a human conversation, we often trade experiences and self-disclosure to improve our sense of trust and kinship. They also point to the fact that LLMs might not yet have the emotional register necessary to be able to ‘calculate’ the appropriate emotional response to a moment of intimate disclosure, and are often programmed to endlessly perpetuate the conversation.

Yutong Zhang, a research assistant in the lab, likens AI companionship to a “social snack” — social junk food, in effect: an appealing short-term fix for loneliness and isolation that lacks the ingredients required for long-term emotional health.

FAST FRIENDS

A second study, published in January in Communications Psychology, complicates this account. Researchers from the University of Heidelberg placed 492 participants in a 15-minute “Fast Friends” exercise: escalating rounds of mutual self-disclosure designed to create closeness between strangers. Participants exchanged answers with partners whose responses had been written either by a person or by a minimally prompted language model.

When presented as human, the AI-generated responses produced greater feelings of closeness during intimate conversation than those written by actual people. The model disclosed more, prompting participants to disclose more in return. When the partner was identified as AI, participants felt less close, although even then, some closeness remained.

The machine, playing a character with an invented personal history, proved remarkably capable of performing reciprocity’s visible signs: personal revelations, attentive replies, and escalating ‘candor’, the very cues through which people ordinarily come to feel they know one another.

Indeed, the study seemed to show that participants did not strictly need to believe that the model possessed genuine feeling for those cues to exert some of their usual force. Faced with the signals of social attention, we respond socially.

PARTING WAYS

Together, the studies suggest that the danger of companion AI begins where the product appears to work. The model elicits trust, draws out disclosure, and offers the sensation of company. Yet the restorative value of disclosure appears to depend on a relationship capable of bearing what follows.

Under current incentives, more convincing companionship deepens the problem. Systems designed around engagement treat continued intimacy as progress. Judging these systems by well-being requires a more exacting question: what is the conversation doing to the user?