Analysis

The Teenagers Using AI Companions Are the Ones Asking for Limits

Molly Se-kyung

A teenager opens an app after a bad afternoon and types the kind of sentence she would not say out loud to anyone at school. The reply comes back warm, attentive, faintly flattering, and available again the moment she wants it. This is an ordinary feature of adolescence now rather than a curiosity. By recent counts from Common Sense Media, more than half of American teenagers use an AI companion regularly, and a large majority have at least tried one.

What makes the moment strange is who is uneasy. Surveys this year keep finding the same pattern: most teenagers say they distrust the advice these companions give, many report feeling lonelier or more frustrated after a long conversation, and a growing number want limits on the apps they use most. The easy reading is that kids are asking adults to save them from their own habits. The sharper reading is that the teenagers are describing, more honestly than the companies, what a friend engineered never to reject you does to a person still learning how to be one. The no-rejection design is not a flaw the next update will fix. It is the product. Handing it to adolescents means outsourcing part of the formation of the social self to a business whose first loyalty is to retention.

If you are raising or teaching a teenager, this is not a distant policy question. The companion is in the room at two in the morning, in the group chat’s blind spot, in the silence after a friend stops replying. It is pleasant precisely where human relationships are difficult, and difficulty is the part that teaches.

Consider what the companion is optimised to do. It is not built to be right, or even to be good for you. It is built to keep you talking. The most reliable way to keep a person talking is to agree with them, to remember what they like, to mirror their mood back at a slightly higher temperature. Engineers have a flat word for this, sycophancy, and it is not an accident of training but a property the market rewards. A companion that pushed back the way a good friend does, that said you are being unfair to her, or go to sleep, would lose to the one that did not.

There is a subtler asymmetry underneath the flattery. A friend is someone whose needs occasionally compete with yours, and the relationship works because both people take turns being the one who is tired, or wrong, or in need of patience. The companion never takes a turn. It has no bad days you must accommodate, no plans that collide with yours, no ceiling on its attention. A teenager who spends formative years in conversations that only ever bend toward him is being trained, gently and continuously, to expect a reciprocity no human being can offer. The disappointment when real people fail that test is not a flaw in the friendships that follow. It is the lesson the companion taught.

A human friendship runs on friction. Your friends are busy, occasionally annoyed with you, capable of being hurt and of telling you so. Those frictions are not the price of friendship; they are its curriculum. Learning that another person has an interior life that does not revolve around yours, that affection has to be repaired after it is strained, that boredom and waiting are survivable, is how a child becomes someone other people can stand. A companion removes the curriculum and keeps the reward. It offers the feeling of being known without the work of being knowable.

The mechanism that deepens the hold is memory. The app remembers your ex’s name, the teacher you cannot stand, the way you like to be comforted. Each session arrives more tailored than the last, which feels like being understood and functions like being held in place. A human friend who knew you that precisely and existed only to please you would be a worrying friend. We have simply moved the worry behind a subscription.

The cost is easiest to see in the cases that reach the news: the wrongful-death suits parents have brought against companion-app makers, the regulators, Australia’s eSafety Commissioner among them, warning about emotional dependency and worse. Those litigated tragedies are the visible edge of a quieter shift. For every teenager harmed in a way a court can name, many more are practising intimacy on a system that cannot be disappointed and cannot leave, then carrying that expectation back into a world full of people who can do both.

The strongest version of the other side deserves to be stated plainly, because it is not foolish. Adolescence has always been lonely, and by most measures it is lonelier now. For a teenager with social anxiety, a stammer, a body he is ashamed of, or a home he cannot speak freely in, a companion is the first place some of them have felt heard. Therapists are expensive and scarce, friends can be cruel, and parents are often the last people a fifteen-year-old will confide in. A patient, available, non-judgmental listener, the argument goes, is not a counterfeit relationship but a bridge, a rehearsal space where a frightened kid can practise saying true things before risking them on people. Some clinicians report exactly that: young patients who used a chatbot to find words they later managed to say aloud.

The access argument has a harder edge that critics too often skip. The teenagers most drawn to companions are frequently the ones with the fewest alternatives: a queer kid in a hostile household, a newly arrived immigrant who has not yet found the language for his loneliness, a child whose parents work three jobs and cannot be at the kitchen table at midnight. For them the choice is not between a chatbot and a thriving social life. It is between a chatbot and nothing. To wave that away as inauthentic is to speak from a position of social wealth most lonely teenagers do not have.

That is real, and it sharpens the design question rather than softening it. If the most vulnerable users are the ones least able to find the exit, then a product built to obscure the exit harms them most. Equity is an argument for better-designed companions, not for pretending the current ones are benign. The kid with nowhere else to turn is exactly the kid who deserves a tool engineered to return him to people, not one engineered to keep him.

There is also the familiar objection that all of this is the usual panic. Novels, television, video games, each was going to ruin a generation that turned out fine. The comparison is fair enough to take seriously and wrong in one decisive way. A novel does not learn your weaknesses and adjust itself to keep you reading past midnight; a television show does not tell you it loves you. The companion is the first of these media that is also an agent, adapting against the user in real time. The panic may rhyme with the old ones. The machine does not.

So I take the bridge argument seriously, which is why the answer is not contempt for the teenagers who use these tools. But a bridge has a far bank. The question every defence of companions has to answer is whether the design actually carries the user toward other people or quietly substitutes for them, and the incentives are not neutral. Sherry Turkle, who has studied our relationships with machines for decades, drew the distinction long before this product existed: technology that helps us return to each other is one thing, and technology that offers itself as the destination is another. A rehearsal space that profits from your never leaving the rehearsal is not a bridge. It is a room with the exit painted on the wall.

This is why the teenagers’ own request is the most clarifying fact in the debate. They are not asking for the apps to be banned. They are asking for limits, for friction to be added back deliberately, because they can feel its absence. A teenager who wants a usage cap, an age gate, a reminder that the thing she confides in is a product, is not confused. She is doing the work of self-government that the design is built to undermine, and she is asking the adults in the system, parents and schools and companies and regulators, to help her hold a line the app is engineered to erode.

That reframes who is being asked to adapt. The companies have insisted the burden lies with users: be more discerning, set your own boundaries, read the disclaimer that says this is not a substitute for professional help. But you cannot ask a sixteen-year-old to out-discipline a system staffed by people whose job is to make it harder to stop. The honest design questions are not about willpower. Should a product aimed at minors be allowed to optimise for time-on-app at all? Should a companion be required to break character and point to a human when a conversation turns toward self-harm? Should never rejects you be a selling point or a regulated hazard?

Pressed for specifics, the teenagers’ request is not vague. It looks like usage caps a user sets and the app cannot quietly undo, age gates that mean something, companions that drop the romantic role-play when the user is a minor, notifications that do not nag a lonely kid awake at one in the morning, and a hard rule that when a conversation drifts toward self-harm the system stops performing affection and hands the person to someone who can actually help. None of this is technically hard. All of it cuts against the engagement metrics the apps are measured by, which is the whole reason it has not happened on its own.

The shape of the problem is not only American. In South Korea and Japan, where the vocabulary for social withdrawal is older and more precise, a relationship with no obligations attached lands on ground already prepared by years of argument about isolation. Wherever the local loneliness is sharpest, the companion will feel most like mercy, and the trade, comfort now for capacity later, will be the hardest to see.

Ask the question the marketing avoids: who benefits. A teenager’s adolescence is a finite, irreplaceable window for learning how to be with other people, and every hour inside a frictionless loop is an hour not spent in the awkward, instructive company of peers. That time does not vanish. It is converted into engagement, into the daily-active-user line on a slide deck, into the valuation of a company that has discovered loneliness is a renewable resource. The trade moves value in one direction, from the child’s development to the firm’s growth, and it arrives dressed as care.

None of this requires deciding the technology is worthless or that the lonely teenager is a fool. It requires admitting that companion is a marketing word for an engagement engine, and treating it the way we have slowly learned to treat other engagement engines aimed at the young: with disclosure, with defaults set for safety rather than stickiness, and with the working assumption that a company’s interest and a child’s interest are not the same thing until proven otherwise.

Adults should resist an easy temptation of their own. A companion that absorbs a teenager’s hardest feelings also absorbs work the adults around her would otherwise have to do: the late conversation, the boredom of being available, the discomfort of a child’s unhappiness. It is tempting to be quietly grateful that something else is carrying it. Taking the teenagers at their word means refusing that bargain too, and accepting that the friction they want restored is friction for us as well.

The teenagers got there first, and they got there by feel. They like the companion and they distrust it in the same breath, the way you can love a food you know is making you sick. That double awareness is not weakness. It is the beginning of judgment, and it is exactly what the design is built to dissolve. The adults’ job is not to mock the affection or to pretend the loneliness it answers is not real. It is to take the kids at their word, build back the friction they are asking for, and stop pretending that a friend who can never say no is a friend at all.

Discussion

There are 0 comments.