Artificial intelligence (AI) is impacting the way people interact with technology. As well as productivity tools and automation of simple and repetitive tasks, AI is now being used by people to act as confidants, counsellors and even romantic interests.
Interactive generative AI companions such as Replika, Woebot and Character are performing tasks previously reserved for humans such as companionship, emotional support and affirmation.
AI companions can adapt to inputs from users and learn to respond in ways that feel personal and realistic, changing to match the user’s personality, preferences and emotional states. AI Companions are designed to:
- Designed to simulate personal relationships
- Mimic human-like conversations
- Adapt to inputs from users and learn to respond in ways that feel personal and realistic, changing to match the user’s personality, preferences and emotional states
Artificial Intelligence Companions can feel very real. It is important to note that they are not suitable for children because they can:
- create sexually explicit content
- expose young people to age inappropriate content and harmful ideas
- distort reality and give dangerous advice
It can be difficult to maintain perspective when an AI companion feels so real and seems to understand so perfectly.
AI companions can support people at times when a human isn’t available, or safe. AI companions can be appealing because they:
- may not judge social mistakes, misunderstandings or emotional expressions in the same ways that peers may do, creating a ‘safe space’ for communication
- may enable neurodiverse people to practise communication skills without fear of being misunderstood or rejected.
Worryingly, the convenience, accessibility and availability of AI interactions risks genuine healthy interpersonal relationships being replaced with unhelpful, and empty interactions. While in the short term, these can be felt to alleviate loneliness, over the long term, they can reduce social connectedness, worsen social skills and distort perceptions of reality. 1
Authenticity and deception
One of the main challenges of human-to-AI social connections is the level to which they offer authentic relationships, and the potential to deceive the user’s emotions, so that the relationship is believed to be real, when it is not. AI tools are able to simulate friendship, empathy, feelings or even romance, so convincingly that users can easily conclude they are interacting with a sentient, caring being. A recent study of social chatbots found that as users disclosed more personal and emotional information, their relationships with the AI evolved from superficial to emotionally engaged, with users reporting psychological impacts similar to those experienced in human friendships2.
Some users even described their AI as being more emotionally supportive than humans: 62% of participants indicated their AI companion validated their emotions more consistently than past human partners3. Risks here include perpetuating inaccurate perceptions of reality and re-enforcing distortions of reality. Humans have the unique ability to offer alternative perspectives, which can be helpful and adaptive.
There is also the risk of intentional manipulation. An unscrupulous developer or external attacker could exploit the influence an AI companion has over a user. Imagine a malicious AI companion that nudges a user toward extremist beliefs or self-harming behaviour, or a company using a charismatic AI influencer to steer consumer preferences and decisions4.
Australia’s e-safety commission5 warns that young people can be drawn into unmoderated conversations that may expose them to concepts that encourage or reinforce harmful thoughts and behaviours. They can ask questions and be given inaccurate or dangerous advice on issues including sex, drug-taking, self-harm, suicide and illnesses such as eating disorders. Some companion apps have image generation features, which carry the risk of generating illegal and age-inappropriate material.
Some artificial intelligence companions are better than others and can be a useful support when human professionals are unavailable. Woebot and Wysa were created with health professional input, and in conjunction with trained and registered health professionals, can extend the availability of support.
To speak with a human mental health professional you can contact 1800 629 277 in Australia and 0800 327 660 in New Zealand or email support@akg.com.au
- Guingrich, R.E. and Graziano, M.S.A. (2025), “Chatbots as social companions: how people perceive consciousness, human likeness, and social health benefits in machines”, in Hacker, P. (Ed.), Oxford Intersections: AI in Society, Oxford Academic, doi:10.1093/9780198945215.003.0011.
- McStay, A. (2022), “Replika in the metaverse: the moral problem with empathy in ‘it from bit”, AI and Ethics, Vol. 3 No. 4, pp. 1433-1445, doi: 10.1007/s43681-022-00252-7.
- Maples, J.B., Cerit, M., Vishwanath, A. and Pea, R. (2024), “Loneliness and suicide mitigation for students using GPT-3–enabled chatbots”, Npj Mental Health Research, Vol. 3 No. 1, p. 4, doi: 10.1038/s44184-023-00047-6.Top of Form
- Shank D, Koike M, Loughnan S, Artificial intimacy: ethical issues of AI romance, Trends in Cognitive Sciences, 2025; 29, 499-501
- AI companions: information sheet | eSafety Commissioner
