Do you talk to AI when you’re feeling down? Here’s where chatbots get their therapy advice

By Centaine Snoswell, Aaron J Snoswell and Laura Neil for 'The Conversation'

Brisbane: As more and more people spend time chatting with artificial intelligence (AI) chatbots such as ChatGPT, the topic of mental health has naturally emerged. Some people have positive experiences that make AI seem like a low-cost therapist.

But AIs aren’t therapists. They’re smart and engaging, but they don’t think like humans. ChatGPT and other generative AI models are like your phone’s auto-complete text feature on steroids. They have learned to converse by reading text scraped from the internet.

When someone asks a question (called a prompt) such as “how can I stay calm during a stressful work meeting?” the AI forms a response by randomly choosing words that are as close as possible to the data it saw during training. This happens so fast, with responses that are so relevant, it can feel like talking to a person.

But these models aren’t people. And they definitely are not trained mental health professionals who work under professional guidelines, adhere to a code of ethics, or hold professional registration.

Where does it learn to talk about this stuff? When you prompt an AI system such as ChatGPT, it draws information from three main sources to respond: background knowledge it memorised during training external information sources information you previously provided.

1. Background knowledge

To develop an AI language model, the developers teach the model by having it read vast quantities of data in a process called “training”.

Where does this information come from? Broadly speaking, anything that can be publicly scraped from the internet. This can include everything from academic papers, eBooks, reports, free news articles, through to blogs, YouTube transcripts, or comments from discussion forums such as Reddit.

Are these sources reliable places to find mental health advice? Sometimes. Are they always in your best interest and filtered through a scientific evidence based approach? Not always. The information is also captured at a single point in time when the AI is built, so may be out-of-date.

A lot of detail also needs to be discarded to squish it into the AI’s “memory”. This is part of why AI models are prone to hallucination and getting details wrong.

2. External information sources

The AI developers might connect the chatbot itself with external tools, or knowledge sources, such as Google for searches or a curated database.

When you ask Microsoft’s Bing Copilot a question and you see numbered references in the answer, this indicates the AI has relied on an external search to get updated information in addition to what is stored in its memory.

Meanwhile, some dedicated mental health chatbots are able to access therapy guides and materials to help direct conversations along helpful lines.

3. Information previously provided

AI platforms also have access to information you have previously supplied in conversations, or when signing up to the platform.

When you register for the companion AI platform Replika, for example, it learns your name, pronouns, age, preferred companion appearance and gender, IP address and location, the kind of device you are using, and more (as well as your credit card details).

On many chatbot platforms, anything you’ve ever said to an AI companion might be stored away for future reference. All of these details can be dredged up and referenced when an AI responds.

And we know these AI systems are like friends who affirm what you say (a problem known as sycophancy) and steer conversation back to interests you have already discussed. This is unlike a professional therapist who can draw from training and experience to help challenge or redirect your thinking where needed.

What about specific apps for mental health? Most people would be familiar with the big models such as OpenAI’s ChatGPT, Google’s Gemini, or Microsofts’ Copilot. These are general purpose models. They are not limited to specific topics or trained to answer any specific questions.

But developers can make specialised AIs that are trained to discuss specific topics, like mental health, such as Woebot and Wysa.

Some studies show these mental health specific chatbots might be able to reduce users’ anxiety and depression symptoms. Or that they can improve therapy techniques such as journalling, by providing guidance. There is also some evidence that AI-therapy and professional therapy deliver some equivalent mental health outcomes in the short term.

However, these studies have all examined short-term use. We do not yet know what impacts excessive or long-term chatbot use has on mental health. Many studies also exclude participants who are suicidal or who have a severe psychotic disorder. And many studies are funded by the developers of the same chatbots, so the research may be biased.

Researchers are also identifying potential harms and mental health risks. The companion chat platform Character.ai, for example, has been implicated in ongoing legal case over a user suicide.

This evidence all suggests AI chatbots may be an option to fill gaps where there is a shortage in mental health professionals, assist with referrals, or at least provide interim support between appointments or to support people on waitlists.

Bottom line At this stage, it’s hard to say whether AI chatbots are reliable and safe enough to use as a stand-alone therapy option.

More research is needed to identify if certain types of users are more at risk of the harms that AI chatbots might bring.

It’s also unclear if we need to be worried about emotional dependence, unhealthy attachment, worsening loneliness, or intensive use.

AI chatbots may be a useful place to start when you’re having a bad day and just need a chat. But when the bad days continue to happen, it’s time to talk to a professional as well.

(Centaine Snoswell is a Senior Research Fellow, Centre for Health Services Research, The University of Queensland; Aaron J Snoswell is a Senior Research Fellow in AI Accountability, Queensland University of Technology; Laura Neil is a PhD Candidate, Centre for Health Services Research, The University of Queensland)