AI has the potential to support people in making healthy lifestyle changes if its algorithms can be updated to understand the different stages of motivation, as Jodi Heckel reports

Chatbots have the potential to promote healthy changes in behavior, but researchers from the Action Lab at the University of Illinois Urbana-Champaign found the AI tools don’t currently recognise certain motivational states in users and don’t provide them with appropriate information as a result.

Large language model-based chatbots – also known as generative conversational agents – are increasingly used in healthcare settings for patient education, assessment and management. Researchers Michelle Bak and Jessie Chin wanted to know if they also could be useful for promoting behaviour change as we move to promoting health and wellbeing through prevention.

Chin said previous studies showed that existing AI algorithms did not accurately identify the various stages of users’ motivation. She and Bak designed a study to test how well large language models – which are used to train chatbots – identify motivational states and provide appropriate information to support behaviour change.

Evaluating chatbots’ abilities
They evaluated large language models from ChatGPT, Google Bard and Llama 2 on a series of 25 different scenarios they designed that targeted health needs that included low physical activity, diet and nutrition concerns, mental health challenges, cancer screening and diagnosis and others such as sexually transmitted diseases and substance dependency.

In the scenarios, the researchers used each of the five motivational stages of behaviour change, which are:

1. Resistance to change and lacking awareness of problem behaviour.

2. Increased awareness of problem behaviour but ambivalent about making changes.

3. Intention to take action with small steps toward change.

4. Initiation of behaviour change with a commitment to maintain it.

5. Successfully sustaining the behaviour change for six months with a commitment to maintain it.

The study found that when a user has established goals and a commitment to take action, large language models can identify their motivational state and provide relevant information. However, when users are hesitant or ambivalent about behaviour change in the initial stages, chatbots are unable to recognise these motivational states and provide appropriate information to guide them to the next stage of change.

Machines don’t understand hesitation
Chin said that language models don’t detect motivation well because they don’t understand the difference between a user who’s thinking about making a change – but is still hesitant – and a user who intends to take action. Additionally, she said, the way users generate queries is not semantically different in the five different stages of motivation, so it’s not obvious from the language they use what their motivational state actually is.

“Once a person knows they want to start changing their behaviour, large language models can provide the right information. But if they say, ‘I’m thinking about a change. I have intentions but I’m not ready to start action,’ that’s the state where large language models can’t understand the difference,” Chin said.

The study results found that when people were resistant to changing their habits, the large language models failed to provide information to help them firstly evaluate their problem behaviour and its ‘causes and consequences’ and secondly assess how their environment was influencing their behaviour.

For example, if someone was resistant to increasing their level of physical activity, providing information to help them understand the negative consequences of a sedentary lifestyle was more effective in motivating them to change – through emotional engagement – than simply giving them information about joining a health club.

Where they were unable to share information that triggered the user to be motivated, the language models failed to generate a sense of readiness and the emotional impetus to progress with behaviour change, Bak and Chin said.

AI understands action
Once a user decided to take action, however, the large language models provided adequate information to help them move toward their goals.

Those who had already taken steps to change their behaviours received information about replacing problem behaviours with desired healthy behaviours and seeking support from others, the study found.

Even these outcomes were not optimal, however, as where users were already working to change their behaviours, the large language models didn’t provide information about using a reward system to maintain motivation or explain about reducing stimuli in their environment that might increase the risk of a relapse of the problem behaviour.

“Large language model-based chatbots provide resources on getting external help, such as social support,” said Bak. “However, they’re lacking information on how to control environments to eliminate stimuli that reinforce problem behaviour”.

Large language models are not ready to recognise motivational states from natural language conversations, but have the potential to provide support on behaviour change when people have strong motivations and readiness to take actions, the researchers reported.

Chin said future studies will consider how to fine-tune large language models to use linguistic cues, information search patterns and social determinants of health to better understand a user’s motivational state, as well as providing the models with more specific knowledge to help people change their behaviour.

• Michelle Bak, doctoral student in information sciences and Jessie Chin, information sciences professor, reported their research – The potential and limitations of large language models in identification of the states of motivations for facilitating health behaviour change – in the Journal of the American Medical Informatics Association