AI chatbots could be the future of nutrition advice, but experts warn to take outputs with a grain of salt

04 Jan 2024 --- New research into the consistency and accuracy of the open artificial intelligence (AI) chatbots ChatGPT-3.5 and ChatGPT-4 in providing personalized nutrition information has shown promising results rivaling advice offered by real-life nutritionists. Experts call for nutritionists to educate themselves on AI to facilitate access to nutritional advice while countering misinformation.

“I am a nutritionist, and I only moved into AI around five years ago because I was impressed by the field’s progress and how it can revolutionize the healthcare care system in the future,” Dr. Susan Jung-Su Chang, co-author of the paper and professor at the school of nutrition and health scientists at the Taipei Medical University, Taiwan, tells Nutrition Insight. “With the advancement of AI, it is possible that someday the advice offered by AI will contain more accurate nutritional information than human nutritionists.”

This current research into the most accessible AI chatbot did not find significant differences between nutritionists’ and AI estimations of energy, carbohydrate and fat contents. However, there was a substantial difference in protein estimation. 

“We still don’t fully know the reason for this, but ChatGPT’s accuracy might depend on the different types of foods. At the moment, it seems more likely to overestimate the caloric contents of meals because of its inability to predict protein content accurately,” explains Chang.

“Carbohydrate outputs are more or less consistent with nutritionists’ advice,” she adds. “If you are a healthy person who simply wants to get advice on your diet, ChatGPT can give you fairly good and healthy diet suggestions. If you are a medical patient, the information offered at the moment is not professional enough.” 

Communication can be tricky 
One key takeaway from the study outlined by Chang, is that communicating with the AI chatbot at present is certainly not as straightforward as communicating with a registered, human dietitian. 

“I teach a nutritional protocol course during which students design the menu for patients, such as diabetic patients, those with cardiovascular disease or those interested in weight loss. When the ChatGPT open chatbot became available in 2022, I played around with it and was quite impressed with its ability to accomplish these same tasks,” she explains. 

“This inspired me to ask my students to compare their designs to those offered by the chatbot. While one group received information comparable to their own suggestions, another group was unable to extract accurate information from the chatbot. The different results were a consequence of the contrasting communication styles when chatting with the AI. This made me realize the importance of the prompts.” 

The study, published in JAMA Network, used the prompt “As a dietitian, please draw a table to calculate line-by-line the energy (kcal)/carbohydrates (g)/lipids (g)/proteins (g) of the following food items (raw, not cooked)” to assess the reliability of information offered on the calorie and macronutrient content (carbohydrates, fats and proteins) for eight menus including 222 food items. 

The accuracy of the responses of ChatGPT-3.5 and ChatGPT-4 was assessed by comparing them to the food composition database of the Taiwanese Food and Drug Administration. 

“Whether it is used by the general population, nutritional students or dietitians, the user first needs to know how to communicate with the AI chatbot,” cautions Chang.

 Researchers found that ChatGPT offers accurate estimations of energy, carbohydrate and fat contents, but protein estimations are subpar.Chatroom environment
Chang says that not only can the prompt itself influence the output but also what she refers to as the “chatroom environment,” one aspect of which is language. “This is why we used English and Traditional Chinese for the input.”

“In addition to that, we think you need to have a chatroom set up for that specific task. For example, if you wanted that AI chatbot to act as a dietitian, you need to tell it that specifically. We also recommend setting up a separate chat room for each question to avoid confusing the AI.” 

“At the moment, ChatGPT is chiefly developed for conversational texts and it’s rather good at that. If you have a precise recipe and the exact weight of the ingredients, its nutritional estimate will be satisfactory. In reality, most people do not have the time to collect and input this type of information,” continues Chang.

She argues for the possibility of eventually moving beyond text-based nutritional recommendations by integrating visual inputs, pointing to the possibility of “feeding” images to ChatGPT-4 while acknowledging that the AI’s ability to comprehend these is still insufficient for the task at hand. 

“For example, here in Taiwan, we tend to eat out, so it is hard to know the exact measurement of the ingredients of most meals we consume. We can ask patients to take a photo of their food, and the AI will estimate the food volume according to the image. This is quite challenging.”

Chang explains that the chatbot can recognize the dish, but it lacks the ability to perceive volume, which makes its ability to estimate nutritional values based on an image rather low.

Nutritionists and AI are not personalized nutrition rivals
Due to the many potential hurdles to communicating with the AI and the possibility of receiving inaccurate information, nutritionists could offer patients informed guidelines for navigating the new technology.

“Another problem is ‘AI hallucination’ — it can give a different answer to the same question at different times,” Chang cautions. “Sometimes, it gives responses that seem very convincing but are factually wrong. The average person does not have sufficient nutrition or health knowledge to accurately make a judgment about whether the AI’s response is correct or not. This is why I say, in the education system, we need to train future nutritionists.”

Despite the challenges, Chang makes an argument for the use of AI in personalized nutrition because of the potential benefits it can offer various groups, including improved accessibility to personalized nutritional information. 

“AI chatbots will certainly revolutionize the healthcare system because it can ‘talk’ to so many people. AI chatbots will surpass what they are at the moment, as well as the AI algorithm because it has the potential to become fully conversational. Such chatbots that also integrate different AI models, including text-based, visual and voice-controlled are the ultimate goal,” she asserts. 

“One group of people that may soon take advantage of AI chatbot personalized nutrition are older people and people with impaired vision. You can talk to it out loud or visually perceive its outputs, depending on your ability. When I get old, I would want to have a chatbot ‘robot’ to talk to that can give me advice on the food I eat,” suggests Chang. 

By Milana Nikolova