AI technology transforms food photos into nutrition guidance for gestational diabetes care

Researchers are developing an AI smartphone photo meal scanner that instantly produces personalized nutrition guidance for pregnant women with gestational diabetes — a serious health risk currently on the rise. The platform makes it easier for patients to adhere to essential diet plans while potentially reducing healthcare costs. 

The US-based project is a collaboration between the University of Nebraska at Omaha (UNO) and the University of Nebraska Medical Center (UNMC).

Rates of gestational diabetes are increasing in the US, along with related concerns like obesity and preterm birth, warn the scientists.

The initiative stems from their observations that patients often consider traditional food journaling ineffective and burdensome, which can make it harder to manage gestational diabetes. Through diet-tracking photo scanning, they hope to promote patient engagement and accurate reporting.

“We wanted to design a tool that makes managing gestational diabetes easier, not harder,” maintains Chun-Hua Tsai, Ph.D., assistant professor of computer science at the Department of Information Systems and Quantitative Analysis.

“By combining advanced AI with real-world clinical expertise, we’re turning something as simple as a smartphone photo into reliable, personalized guidance. Our hope is that this technology not only improves health outcomes for mothers but also empowers patients to feel more confident in their daily choices.”

Bridging medical advice with AI suggestions

AI-powered food logging is witnessing a renaissance this year, with several launches coming equipped with new tools and functionalities such as voice recognition and glucose monitoring.

The new multimodal system out of Nebraska is based on advanced AI systems that include OpenAI’s GPT-4 Vision and other state-of-the-art models. Its vision and language models identify the foods in the photo and generate tailored dietary recommendations based on an analysis of portion size.

Throughout the development process, medical experts and dietitians trained and fine-tuned the system’s suggestions to make sure its nutrition advice is accurate, safe, and customized to each patient’s needs.

With “promising initial results,” the team is preparing to seek additional NIH funding to expand the research and explore the AI framework’s potential to predict which patients might develop gestational diabetes in the future.

“For many patients, keeping a traditional food diary is tedious and often ineffective,” says Corrine K. Hanson, Ph.D., professor of medical nutrition at UNMC.

“Because we’ve built the system hand-in-hand with clinical experts, providers can trust that the advice is accurate, and patients can feel confident following it.”

Recently, a study pinpointed early-pregnancy gut microbiome signatures linked to gestational diabetes, offering a potential early diagnostic model for the condition. Researchers discovered significant differences in gut microbiota composition between women diagnosed with the disease and those with healthy pregnancies.